A Web-based program for coding open-ended response protocols |
| |
Authors: | Email author" target="_blank">S?L?AmesEmail author P?E?Gallaher P?Sun S?Pearce J?B?Zogg B?R?Houska B?C?Leigh A?W?Stacy |
| |
Institution: | (1) Language Technologies Institute, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA;(2) University of Munich, Munich, Germany |
| |
Abstract: | A Web-based coding application was designed to improve coding efficiency and to provide a systematic means of evaluating responses
to open-ended assessments. The system was developed for use by multiple raters to assign open-ended responses to predetermined
categories. The application provides a software environment for efficiently supervising the work of coders and evaluating
the quality of the coding by (1) systematically presenting open-ended responses to coders, (2) tracking each coder’s categorized
responses, and (3) assessing interrater consistency at any time in order to identify coders in need of further training. In
addition, the application can be set to automatically assign repeated responses to categories previously identified as appropriate
for those responses. To evaluate the efficacy of the coding application and to determine the statistical reliability of coding
open-ended data within this application, we examined data from two empirical studies. The results demonstrated substantial
interrater agreement on items assigned to various categories across free and controlled association tasks. Overall, this new
coding application provides a feasible method of reliably coding open-ended data and makes the task of coding these data more
manageable. |
| |
Keywords: | |
本文献已被 PubMed SpringerLink 等数据库收录! |
|