On Bayesian Analysis of Multirater Ordinal Data: An Application to Automated Essay Grading
作者:
ValenE. Johnson,
期刊:
Journal of the American Statistical Association
(Taylor Available online 1996)
卷期:
Volume 91,
issue 433
页码: 42-51
ISSN:0162-1459
年代: 1996
DOI:10.1080/01621459.1996.10476662
出版商: Taylor & Francis Group
关键词: Bayesian inference;Categorical data;Gibbs sampling;Hierarchical models;Latent structure models;Markov chain Monte Carlo
数据来源: Taylor
摘要:
A framework is proposed for the analysis of ordinal categorical data when ratings from several judges are available. I emphasize the tasks of estimating latent trait characteristics of individual items, regressing these latent traits on observed covariates, and comparing the performance of raters. The model is illustrated in the design and evaluation of an automated essay grader. This grader is based on a regression of variables, obtained from a grammar checker, on essay scores estimated from a panel of experts. The performance of the grader is evaluated relative to human graders, and implications on the reliability and repeatability of both automated and human raters is investigated.
点击下载:
PDF (1470KB)
返 回