Analyzing small psychological experiments with item response models

Richard J. Patz, Brian W. Junker, F. Javier Lerch and Brian R. Huguenard


For discrete repeated measures data, item response theory (IRT) models and Rasch-type models in particular provide a direct way to model within-subjects dependence. Experimental repeated measures data, commonly gathered by psychologists and educational researchers, exhibit within-subjects dependence, and yet sample sizes are often too small to allow stable fitting using standard IRT programs. In this paper we discuss the application of IRT models in this small-scale experimental context. We show that when attention is focused on a small number of hypotheses, IRT models may be adapted to evaluate the hypotheses while accounting for within-subjects, between-measures dependence. We illustrate our approach by analyzing polytomous response data from a small experiment in cognitive psychology that examines the stresses that telephone-based menu systems place on human working memory. We develop a polytomous Rasch model, and we generalize Rigdon and Tsutakawa's (1983) E-M algorithm along the lines of Glas and Verhelst (1989) to obtain a stable model fit. We discuss hypothesis evaluation and important sensitivity issues, and we compare different strategies for efficient numerical computation.

Index terms: cognitive psychology, computational issues, E-M algorithm, item response models, maximum likelihood, polytomous response data, small samples

Here is the full postscript text for this technical report. It is 488239 bytes long.