Department of Statistics Unitmark
Dietrich College of Humanities and Social Sciences

Analyzing small psychological experiments with item response models

Publication Date

February, 1996

Publication Type

Tech Report

Author(s)

Richard J. Patz, Brian W. Junker, F. Javier Lerch and Brian R. Huguenard

Abstract

For discrete repeated measures data, item response theory (IRT) models and Rasch-type models in particular provide a direct way to model within-subjects dependence. Experimental repeated measures data, commonly gathered by psychologists and educational researchers, exhibit within-subjects dependence, and yet sample sizes are often too small to allow stable fitting using standard IRT programs. In this paper we discuss the application of IRT models in this small-scale experimental context. We show that when attention is focused on a small number of hypotheses, IRT models may be adapted to evaluate the hypotheses while accounting for within-subjects, between-measures dependence. We illustrate our approach by analyzing polytomous response data from a small experiment in cognitive psychology that examines the stresses that telephone-based menu systems place on human working memory. We develop a polytomous Rasch model, and we generalize Rigdon and Tsutakawa's (1983) E-M algorithm along the lines of Glas and Verhelst (1989) to obtain a stable model fit. We discuss hypothesis evaluation and important sensitivity issues, and we compare different strategies for efficient numerical computation.