A Survey of Theory and Methods of Invariant Item Ordering

Klaas Sijtsma
Vakgroep Methodenleer en Statistiek
Faculteit der Sociale Wetenschappen
Universiteit Utrecht
P.O. Box 80.140
3508 TC Utrecht
The Netherlands
sijtsma@fsw.ruu.nl

Brian W. Junker
Department of Statistics
Carnegie Mellon University
Pittsburgh PA 15213
brian@stat.cmu.edu

May 29, 1994

Abstract:

In many testing situations, ordering the items by difficulty is helpful in analyzing the testing data; examples include intelligence testing, analysis of differential item functioning, person-fit analysis, and exploring hypotheses about the order in which cognitive operations are acquired by children. In each situation, interpretation and analysis is made easier if the items are ordered by difficulty in the same way for every individual taking the test, i.e. the item response functions do not cross. This is an invariant item ordering.
In this paper we review a class of nonparametric unidimensional item response models in which the ordinal properties of items (and persons) can be studied, and survey both old and new methods for the investigaton of invariant item ordering in empirical data sets. Our model formulation derives in particular from the work of Mokken (1971), Holland Rosenbaum (1986) and Junker (1993). We survey methods based on the work of Mokken (1971), Rosenbaum (1987a, 1987b), and Sijtsma Meijer (1992), and we also discuss some new proposals for checking invariant item ordering. When violations are detected, these methods allow a rough assessment of where on the latent scale the item response functions cross.
We also study similarities and differences between these various methods and provide guidelines for their use. Finally, the methods are illustrated with data from a developmental psychology experiment in which the ability to draw inferences about transitive relations is explored.

Here is the full postscript text for this technical report. It is 294 kbytes.