In many testing situations, ordering the items by difficulty is helpful in analyzing the testing data; examples include intelligence testing, analysis of differential item functioning, person-fit analysis, and exploring hypotheses about the order in which cognitive operations are acquired by children. In each situation, interpretation and analysis is made easier if the items are ordered by difficulty in the same way for every individual taking the test, i.e. the item response functions do not cross. This is an invariant item ordering.
In this paper we review a class of nonparametric unidimensional item response models in which the ordinal properties of items (and persons) can be studied, and survey both old and new methods for the investigaton of invariant item ordering in empirical data sets. Our model formulation derives in particular from the work of Mokken (1971), Holland Rosenbaum (1986) and Junker (1993). We survey methods based on the work of Mokken (1971), Rosenbaum (1987a, 1987b), and Sijtsma Meijer (1992), and we also discuss some new proposals for checking invariant item ordering. When violations are detected, these methods allow a rough assessment of where on the latent scale the item response functions cross.
We also study similarities and differences between these various methods and provide guidelines for their use. Finally, the methods are illustrated with data from a developmental psychology experiment in which the ability to draw inferences about transitive relations is explored.