Decision Criteria for Determining Unidimensionality

Abstract

<p>One of the fundamental assumptions of measurement theory is that a set of its items forming a test is unidimensional. The purposes of this dissertation were (1) to review various methods for determining unidimensionality and to assess the rationale of those methods; (2) to attempt to clarify the term unidimensionality, and to show how it differs from other terms often used interchangeably; and (3) to assess the effectiveness of various indices proposed to determine unidimensionality.</p> <p>Indices based on answer patterns, reliability, component and factor analysis, and latent traits were reviewed and it was shown that many of these lacked a rationale, that for many the sampling distributions were not known, and that many were adjustments to an established index to take into account some criticism of it. Altogether 87 indices were reviewed.</p> <p>It was demonstrated that unidimensionality often is used interchangeably with reliability, internal consistency, and homogeneity. Reliability was defined as the ratio of the true score variance to obsesrved score variance. Internal consistency has been used often as a synonym for unidimensionality, and it also denotes a group of methods that are intended to estimate reliability. Internal consistency methods are based on the variances and covariances of test-items, and depend on only one administration of a test. Homogeneity seems to refer more specifically to the similarity of the item correlations, but the term is often used as a synonym for unidimensionality. Unidimensionality was defined as the existence of one latent trait underlying the data. The usefulness of the terms internal consistency and homogeneity was questioned.</p> <p>A Monte Carlo simulation was conducted to assess the 87 indices under known conditions. A three-parameter, multivariate, logistic latent-trait model was used to generate item responses. Difficulty, guessing, discrimination, and the number of factors underlying the data were varied.</p> <p>Many of the indices were highly correlated, some resulted in estimates outside their theoretical bounds, and most were particularly sensitive to the intercorrelations between the factors. Indices based on answer patterns, reliability, component analysis, linear factor analysis, and one the one-parameter latent trait model were ineffective. The sums of absolute residuals from a nonlinear factor analysis (specifying one factor with cubic terms) and from two-parameter latent trait models (Christoffersson, 1975; McDonald, 1980; Muthen, 1978) were able to discriminate between cases with one latent trait and cases with more than one latent trait.</p>