Journal of Consumer Research Inc.
Modeling Distance Structures in Consumer Research: Scale Versus Order in Validity Assessment Author(s): Daniel R. Denison and Claes Fornell Source: The Journal of Consumer Research, Vol. 16, No. 4 (Mar., 1990), pp. 479-489 Published by: The University of Chicago Press Stable URL: http://www.jstor.org/stable/2489459 . Accessed: 18/02/2011 00:04
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .
http://www.jstor.org/action/showPublisher?publisherCode=ucpress. . Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact email@example.com.
The University of Chicago Press and Journal of Consumer Research Inc. are collaborating with JSTOR to digitize, preserve and extend access to The Journal of Consumer Research.
Modeling Research: Validity
Distance Scale Assessment
Structures Versus Order
DANIELR. DENISON CLAES FORNELL*
Confirmatory multidimensional scaling (CMDS) is presented as a spatial technique for structural modeling based on ordinal assumptions, and as an alternative to metric techniques such as LISREL. The article links both techniques to the multitraitmultimethod matrix and presents a system for deriving measures of symmetric construct relationships, measurement error, and goodness of fit. Examples show that CMDS and
LISRELoften give comparable results, but that LISRELis sensitive to the magnitude of correlations whereas CMDS is sensitive only to their order. The trade-offs involved in assumptions, results, and interpretations with these methods are highlighted in the conclusion.
the assessment of convergent and discriminant validity in consumer research, one of the dominant approaches for the past 25 years has been Campbelland Fiske's (1959) multitrait-multimethod matrix (MTMM).l Since its inception, many methods have been suggested for its analysis (Levin, Montag, and Comrey 1983; Schmitt and Stults 1986). However, because of the ambiguous rules for assessingthe various validity criteria, researchers have often turned to techniques such as analysis of variance (Stanley 1961), factor analysis (Jackson 1975), principal component analysis (Golding and Seidman 1974) and various types of causal modeling (Werts and Linn 1970), including covariance structureanalysis via LISREL(Bagozzi 1980). Using these more powerful methods of data analysis has overshadoweda critical property of the original MTMM approach-the ordinal nature of the analysis. For example, the basic MTMM criteria for discriminant validity are expressed in ordinal terms; each indicator of a given trait must be more highly correlatedwith other indicators of that trait than with the indicators of another trait (particularlyif the indicators share a common method). In contrast, metric
*Daniel R. Denison is AssistantProfessorof
OrganizationalBehaviorand Human ResourceManagementand ClaesFornellis the Donald C. Cook Professorof BusinessAdministration,both at the GraduateSchool of BusinessAdministration,University of Michigan, Ann Arbor, MI 48109. The authors wish to thank Robert Hooijbergand Aneil Mishrafor their contributionsto this article, and James C. Lingoes, Wayne DeSarbo, and the anonymous JCR reviewersfor their comments on earlierversions. 479
methods such as factor analysis or covariance structure analysis move well beyond ordinal criteria and specify MTMM criteriain parametricterms. This article discusses confirmatory multidimensional scaling (CMDS) as a structuralmodeling technique based on ordinal assumptions about the level of measurement, ratherthan the metric assumptions of covariance structure analysis. We add to earlier work on confirmatory multidimensional scaling (Denison 1982; Fornell and Denison 1981, 1982) by presenting(1) a new system for
placingmeasurement and theory constraints on multidimensional scaling (MDS) solutions, (2) a new method for estimating constructs, symmetric construct relationships, and measurement error,and (3) a new approachto assess goodness of fit. It is suggested that CMDS can be both an alternative and a complement to covariance structureanalysis. Both
methods deal with multiple measures,multiple constructs, and the incorporation of a priori theory- and measurement-basedconstraintsin a solution. We also suggest that nonmetric methods are, in many ways, closer to the original MTMM formulation than is covariance structureanalysis. After a presentation of the CMDS approach to structuralmodeling, our approach is illustrated with two examples. These results are compared to parallel analyses using LISREL VI (Joreskog and Sorbom 1984). The subsequent discussion compares the assumptions, parameter estimations, and interpre'See the review by Peter (198 1). ? JOURNAL CONSUMER OF * RESEARCH Vol. 160 March1990
THE JOURNAL OF CONSUMER RESEARCH
tations of the two methods, and highlights the inherent trade-offspresented by structural modeling with CMDS.
discriminant validity implies that constructs can be distinguished from one another through their measures.
CMDS AND THE MULTITRAITMULTIMETHOD MATRIX
The interpretationof a CMDS solution follows the MTMM logic very closely. Measures of proximity (such as correlations) are represented as distances such that variableswith the highest proximity appear closest together in the solution. The clusters of variables closest togetherare treatedas multiple measures of the same construct. Adjacent clusters of variables are interpreted as related constructs. Thus, the basic MTMM logic necessary to distinguish convergent and discriminantvalidity can easily be translatedinto a multidimensional scaling model. Imposing theoretical constraints on an MDS solution via CMDS makes the MTM-M logic more explicit by providingan additional set of theoretically derived distance constraints, defined in MTMM terms, that must be satisfiedby the solution. Thus, the final
confirmatorysolution must represent not only the original correlations or proximity measures, but also the constraints implied by convergent, discriminant, and nomological validity. One fundamental difference between factor analysis and multidimensional scaling'must be acknowledged at the outset: in multidimensional scaling, constructs are primarily represented by clusters of variables, rather than by underlying dimensions as in factor analysis. Although the underlying dimensions themselves may have meaning in MDS, they also serve to translate the proximity data into distances. Once dimensionality has been established, assessment of validity is based upon an analysis of the distances between points, ratherthan the dimensions in which distances are displayed. Clusters of variables, as well as dimensions, representconstructs. There are several computer algorithms now available for CMDS. The analyses presented here use CMDA (Borgand Lingoes 1980), although similar results can be obtained with other
techniques such as MDSCAL-5 and KYST (Kruskal and Wish 1978). Other techniques permit linear constraints (Bentler and Weeks 1978; Carroll, Pruzansky, and Kruskal 1980), non-linear constraints (Lee and Bentler 1980) and equality constraints (Bloxom 1978) to be imposed on MDS solutions.
Convergent and Discriminant Validity
To define a set of constraintsthat assess convergent and discriminant validity, we propose a system that provides sets of constraintsthat varyin the stringency of the
convergent-discriminantvalidity requirements (cf. Borg 1977; Guttman 1959). To illustrate, suppose that two constructs under consideration are represented by two geometric regions Ra and Rb, with na and nb points, respectively. In this case, a definitional mapping system for clustering points according to convergent-discriminantrequirements can be expressedas follows: each point of Ra must be closer to (na, na - 1, . . ., 1) points of Ra
than it is to
1, . . .,
1) points of Rb.
Constraint condition nanb, for example, requires that each point within Ra (or each indicator of construct 1) be closer to all other points in that region (all other indicators of the same construct) than to any point in region b. Constraint condition na, nb - 1 implies that each point within Ra be closer to all other points within the region than to all but one point in region b. The nanb set of constraints represents the most stringent form of convergent-discriminantvalidity, while the other possible constraint conditions
A similar approachcan also be used to define a system for imposing a set of theory-based constraints. This form of validity requiresthat the solution reflect links between constructs as suggestedby a substantive theory. As an illustration, consider a three-construct model where the theory specifies that construct B mediates the relationship between construct A and construct C. In this case, a definitional mapping system that operationalizesa set of nomological validity constraints would specify that each point of Ra must be closer to (nb, nb - 1, . . ., 1) points of Rb than to (ne, n, - 1, . . ., 1) points of Rc, where Rbis a region adjacent to Ra; R, is a region distant from Ra; and na, nb, and n, are the number of points in each of the respective regions. So, for example, constraint condition ng, nc*
requires that each point of region A be closer to all points within region b than to any point within region c. (An asterisk is added to distinguish nomological from convergent-discriminantconstraints.) This representstesting criteriathat operationalizethe
theoret2Thisapproachrepresents extensionof Lingoes'( 1981, p. 290) an system for definingregionalityand contiguity.
IMPOSING VALIDITY CONSTRAINTS
This section defines a system for placing theoryand measurement-based constraints, derived using the MTMM framework, on a CMDS solution. Convergent validity requires that multiple measures of the same construct converge on that construct, and
MODELING DISTANCE STRUCTURES
ical assertionthat construct A should be more closely related to construct B than to construct C. As in the convergent-discriminant example presented previously, the other possible conditions specified by the mapping schema (nt, n* - 1 and nt - 1, nc*, and so on) provide operationalizations of weaker forms of nomological validity. Since structuralmodeling requires that both measurement- and
theory-basedconstraints be placed on a solution simultaneously, the analyst'stask is to pick a combination of more or less stringent operationalizations of these two types of validity. The combination na nbn* n* represents the most stringent set of constraints. Less stringent interpretations of measurement- and theory-based validity may also be defined. One of the features of the system is flexibility. Any combination of constraints can be imposed, and the system can easily be extended to more complex models with a greaternumber of latent constructs and variables.
ture analysis developed by Bentler and Bonett (1980), Sorbom and Joreskog (1982), and Fornell and Larcker( 1981). The statistical argumentis analogous to the rule of thumb that measures of association two to three times greaterthan their standarderrorare interpreted as "significant." For intermediate cases with a ratio between 1 and 3, Lingoes and Borg (1983b) describe an alternative decision rule that takes into account the sample size, the number of variables, the number of dimensions, and the percentage of distances that has been constrained.
If a model demonstratesan acceptablefit according to the criteria discussed in the previous section, the estimation of constructs, the indicator-constructrelationships, and the construct-construct relationships become of interest. The scaling procedure presented here allows numerical values to be assignedto each of these parameters.
EVALUATIVE STATISTICS: ASSESSING FIT
Before estimating the relationships between specific variables,it is necessaryto assess the congruence between the model and the data. Some measure of fit is needed for any procedurethat attempts to compare the congruence of a theoretical model and empirical data. Much like the literature on evaluating the fit of covariancestructuremodels, the choice of fit index in distance models is not without controversy. This article follows the approach taken by Lingoes and Borg (1983a, 1985), who suggest using an "efficacy coefficient"as an overall measure of fit. This coefficient is based on the partial correlation between the order of the distances inthe theoretically constrainedand unconstrainedconfigurations,partialing out the orderof the original proximity data. This partial correlation, p(X, Z. Y)-where X represents the order of the distances in an unconstrained MDS configuration; Z, the order of the distances in a corresponding MDS configuration with theoretical constraints; and Y, the order of the original proximity data-thus represents the association between the constrained and unconstrained configurations that cannot be attributed to the original proximity data. When the ratio of the partial correlation p to the coefficient of alientation (K),
p(X, Z. Y) 1l- p(XZ)2 ' (1)
For any set of multiple measures of a single construct in covariance structure analysis, it is assumed that at least some of the variation is due to a "true" value. If measurement errors are random, classical measurementtheory implies that the true value of the unobserved variableis approximatedby the expected value of the observed indicators. Thus, the true value for an MDS cluster of indicators representingan unobserved construct may be calculated by using the mean value on each dimension for the points in the region. That is,
nij=1 where Cik is the projection of the ith centroid on the kth dimension, Xijk is the projection of the jth indicator of the ith construct in the kth dimension, and ni is the number of indicators in the ith construct.
Once the centroid is determined, the Euclidian distance between an indicator and its associated construct serves as a measure of association that provides a basis for addressingmeasurement error.3Similar to true score theory, measurement error is thus considered to be equal to the difference between observed and true values.
Summing over dimensions, we represent errorsin measurement as:
3Itshould be recognizedthat this estimate of measurementerror is relative and dependent on other variablesin the model, rather than an absolute measure of reliability that could be compared acrossanalyses,such as an alphacoefficient.
exceeds 3, there is evidence for an acceptable fit, but if the ratio is less than 1, there is a lack of fit. These cutoffs are obviously somewhat arbitrary, although not totally void of a statistical argument.In this sense, they are similar to the fit indices for covariance struc-
482 TABLE 1
CORRELATIONMATRIXFOR OBSERVED INDICATORS
xY 1.00 .523 .611 .571 .696 .692 .656 .537 .523
1.00 .522 .781 .707 .585 .801 .668 .775
1.00 .481 .659 .659 .508 .417 .537
1.00 .826 .533 .875 .815 .755
1.00 .613 .819 .770 .740
1.00 .599 .493 .578
1.00 .825 .808
Association. Marketing fromthe American with N data is fromJagpaland Hui(1980, p. 359), and is reprinted permission NOTE: = 100; simulated
where eijis the distance between the ith construct and itsjth indicator.
Euclidian distances between the centroids that represent the constructs in a structuralmodel are the basis for computing construct-constructlinks. That is,
= dpq [ E (Cpk - Cqk)2]1/2 k= 1
is where dpq the distance between the pth and qth constructs, m is the number of dimensions, Cpkis the projection of the pth centroid in the kth dimension, and Cqk iS the projection of the qth centroid in the kth
AN ANALYSIS STRATEGY
The previous sections outlined methods for imposing validity constraints, assessing fit, and obtaining model parameters. This section describes an approach that combines these elements into an analysis strategy and an integrated system for modeling distance structures.A brief overview of the general strategy precedesan illustration of the procedurewith two empirical examples. Modeling distance structures via CMDS begins with a matrix of proximity measures representingthe relationships between the observed variables to be modeled. Proximity measures may be correlations, similarityjudgments, or other measuresof proximity, as long as at least ordinal-level assumptions are met. This matrix is then scaled to determine the number of dimensions needed to adequately represent the proximity measuresas distances in multidimensional space. Once dimensionality is established, confirmatory testing can begin by imposing the most rigorous set
of measurement and theory constraints, such as the strong cluster/strong theory condition (nfanbnt n*)) discussed earlier. This allows for the strongest possible test of a theory and
clearlyrevealsthose indicators not in keeping with the constraints implied by the combined set of convergent-discriminantand nomological validity constraints. If the strongestset of constraints is satisfied, the model and data are assumed congruent, and the model parameters can be computed. The analysis, in this instance, is straightforward. The more typical result is a less than perfect fit.
Convergent-discriminantor nomological constraints may be violated by
particularindicators, and clusters may overlap to varying degrees. Again, this is similar to the analysis of fit in covariance structureanalysis. Rarely, if ever, does the fitting function reach its minimum of zero. Even when a likelihood ratio statistic is used, a decision still must be made regardingthe acceptability of the resulting fit. When an acceptable fit is not obtained, one may either reject the model or pose an alternativeset of constraints that is less rigorous, yet still theoretically defensible. If the less rigorous set of constraints can be satisfied, one may then conclude that the model fits well enough to justify computing parameters. The goal is to obtain a parsimonious representation of the proximities, the distances, and the theoretical constraints. The data must first be representedin the smallest geometric space, so that little or no new information can be added by moving to a higherdimensionality. Then, theoretical constraints are added to make the alternative interpretations of the distances between points and clusters explicit, and to provide a test of the degree of fit between the data and the theory. Until a solution can satisfy both data and theory constraints, a truly parsimonious expression of the model has not been obtained.
Two step-by-step applications of the suggested analysis strategy are presented, and the results are
MODELING DISTANCE STRUCTURES FIGURE A
80 70 -X 60 50
(Z2 -\1 ,/
30 .20 10 0 \ , ,
-60 -70 -X
-I I I-I I -IA I
NOTE: Xi = awareness measures; Yi = preference measures; Zi = intention measures.
compared to a covariance structure analysis using LISRELto highlight some of the relative advantages and disadvantages of modeling distance and covariance structures.
sented in Table 1.4 A path analysis of these data was interpreted as providing support for the hierarchyof effects model by the original authors.
4Thereason we are using a correlationmatrixin our illustration is to demonstratethe differencesand similaritiesbetween distance structuresand covariancestructuresmodeling. Correlationaldata generallyrepresentmetricinput, necessaryfor covariancestructure