DOC

# One-Way Multiple Analysis of Variance

By Lisa Knight,2014-03-03 02:14
30 views 0
I have never seen anyone else compute canonical scores like this and then do pairwise comparisons on them, but it seems sensible to me, and such analysis

One-Way Multiple Analysis of Variance

When one wishes to determine whether two or more groups differ significantly on one or more optimally weighted linear combinations (canonical variates or

discriminant functions) of two or more normally distributed dependent variables, a one-way multiple analysis of variance is appropriate. We have already studied Discriminant Function Analysis, which is mathematically equivalent to a one-way MANOVA, so we are just shifting perspective here.

A Manipulation Check for Michelle Plaster’s Thesis

Programs page. Run the program. The data are from Michelle Plaster's thesis, which you may wish to look over (in Joyner or in the Psychology Department). The analyses she did are very similar to, but not identical to, those we shall do for instructional purposes. Male participants were shown a picture of one of three young women. Pilot work had indicated that the one woman was beautiful, another of average physical attractiveness, and the third unattractive. Participants rated the woman they saw on each of twelve attributes. Those ratings are our dependent variables. To simplify the analysis, we shall deal with only the ratings of physical attractiveness (PHYATTR), happiness (HAPPY), INDEPENdence, and SOPHISTication. The purpose of the research was to investigate the effect of the defendant’s physical attractiveness (and some other variables which we shall ignore here) upon the sentence recommended in a simulated jury trial. The MANOVA on the ratings served as a check on the effectiveness of our manipulation of the physical attractiveness (PA) independent variable.

Screening the Data

The overall means and standard deviations on the dependent variables were obtained so that I could standardize them and then compute scores on the canonical variates. Although you can not see it in this output, there is heterogeneity of variance

on the physical attractiveness variable. The F is 4.62. With n's approximately equal, max

I'm not going to worry about it. If F were larger and sample sizes less nearly equal, I max

might try data transformations to stabilize the variance or I might randomly discard scores to produce equal n's and thus greater robustness.

One should look at the distributions of the dependent variables within each cell. I have done so, (using SAS), but have not provided you with the statistical output. Such output tends to fill many pages, and I generally do not print it, I just take notes from the screen. Tests of significance employed in MANOVA do assume that the dependent variables and linear combinations of the dependent variables are normal

MANOVA1.doc

2

within each cell. With large sample sizes (or small samples with approximately equal n’s), the tests may be fairly robust to violations of this assumption. I must confess that there are some problems with the normality assumption for these data, especially the phyattr variable. It is negatively skewed in one cell, positively skewed in another, due to ceiling and floor effects. I was unable to find anything in my bag of tricks (nonlinear transformations and deletion of outliers) that ameliorated this problem to my satisfaction, so I decided to stick with the data as they are, with the caution that they are not exactly normal.

The MANOVA with Canonical Analysis

PROC ANOVA is employed to do the multiple analysis of variance. Note that there are multiple outcome variables listed in the model statement. The NOUNI option

is used to suppress output of the univariate analyses of variance.

MANOVA H = PA / CANONICAL indicates that we want to do a MANOVA test of the

null hypothesis that there is no effect of our physical attractiveness manipulation and that we want statistics about the canonical variates.

Canonical Analysis (page 3)

An effect will have as many roots (discriminant functions, canonical variates)

as it has degrees of freedom, or it will have as many roots as there are DVs, whichever is fewer. Our manipulation has 3 levels, 2 df, so we have two roots. Each canonical

variate is a weighted linear combination of the variables. The first canonical variate is

SSamong_groupsthat weighted linear combination that maximizes the ratio . These sums SSwithin_groups

of squares are those that would be obtained were we to do a univariate ANOVA using our manipulation as the grouping variable and the weighted linear combination of our four ratings variables as the single outcome variable. This ratio is called the eigenvalue. For our data, it is 1.7672. The proportion for the first root, .9133, is

simply the first root's eigenvalue expressed as a percentage of the simple sum of all the roots' eigenvalues 1.7672 / (1.7672 + .1677).

The second root, orthogonal to the first, is that weighted linear combination of the dependent variables that maximizes the eigenvalue using variance in the variables not already captured by the earlier root(s).

SSamonggroups_ Were we to do a univariate ANOVA on the first canonical variate, SStotal

would be the first squared canonical correlation. You should recognize this ratio as 22being . For our data, , the squared canonical correlation, for the first root is .64, that is, 64% of the variance in the first canonical variate is accounted for by our manipulation. For the second canonical variate it is 14%.

SAS gives us, along with the eigenvalues, tests of the significance of the roots. The first row gives us the test of the null hypothesis that all of the canonical variates are unrelated to the grouping variable. The second row tests the null hypothesis that

3

all of the canonical variates from the second are unrelated to the grouping variable, etc. As you can see, both of our canonical variates are significantly affected by our physical attractiveness manipulation.

We found that our groups differ significantly on two dimensions. Let us now look at the standardized discriminant function coefficients and the correlations between observed variables and canonical variates to interpret those dimensions.

The standardized canonical coefficients may be treated as Beta weights in a

multiple regression predicting C from standardized X scores, where C is the canonical variate, the weighted linear combination of the variables which maximizes the eigenvalue.

C = B Z + B Z + . . . + B Z. i1122pp

Under the heading “Canonical Coefficients” are the total-sample standardized

canonical correlation coefficients -- these are the coefficients obtained if the Z

scores above are computed by using, for each variable, its mean and standard deviation ignoring groups (rather than within groups). Of course, one must realize that these coefficients reflect the contribution of one variable in the context of the other

variables in the model. A low standardized coefficient might mean that the groups do not differ much on that variable, or it might just mean that that variable's correlation with the grouping variable is redundant with that of another variable in the model. Suppressor effects can also occur. Raw (unstandardized) coefficients are presented as well, but are rarely of any use.

Loadings (correlations between the dependent variables and the canonical variates) may be more helpful in terms of understanding the canonical variates we have created. These are available under the heading “Canonical Structure.” I am accustomed to using the “Within” loadings, which are computed by computing the correlations within each group and then averaging them across groups. Generally, any variate with a loading of .40 or more must be considered to be important. One might

apply a “meaningfulness criterion” with loadings between .30 and .40 -- that is, if they

make sense, interpret them, if not, declare them too small to be worthy of your attention. ;

Our loadings and our standardized coefficients indicate that canonical variate 1 is largely a matter of physical attractiveness and canonical variate 2 is largely happiness and independence. The role of the sophistication rating is less clear. Neither of its loadings is high and it is involved in some sort of suppressor effect on the second canonical variate. It might be interesting to drop this variable and redo the analysis.

MANOVA Test Criteria and F Approximations for the Hypothesis of No Overall PA Effect (page 4)

Following the canonical statistics, SAS gives us four different tests of the null hypothesis that none of the canonical variates is correlated with our treatment. Hotelling's trace is the simple sum of all the roots' eigenvalues.

4

If we were ambitious (or foolish) enough to work out our MANOVA by hand (using matrix algebra, which is very arithmetic intensive, yuk), we would calculate treatment (among groups) and error (within groups) SSCP (sums of squares and cross

products) matrices (like variance-covariance matrices, but with sums of squares on the main diagonal and cross products elsewhere -- dividing the elements by degrees of freedom would produce variances and covariances). Since we have 4 outcome variables, each matrix (treatment and error) would be a 4 x 4 matrix. In univariate ANOVA we test an effect by computing F, the ratio of a treatment variance (mean

square) to error variance.

In MANOVA we may test our treatment by finding the ratio of the determinant (generalized variance) of the error SSCP matrix to the determinant of the sum of the

treatment and error SSCP matrices. This ratio is called Wilks' Lambda ( ). Since

errorthe ratio is , one may interpret the Wilks' Lambda as the proportion of error ; treatment

variance in the DVs that is not accounted for by the IV. Unlike F, where large values

cast doubt on the null hypothesis that the treatment groups do not differ, small values of Wilks’ Lambda cast doubt on the null hypothesis.

2 An eta-squared statistic can be computed as = 1 - , .69 for our data. We

may interpret this statistic as being the proportion of variance in the dependent variables that is accounted for by the physical attractiveness manipulation. When there 2are only two treatment groups, this will equal the squared canonical correlation.

I shall use lower case lambda ( ) to stand for an eigenvalue. If we define theta

as , then Wilks' can be computed from . For our data, for the first root = ;1

1.767/2.767 = 0.6386, and, for the second root, .168/1.168 = 0.1438. To obtain Wilks'

, subtract each theta from one and then find the product of these differences: (1 - .6386)(1 - .1438) = .309.

Pillai's Trace is the sum of all the thetas, .6386 + .1438 = .782.

Roy's greatest characteristic root is simply the eigenvalue for the first root.

Roy's gcr should be the most powerful test when the first root is much larger than the other roots.

Each of these statistics' significance levels is approximated using F.

Univariate ANOVAs on the Canonical Variates and the Original Variables

Now, look back at the data step of our program. I used the total-sample means and standard deviations to standardize the variables into Z_scores. I then computed, for each subject, canonical variate scores, using the total-sample standardized canonical correlation coefficients. CV1 is canonical variate 1, CV2 is canonical variate

2. I computed these canonical variate scores so I could use them as outcome variables in univariate analyses of variance. I used PROC ANOVA to do univariate ANOVAs with Fisher’s LSD contrasts on these two canonical variates and also, for the benefit of

5

those who just cannot understand canonical variates, on each of the observed variables.

Please note that for each canonical variate:

( If you take the SS and divide by the SS , you get the eigenvalue among groupswithin groups

reported earlier for that root.

( If you take the ratio of SS to SS , you get the root's squared canonical among groupstotal

correlation.

( The group means are the group centroids that would be reported with a

discriminant function analysis.

I have never seen anyone else compute canonical scores like this and then do pairwise comparisons on them, but it seems sensible to me, and such analysis has been accepted in manuscripts I have published. As an example of this procedure, see

Mock Jurors’ Insanity Defense Verdict Selections. my summary of an article on

Note that on the "beauty" canonical variate, the beautiful group's mean is significantly higher than the other two means, which are not significantly different from one another. On the “happily independent” variate, the average group is significantly higher than the other two groups.

The univariate ANOVAs reveal that our manipulation significantly affected every one of the ratings variables, with the effect on the physical attractiveness ratings being 2very large ( = .63). Compared to the other groups, the beautiful woman was rated significantly more attractive and sophisticated, and the unattractive woman was rated significantly less independent and happy.

Unsophisticated Use of MANOVA

Unsophisticated users of MANOVA usually rely on the Univariate F-tests to try

to interpret a significant MANOVA. These are also the same users who believe that the purpose of a MANOVA is to guard against inflation of alpha across several DVs.

They promise themselves that they will not even look at the univariate ANOVAs for any effect which is not significant in the MANOVA. The logic is essentially the same as that in the Fisher’s LSD protected test for making pairwise comparisons between means following a significant effect in ANOVA (this procedure has a lousy reputation -- not conservative enough -- these days, but is actually a good procedure when only three means are being compared; the procedure can also be generalized to other sorts of analyses, most appropriately those involving effects with only two degrees of freedom). In fact, this "protection" is afforded only if the overall null hypothesis is true (none of the outcome variables is associated with the grouping variable), not if some outcome variables are associated with the grouping variable and others are not. If one outcome variable is very strongly associated with the grouping variable and another very weakly (for practical purposes, zero effect), the probability of finding the very weak effect to be "significant" is unacceptably high.

6

Were we so unsophisticated as to take such a "MANOVA-protected univariate tests" approach, we would note that the MANOVA was significant, that univariate tests on physically attractive, happy, independent, and sophisticated were significant, and then we might do some pairwise comparisons between groups on each of these "significant" dependent variables. I have included such univariate tests in my second invocation of PROC ANOVA. Persons who are thinking of using MANOVA because they have an obsession about inflated familywise alpha and they think MANOVA somehow protects against this should consider another approach -- application of a Bonferroni adjustment to multiple univariate ANOVAs. That is, dispense with the MANOVA, do the several univariate ANOVAs, and evaluate each ANOVA using an adjusted criterion of significance equal to the familywise alpha you can live with divided by the number of tests in the family. For example, if I am willing to accept only a .05 probably of rejecting one or more of four true null hypotheses (four dependent variables, four univariate ANOVAs), I used an adjusted criterion of .05/4 = .0125. For each of the ANOVAs I declare the effect to be significant only if its p .0125. This will, of course,

increase the probability of making a Type II error (which is already the most likely sort of error), so I am not fond of making the Bonferroni adjustment of the per-comparison criterion of statistical significance. Read more on this in my document Familywise

Alpha and the Boogey Men.

While I have somewhat disparaged the “protected test” test use of MANOVA, I must confess that I sometimes employ it, especially in complex factorial designs where, frankly, interpreting the canonical variates for higher order effects is a bit too challenging for me. With a complex factorial analysis (usually from one of my colleagues, I usually having too much sense to embark on such ambitious projects) I may simply note which of the effects are significant in the MANOVA and then use univariate analyses to investigate those (and only those) effects in each of the dependent variables. Aside from any protection against inflating familywise alpha, the main advantage of this approach is that it may impress some reviewers of your manuscript, multivariate analyses being popular these days.

This unsophisticated approach just described ignores the correlations among the outcome variables and ignores the dimensions (canonical variates, discriminant functions) upon which MANOVA found the groups to differ. This unsophisticated user may miss what is really going on -- it is quite possible for none of the univariate tests to be significant, but the MANOVA to be significant.

Relationship between MANOVA and DFA

I also conducted a discriminant function analysis on these data just to show you the equivalence of MANOVA and DFA. Please note that the eigenvalues, canonical correlations, loadings, and canonical coefficients are identical to those obtained with the MANOVA.

SPSS MANOVA

7

SPSS has two routines for doing multiple analysis of variance, the GLM routine and the MANOVA routine. Let us first consider the GLM routine. Go to my SPSS Data

Page and download the SPSS data file, Plaster.sav. Bring it into SPSS and click Analyze, General Linear Model, Multivariate. Move our outcome variables (phyattr, happy, indepen, and sophist) into the dependent variables box and the grouping variable (PA, manipulated physical attractiveness) into the fixed factor box.

Under Options, select descriptive statistics and homogeneity tests. Now, look at the output. Note that you do get Box’s M, which is not available in SAS. You also get

the tests of the significance of all roots simultaneously tested and Roy’s test of only the first root, as in SAS. Levene’s test is used to test the significance of group differences in variance on the original variables. Univariate ANOVAs are presented, as are basic descriptive statistics. Missing, regretfully, are any canonical statistics, and these are not available even if you select every optional statistic available with this routine. What a bummer. Apparently the folks at SPSS have decided that people that can only point and click will never do a truly multivariate analysis of variance, that they only use what I have called the unsophisticated approach, so they have made the canonical statistics unavailable.

Fortunately, the canonical statistics are available from the SPSS MANOVA routine. While this routine was once available on a point-and-click basis, it is now available only by syntax -- that is, you must enter the program statements in plain text, just like you would in SAS, and then submit those statements via the SPSS Syntax Editor. Point your browser to my SPSS programs page and download the file

MANOVA1.sps to your hard drive or diskette. From the Windows Explorer or My

8

Computer, double click on the file. The SPSS Syntax Editor opens with the program displayed. Edit the syntax file so that it points to the correct location of the Paster.sav file and then click on Run, All, The output will appear.

I did not ask SPSS to print the variance/covariance matrices for each cell, but I did get the determinants of these matrices, which are used in Box's M. You may think

of the determinants as being indices of the generalized variance within a variance/covariance matrix. For each cell, for our 4 dependent variable design, that matrix is a 4 x 4 matrix with variances on the main diagonal and covariances (between each pair of DVs) off the main diagonal. Box's M tests the null hypothesis that the

variance/covariance matrices in the population are identical across cells. If this null hypothesis is false, the pooled variance/covariance matrix used by SPSS is inappropriate. Box's M is notoriously powerful, and one generally doesn't worry unless p <.001 and sample sizes unequal. Using Pillai's trace (rather than Wilks' ) may

improve the robustness of the test in these circumstances. One can always randomly discard scores to produce equal n's. Since our n's are nearly equal, I'll just use Pillai's

trace and not discard any scores.

Look at the pooled within cells (thus eliminating any influence of the grouping variable) correlation matrix it is now referred to as the WITHIN+RESIDUAL

correlation matrix. The dependent variables are generally correlated with one another. Bartlett's test of sphericity tests the null hypothesis that in the population the

correlation matrix for the outcome variables is an identity matrix (each r = 0). That is ij

clearly not the case with our variables. Bartlett’s test is based on the determinant of the

within-cells correlation matrix. If the determinant is small, the null hypothesis is rejected. If the determinant is very small (zero to several decimal places), then at least one of the variables is nearly a perfect linear combination of the other variables. This creates a problem known as multicollinearity. With multicollinearity your solution is

suspect -- another random sample from the same population would be likely to produce quite different results. When this problem arises, I recommend that you delete one or more of the variables. If one of your variables is a perfect linear combination of the others (for example, were you to include as variables SAT, SAT and SAT), TotalMathVerbal

the analysis would crash, due to the singularity of a matrix which needs to be inverted

as part of the solution. If you have a multicollinearity problem but just must retain all of the outcome variables, you can replace those variables with principal component

scores. See Principal Components Discriminant Function Analysis for an example.

For our data, SPSS tells us that the determinant of the within cells correlation

matrix has a log of -.37725. Using my calculator, natural log (.689) = -.37725 -- that is, the determinant is .689, not near zero, apparently no problem with multicollinearity. 2Another way to check on multicollinearity is to compute the R between each variable 2and all the others (or 1-R, the tolerance). This will help you identify which variable you might need to delete to avoid the multicollinearity problem. I used SAS’ PROC 2FACTOR to get the Rs, which are:

Prior Communality Estimates: SMC

9

PHYATTR HAPPY INDEPEN SOPHIST

0.136803 0.270761 0.133796 0.289598

Note that the MANOVA routine has given us all of the canonical statistics that we are likely to need to interpret our multivariate results. If you compare the canonical coefficients and loadings to those obtained with SAS, you will find that each SPSS coefficient equals minus one times the SAS coefficient. While SAS constructed canonical variates measuring physical attractiveness (CV1) and

happiness/independence (CV2), SPSS constructed canonical variates measuring physical unattractiveness and unhappiness/dependence. The standardized coefficients presented by SPSS are computed using within group statistics. Return to my Stats Lessons Page