This example, from MacMillan et al. (1981), illustrates a repeated measures analysis in which there are two repeated measurement factors. Two diagnostic procedures
(standard and test) are performed on each subject, and the results of both are evaluated at each of two times as being positive
or negative. In the following DATA step, std1
and std2
are the two measurements of the standard procedure, and test1
and test2
are the two measurements of the test procedure:
data a; input std1 $ test1 $ std2 $ test2 $ wt @@; datalines; neg neg neg neg 509 neg neg neg pos 4 neg neg pos neg 17 neg neg pos pos 3 neg pos neg neg 13 neg pos neg pos 8 neg pos pos pos 8 pos neg neg neg 14 pos neg neg pos 1 pos neg pos neg 17 pos neg pos pos 9 pos pos neg neg 7 pos pos neg pos 4 pos pos pos neg 9 pos pos pos pos 170 ;
For the initial model, the response functions are marginal probabilities, and the repeated measurement factors are Time
and Treatment
. The model is a saturated one, containing effects for Time
, Treatment
, and Time
*Treatment
. The following statements produce Output 30.9.1:
proc catmod data=a; title2 'Marginal Symmetry, Saturated Model'; weight wt; response marginals; model std1*test1*std2*test2=_response_ / freq design noparm; repeated Time 2, Treatment 2 / _response_=Time Treatment Time*Treatment; run;
The analysis of variance table in Output 30.9.1 shows that there is no significant effect of Time
, either by itself or in its interaction with Treatment
. The second model includes only the Treatment
effect. Again, the response functions are marginal probabilities, and the repeated measurement factors are Time
and Treatment
.
Output 30.9.1: Diagnosis Data: Two Repeated Measurement Factors
Diagnostic Procedure Comparison |
Marginal Symmetry, Saturated Model |
Data Summary | |||
---|---|---|---|
Response | std1*test1*std2*test2 | Response Levels | 15 |
Weight Variable | wt | Populations | 1 |
Data Set | A | Total Frequency | 793 |
Frequency Missing | 0 | Observations | 15 |
Population Profiles | |
---|---|
Sample | Sample Size |
1 | 793 |
Response Profiles | ||||
---|---|---|---|---|
Response | std1 | test1 | std2 | test2 |
1 | neg | neg | neg | neg |
2 | neg | neg | neg | pos |
3 | neg | neg | pos | neg |
4 | neg | neg | pos | pos |
5 | neg | pos | neg | neg |
6 | neg | pos | neg | pos |
7 | neg | pos | pos | pos |
8 | pos | neg | neg | neg |
9 | pos | neg | neg | pos |
10 | pos | neg | pos | neg |
11 | pos | neg | pos | pos |
12 | pos | pos | neg | neg |
13 | pos | pos | neg | pos |
14 | pos | pos | pos | neg |
15 | pos | pos | pos | pos |
Response Frequencies | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Sample | Response Number | ||||||||||||||
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | |
1 | 509 | 4 | 17 | 3 | 13 | 8 | 8 | 14 | 1 | 17 | 9 | 7 | 4 | 9 | 170 |
Response Functions and Design Matrix | ||||||
---|---|---|---|---|---|---|
Sample | Function Number |
Response Function |
Design Matrix | |||
1 | 2 | 3 | 4 | |||
1 | 1 | 0.70870 | 1 | 1 | 1 | 1 |
2 | 0.72383 | 1 | 1 | -1 | -1 | |
3 | 0.70618 | 1 | -1 | 1 | -1 | |
4 | 0.73897 | 1 | -1 | -1 | 1 |
Analysis of Variance | |||
---|---|---|---|
Source | DF | Chi-Square | Pr > ChiSq |
Intercept | 1 | 2385.34 | <.0001 |
Time | 1 | 0.85 | 0.3570 |
Treatment | 1 | 8.20 | 0.0042 |
Time*Treatment | 1 | 2.40 | 0.1215 |
Residual | 0 | . | . |
A main effect model with respect to Treatment
is fit. The following statements produces Output 30.9.2:
title2 'Marginal Symmetry, Reduced Model'; model std1*test1*std2*test2=_response_ / corrb design noprofile; repeated Time 2, Treatment 2 / _response_=Treatment; run;
The analysis of variance table for the reduced model (Output 30.9.2) shows that the model fits (since the residual chi-square is nonsignificant) and that the treatment effect is significant.
The negative parameter estimate for Treatment
shows that the first level of treatment (std) has a smaller probability of the first response level (neg) than the second
level of treatment (test). In other words, the standard diagnostic procedure gives a significantly higher probability of a
positive response than the test diagnostic procedure.
Output 30.9.2: Diagnosis Data: Reduced Model
Diagnostic Procedure Comparison |
Marginal Symmetry, Reduced Model |
Data Summary | |||
---|---|---|---|
Response | std1*test1*std2*test2 | Response Levels | 15 |
Weight Variable | wt | Populations | 1 |
Data Set | A | Total Frequency | 793 |
Frequency Missing | 0 | Observations | 15 |
Response Functions and Design Matrix | ||||
---|---|---|---|---|
Sample | Function Number |
Response Function |
Design Matrix | |
1 | 2 | |||
1 | 1 | 0.70870 | 1 | 1 |
2 | 0.72383 | 1 | -1 | |
3 | 0.70618 | 1 | 1 | |
4 | 0.73897 | 1 | -1 |
Analysis of Variance | |||
---|---|---|---|
Source | DF | Chi-Square | Pr > ChiSq |
Intercept | 1 | 2386.97 | <.0001 |
Treatment | 1 | 9.55 | 0.0020 |
Residual | 2 | 3.51 | 0.1731 |
Analysis of Weighted Least Squares Estimates | |||||
---|---|---|---|---|---|
Effect | Parameter | Estimate | Standard Error |
Chi- Square |
Pr > ChiSq |
Intercept | 1 | 0.7196 | 0.0147 | 2386.97 | <.0001 |
Treatment | 2 | -0.0128 | 0.00416 | 9.55 | 0.0020 |
Correlation Matrix of the Parameter Estimates |
||
---|---|---|
Row | Col1 | Col2 |
1 | 1.00000 | 0.04194 |
2 | 0.04194 | 1.00000 |
The next example illustrates a RESPONSE statement that, at each time, computes the sensitivity and specificity of the test diagnostic procedure with respect to the standard procedure. Since these are measures of the relative accuracy of the
two diagnostic procedures, the repeated measurement factors in this case are labeled Time
and Accuracy
. Only 15 of the 16 possible responses are observed, so additional care must be taken in formulating the RESPONSE statement for computation of sensitivity and specificity.
The following statements produce Output 30.9.3 and Output 30.9.4:
title2 'Sensitivity and Specificity Analysis, ' 'Main-Effects Model'; model std1*test1*std2*test2=_response_ / covb design noprofile; repeated Time 2, Accuracy 2 / _response_=Time Accuracy; response exp 1 -1 0 0 0 0 0 0, 0 0 1 -1 0 0 0 0, 0 0 0 0 1 -1 0 0, 0 0 0 0 0 0 1 -1 log 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1, 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1, 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0, 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0, 0 0 0 1 0 0 1 0 0 0 1 0 0 0 1, 0 0 1 1 0 0 1 0 0 1 1 0 0 1 1, 1 0 0 0 1 0 0 1 0 0 0 1 0 0 0, 1 1 0 0 1 1 0 1 1 0 0 1 1 0 0; quit;
For the sensitivity and specificity analysis, the four response functions displayed next to the design matrix (Output 30.9.3) represent the following:
sensitivity, time 1
specificity, time 1
sensitivity, time 2
specificity, time 2
The sensitivities and specificities are for the test diagnostic procedure relative to the standard procedure.
Output 30.9.3: Diagnosis Data: Sensitivity and Specificity Analysis
Diagnostic Procedure Comparison |
Sensitivity and Specificity Analysis, Main-Effects Model |
Data Summary | |||
---|---|---|---|
Response | std1*test1*std2*test2 | Response Levels | 15 |
Weight Variable | wt | Populations | 1 |
Data Set | A | Total Frequency | 793 |
Frequency Missing | 0 | Observations | 15 |
Response Functions and Design Matrix | |||||
---|---|---|---|---|---|
Sample | Function Number |
Response Function |
Design Matrix | ||
1 | 2 | 3 | |||
1 | 1 | 0.82251 | 1 | 1 | 1 |
2 | 0.94840 | 1 | 1 | -1 | |
3 | 0.81545 | 1 | -1 | 1 | |
4 | 0.96964 | 1 | -1 | -1 |
Analysis of Variance | |||
---|---|---|---|
Source | DF | Chi-Square | Pr > ChiSq |
Intercept | 1 | 6448.79 | <.0001 |
Time | 1 | 4.10 | 0.0428 |
Accuracy | 1 | 38.81 | <.0001 |
Residual | 1 | 1.00 | 0.3178 |
The ANOVA table in Output 30.9.3 shows that an additive model fits, that there is a significant effect of time, and that the sensitivity is significantly different from the specificity.
Output 30.9.4 shows that the predicted sensitivities and specificities are lower for time 1 (since parameter 2 is negative). It also shows that the sensitivity is significantly less than the specificity.
Output 30.9.4: Parameter Estimates
Analysis of Weighted Least Squares Estimates | |||||
---|---|---|---|---|---|
Effect | Parameter | Estimate | Standard Error |
Chi- Square |
Pr > ChiSq |
Intercept | 1 | 0.8892 | 0.0111 | 6448.79 | <.0001 |
Time | 2 | -0.00932 | 0.00460 | 4.10 | 0.0428 |
Accuracy | 3 | -0.0702 | 0.0113 | 38.81 | <.0001 |
Covariance Matrix of the Parameter Estimates | |||
---|---|---|---|
Row | Col1 | Col2 | Col3 |
1 | 0.00012260 | 0.00000229 | 0.00010137 |
2 | 0.00000229 | 0.00002116 | -.00000587 |
3 | 0.00010137 | -.00000587 | 0.00012697 |