Difference scores: A caveat illustrated with neuropsychological measures

Relationships among preferred minus non-preferred hand difference scores from four neuropsychological sensorimotor tests were evaluated in Community (N= 121), Prison Inmate (N = 350), and Psychiatric Inpatient (N = 398) samples. Across the three samples, the average absolute off-diagonal correlations among the four difference scores ranged from .07 to .13, and the squared multiple correlations (used to predict any difference score from the other three) did not exceed .07. Based on the determinant of the correlation matrix (i.e., |R|), the null hypothesis of complete independence was rejected in only the Prison Inmate sample (p < .001), however, departure from the null hypothesis was negligible (i.e., |R| = .94, expected value under the null = .98). Preferred/non-preferred hand ratio scores were very highly correlated with the difference scores and consequently analyses of the ratio scores yielded results that were comparable to those obtained for the difference scores. Results indicate that the four difference scores or four ratio scores measure little in common and therefore cannot corroborate each other. In general, it would be prudent to exercise considerable caution when using difference scores or ratio scores obtained from non-independent measures, either singly or in combination.