Beyond the Spearman-Brown: a structural approach to maximal reliability

The requirement of parallel parts has long been the cornerstone of classic reliability theory. By recasting reliability in a structural equation framework, items, raters, or judges no longer need to be treated as equivalent entities. Instead, unique reliability estimates can be determined for each and collectively used to assess the maximal reliability of a weighted composite, with the composite reliability submitted to inferential test. Procedures are shown to generalize from single to multifactor applications. Ramifications of a structural approach to reliability determination are probed, and the dilemma posed by possible falsification of the true score hypothesis presented for individual researcher consideration.     

The impact of modified instructions on ego-level scores: a psychometric hazard or indication of optimal ego level?

Various modified instructions have different effects on the Washington University Sentence Completion Test for Ego Development (Loevinger, 1985). These effects were reviewed and 2 alternative explanations were explored: The "measurement unreliability explanation" versus the "optimal level explanation." Both explanations were systematically studied in 2 test-retest experiments with the Sentence Completion Test for Children and Youths (Westenberg, Treffers, & Drewes, 1998). The modified instructions were to make a favorable impression on the tester ("fake good" condition) or to complete the sentences in as adult and mature a manner as one can ("be mature" condition). Both experiments were conducted with 9- to 15-year-old children and adolescents (N = 127, 128). As was anticipated, neither the fake good nor the be mature condition yielded higher (or lower) reliability indexes as compared to the standard instructions, hence discounting the measurement unreliability explanation. Also as expected, the fake good condition did not yield significantly higher ego-level scores, whereas the be mature condition did yield significantly higher ego-level scores. The former instructions did not convey information relevant to the construct or measure of ego development, whereas the latter instructions did convey information relevant for raising ego-level scores. It is argued that the higher scores under the be mature instructions might reflect the respondents' "optimal" ego level (best functioning), whereas the ego-level score under the standard instructions might reflect their "functional" level (normal functioning).