Guidelines Versus Practices in Cross-Lingual Assessment: A Disconcerting Disconnect

The International Test Commission's Guidelines for Translating and Adapting Tests (2010) provide important guidance on developing and evaluating tests for use across languages. These guidelines are widely applauded, but the degree to which they are followed in practice is unknown. The objective of this study was to perform a systematic review of published articles to determine whether the methodologies related to test adaptation and evaluation of cross-lingual measurement bias have improved since the initial publication of the Guidelines. Results demonstrated that although reporting reliability estimates and statistically analyzing construct equivalence improved since publication of the Guidelines, the majority of test development and validation practices in the published literature have not embraced the recommendations put forth by the International Test Commission. Overall, these findings call into question many of the inferences derived from cross-linguistic comparisons and clearly signal a need for better dissemination mechanisms of the Guidelines.