Criterion-referenced and norm-referenced assessment of minority group children

A variety of problems have been experienced with psychological assessment of minority children. Traditional norm-referenced measurement has repeatedly received criticism concerning cultural unfairness or bias. Responses to such accusations primarily have been in the form of new instrumentation aimed at attaining a culture fair assessment. Little response has been evident from a conceptual standpoint addressing the issues of purpose and use of test results Although many have turned to criterion-referenced measurement as an answer to the problems of norm-referenced evaluation, cultural bias is not necessarily avoided in this framework either. Issues of who determines criteria and what those criteria include must be addressed if criterion-referenced measurement is to meet adequately the challenge of multicultural evaluation.     

大学生网络成瘾量表（修订版）的多元概化分析*

用多元概化理论考察大学生网络成瘾量表在大学生群体中应用的测量学性能。以随机测量模式的概化设计,针对1200名在校大学生进行问卷调查。结果显示双因子结构的相关程度在0.92以上,五因子结构的相关程度均在0.76~0.97间;整体量表的概化系数和可靠性指数均达到了0.94以上,而双因子结构各因子在0.90左右,五因子结构各因子均在0.74~0.85间。所以,整体量表及各因子在大学生群体中应用的信效度较高,可用作常模和标准参照测验;无论双因子还是五因子结构,CIAS-R各因子在分值比和项目数上,设计非常合理和完善。     

Assessment of Attention in Preschoolers

In the past two decades, there has been an increased interest in the assessment and treatment of preschool children presenting with concerns about attention problems. This article reviews the research and clinical literature involving assessment of attention and related skills in the preschool years. While inattention among preschoolers is common, symptoms alone do not necessarily indicate a disorder, and most often represent a normal variation in typical preschool child development. Thus, accurate identification of ??disordered?? attention in preschoolers can be challenging, and development of appropriate, norm-referenced tests of attention for preschoolers is also difficult. The current review suggests that comprehensive assessment of attention and related functions in the preschool child should include thorough review of the child??s history, planned observations, and formal psychometric testing. The three primary methods of psychometric assessment that have been used to characterize attentional functioning in preschool children include performance-based tests, structured caregiver interviews, and rating scales (parent, teacher, and clinician). Among performance-based methods for measurement of attention in the preschool years, tests have been developed to assess sustained attention, selective (focused) attention, span of attention (encoding/manipulation), and (top-down) controlled attention??including freedom from distractibility and set shifting. Many of these tests remain experimental in nature, and review of published methods yields relatively few commercially available, nationally normed tests of attention for preschoolers, and an overall dearth of reliability and validity studies on the available measures.     

Test Anxiety: A Multifaceted Study on the Perceptions of Teachers,Principals, Counselors,Students, and Parents

The passage of "No Child Left Behind" (NCLB) legislation has led to an increased awareness of testing and assessment in public school systems and its impact. A cursory review of the academic literature and national news sources on the impact of standardized testing revealed a plethora of anecdotal cases of students experiencing illness, anxiety, and heightened levels of stress all attributed to the administration of these examinations. Furthermore, numerous studies have surveyed teachers regarding the impact of standardized tests, producing similar overviews that low performance on these examinations is correlated with increased levels of anxiety and stress. An element glaring by its omission is a formal study that surveys all stakeholders surrounding public school systems, including students, parents, teachers, principals, and counselors to gain a comprehensive understanding of the perceptions of standardized testing. Furthermore, a unique aspect of this study is the linking of student, parent, and teacher responses directly to student performance on a criterion-referenced state examination and a national norm-referenced examination. The results from these studies suggest that most of the "dangers" of standardized testing are overstated and misrepresented and that most students, parents, principals, and counselors value these tests and do not report increased levels of stress or anxiety. However, teachers as a group do present strong misgivings about standardized testing.     

A Systematic Review and Psychometric Evaluation of Adaptive Behavior Scales and Recommendations for Practice

Adaptive behavior scales are vital in assessing children and adolescents who experience a range of disabling conditions in school settings. This article presents the results of an evaluation of the design characteristics, norming, scale characteristics, reliability and validity evidence, and bias identification studies supporting 14 norm-referenced, informant-based interviews and rating scales designed to measure adaptive behaviors. To derive these results, the manuals for each of these scales were reviewed using a standardized coding procedure, and information about each scale was double-coded by reviewers. Findings reveal that several evidence-based adaptive behavior scales are available to school psychologists. Concluding recommendations address selection and use of adaptive behavior scales as part of a comprehensive assessment, using the optimal methods of administration of adaptive behavior scales, and interpreting resultant scores that have demonstrated the highest levels of reliability and the largest body of validity evidence.     

Bias and equivalence in cross-cultural assessment: an overview

In every cross-cultural study, the question as to whether test scores obtained in different cultural populations can be interpreted in the same way across these populations has to be dealt with. Bias and equivalence have become the common terms to refer to the issue. Taxonomy of both bias and equivalence is presented. Bias can be engendered by the theoretical construct (construct bias), the method such as the form of test administration (method bias), and the item content (item bias). Equivalence refers to the measurement level at which scores can be compared across cultures. Three levels of equivalence are possible: the same construct is measured in each cultural group but the functional form of the relationship between scores obtained in various groups is unknown (structural equivalence), scores have the same measurement unit across populations but have different origins (measurement unit equivalence), and scores have the same measurement unit and origin in all populations (full scale equivalence). The most frequently encountered sources of bias and their remedies are described.     

Cross‐cultural Measurement Invariance of the Employment Opportunity Index (EOI) in Mexican and Brazilian Professionals

Perceived employment alternatives have been shown to add incremental variance in understanding voluntary turnover as well as factors related to recruiting and staffing. The Employment Opportunity Index (EOI) is a multidimensional assessment of perceived employment alternatives that has exhibited predictive validity across multiple job types and populations. This study assessed the measurement invariance of the EOI in American, Mexican, and Brazilian professionals. We found support for configural and metric invariance in all five dimensions of the EOI with these populations, and support for scalar invariance three of the five EOI dimensions. The construct validity of the EOI appears to be relatively robust in Latin American populations, although cultural and macroeconomic factors may impose some response bias in these groups.     

The priority of just-identified,recursive models

Theories often place constraints on causal relationships, and such constraints are often assessed with causal models. Causal models should be recursive and just identified because cause is recursive and is more likely to be just identified than overidentified. A just-identified, recursive model (JIRM) is specified that satisfies both requirements and that can be used to assess a wide range of causal implications in either a norm-referenced or criterion-referenced manner. P. E. Meehl and N. G. Waller (2002) proposed an innovative method for theory appraisal called the delete one-add one (D1-A1) method, which assesses a relatively narrow range of causal implications, allows nonrecursive models, and is only norm referenced. The JIRM and D1-A1 methods are compared.     

Measuring logistics performance of OECD countries via fuzzy linear regression

In logistics, performance measurement has been considered as a key competency to acquire world class performance. In light of this, we presented a robust methodology to establish an analysis framework for measuring logistics performance. The proposed hybrid methodology is a combination of criteria importance through intercritera correlation (CRITIC), simple additive weighting (SAW), and Peters' fuzzy regression methods. To the best of our knowledge, country‐based logistics performance is seldom studied in the literature. Therefore, we measured the logistics performance of Organization for Economic Cooperation and Development (OECD) countries using the devised model based on the data of Logistics Performance Index 2014 provided by the World Bank. The introduced methodology, which is suitable to model imprecise relationships among system parameters, appears to be a practical alternative approach for the assessment of logistics performance. It should be noted that the evaluation framework presented in this paper is not confined to performance measurement case and can also be exploited in addressing other multiple criteria decision‐making problems incorporating uncertainty.     

An item response theory analysis of DSM-IV personality disorder criteria across younger and older age groups

Many of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV; American Psychiatric Association, 1994) personality disorder (PD) diagnostic criteria focus on a younger social and occupational context. The absence of age-appropriate criteria for older adults forces researchers and clinicians to draw conclusions based on existing criteria, which are likely inadequate. To explore which DSM-IV PD criteria contain age group measurement bias, the authors report 2 analyses of data on nearly 37,000 participants, ages 18-98 years, taken from a public data set that includes 7 of the 10 PDs (antisocial, avoidant, dependent, histrionic, obsessive-compulsive, paranoid, and schizoid). The 1st analysis revealed that older age groups tend to endorse fewer PD criteria than younger age groups. The 2nd analysis revealed that 29% of the criteria contain measurement bias. Although the latent variable structure for each PD was quite similar across younger and older age groups, some individual criteria were differentially endorsed by younger and older adults with equivalent PD pathology. The presence of measurement bias for these criteria raises questions concerning the assessment of PDs in older adults and the interpretation of existing data.     

An International Assessment of the Emotional and Behavioral Strengths of Youth

The assessment of emotional and behavioral strengths has been identified as an important part of the assessment process for children referred for specialized services. The Behavioral and Emotional Rating Scale-2 (BERS-2; Epstein, Behavioral and Emotional Rating Scale: a strength-based approach to assessment. PRO-Ed, Austin, TX, 2004) was developed as a standardized, norm-referenced instrument to measure strengths of children and youth, and has been found to be a psychometrically sound instrument. We determined the psychometric characteristics of the BERS-2 with a Finnish sample. The BERS-2 was translated in Finnish and 608 Finnish 9th graders filled in the self-evaluation. Results showed that the five subscales of BERS-2 have good reliability and formed a strong strength index. In general, females scored higher than males and special education students scored lower than non-special education students. This study suggests that the translated version of BERS-2 is a valid and reliable instrument in Finland.     

Norm-Referenced Testing and Low-Income Blacks

The purpose of this article is to highlight biases, that is, learning styles and test bias, involved in testing low-income Blacks with norm-referenced, achievement tests and to offer suggestions to assist counselors in improving their students' test performance.     

Assessing the validity of the achievement motive in the presence of random measurement error

The presence of random measurement error in indicators of theoretical constructs biases observed estimates of relations among those constructs. Correcting for this bias is particularly important when random measurement error is substantial, or is substantially different for indicators of distinct constructs included in a theoretical model. Validity assessment in the case of thematic apperceptive measures of the achievement motive (TAT n Achievement) has been vulnerable to interpretive errors because these indicators of the achievement motive are typically much less reliable than indicators of other constructs to which the motive may be related, and no correction has been made for the bias introduced by such differential measurement error. Here we illustrate a causal modeling approach to validity assessment for TAT n Achievement that incorporates explicit true-score measurement models of theoretical constructs. We confirm the hypothesis that the achievement motive construct is positively related to work satisfaction in a representative sample of adult males in the United States, taken in 1976 (N = 413). Evidence for the discriminant validity of story content versus story length, an issue raised in the literature on the TAT, is also presented in this nomological network.     

空间偏向成因研究:理论解释与前景展望

空间偏向意指人们在注意刺激物时普遍具有方向上的偏好.对其成因,目前主要存在“大脑说”和“文化说”两派观点.前者认为大脑的不对称性是决定因素,而后者则认为阅读和书写习惯等文化因素是主要成因.本文在综述空间偏向研究后指出,以上两派学说在解释力上各有得失,但并非完全对立,将二者结合起来的“大脑-文化说”才能更好地解释空间偏向现象.     

青少年学习倦怠量表的多元概化分析

运用多元概化理论考察青少年学习倦怠量表（ASBI）的测量学属性。结果发现,ASBI各因子以及全量表的概化系数和可靠性指数均达到较好水平,既可用于常模参照测验,也可用于标准参照测验;各因子的分量比例还不尽完善,低成就感对全域总分的贡献略低于在全量表中的分值比重,对相对误差方差贡献最大的是低成就感,学业疏离对绝对误差方差的贡献最大;今后ASBI的修订工作,可考虑对学业疏离和低成就感因子条目质量的改进,同时适当增加身心耗竭因子的条目数以提高其测量信度。     

Issues in Advancing Diversity Through Assessment

Five issues that affect assessment with diverse populations are presented and discussed. The “quest for the golden label” issue concerns problems with labels rather than operational measurement. The “three musketeers” issue is the tendency to believe one measure can work equally well for all. The “horizontal research” issue refers to the tendency to do nondevelopmental research on nontraditional groups. Sampling problems are discussed under the “bias is bias” issue. The “I'm okay, you're not” issue is the tendency to train few assessment specialists who are knowledgeable about diversity issues.     

Practical Applications of Generalizability Theory for Designing,Evaluating, and Improving Psychological Assessments

In this article, we illustrate how generalizability theory (G-theory) can extend traditional assessment methods for designing, improving, and evaluating results from both objectively and subjectively scored measures of individual differences. Our illustrations include quantification of multiple sources of measurement error, derivation of unique indexes of consistency for norm- and criterion-referenced interpretations of scores, estimation of score consistency when changing a measurement procedure, and disattenuation of correlation coefficients for measurement error. We also expand G-theory analyses beyond the item level to include parcels and split measures and highlight linkages among G-theory, classical test theory, and structural equation modeling. Computer code and sample data are provided in online supplements to help readers apply the demonstrated techniques to their own assessments.     

The measurement of self-esteem for program evaluation

The purpose of this paper is to examine the area of self-esteem in the professional literature, its relationship to educational programs, its measurement problems, and one approach to program evaluation in terms of students' self-esteem. Self-esteem has been an important topic in the educational literature in recent years. There have been many important contributions by investigators on the relationship between self-esteem and educational achievement. Until recently, the results of self-esteem investigations have yielded a somewhat negative appraisal from reviewers, but there is optimism for the future possibilities of assessing this elusive construct. The investigator focusing upon program evaluation has advantages over his predecessors. He can utilize different measurement strategies. The strengths and criticisms of Q methodology are discussed. The L-L Q-Sort is presented as one approach to the evaluation of self-esteem in educational program evaluation.     

A Positive Perspective on Attentional Bias: Positive Affectivity and Attentional Bias to Positive Information

In recent years there has been growing interest in determining the attentional basis of positive affectivity. Investigators have sought to evaluate the hypothesis that elevated positive affectivity is characterized by an attentional bias that favours the processing of positive information over neutral information. Unfortunately, in the studies carried out to date, the assessment procedures used by researchers to assess such attentional selectivity have either been methodologically limited or methodologically flawed. Hence, the validity of the hypothesis has remained indeterminate. To overcome the problems associated with attentional assessment procedures used in previous studies, and so shed light on the attentional basis of positive affectivity, we employed an appropriate variant of the attentional probe task. Our findings show that participants high in positive affectivity displayed greater relative attention to positive information compared to neutral information, than was the case for their low positive affectivity counterparts. Thus, the present results support the hypothesis that an attentional bias to positive information, relative to neutral information, is characteristic of elevated positive affectivity.     

Assessing Growth in a Diagnostic Classification Model Framework

A common assessment research design is the single-group pre-test/post-test design in which examinees are administered an assessment before instruction and then another assessment after instruction. In this type of study, the primary objective is to measure growth in examinees, individually and collectively. In an item response theory (IRT) framework, longitudinal IRT models can be used to assess growth in examinee ability over time. In a diagnostic classification model (DCM) framework, assessing growth translates to measuring changes in attribute mastery status over time, thereby providing a categorical, criterion-referenced interpretation of growth. This study introduces the Transition Diagnostic Classification Model (TDCM), which combines latent transition analysis with the log-linear cognitive diagnosis model to provide methodology for analyzing growth in a general DCM framework. Simulation study results indicate that the proposed model is flexible, provides accurate and reliable classifications, and is quite robust to violations to measurement invariance over time. The TDCM is used to analyze pre-test/post-test data from a diagnostic mathematics assessment.