Training and Maintaining System-Wide Reliability in Outcome Management

The Child and Adolescent Functional Assessment Scale (CAFAS) is widely used for outcome management, for providing real time client and program level data, and the monitoring of evidence-based practices. Methods of reliability training and the assessment of rater drift are critical for service decision-making within organizations and systems of care. We assessed two approaches for CAFAS training: external technical assistance and internal technical assistance. To this end, we sampled 315 practitioners trained by external technical assistance approach from 2,344 Ontario practitioners who had achieved reliability on the CAFAS. To assess the internal technical assistance approach as a reliable alternative training method, 140 practitioners trained internally were selected from the same pool of certified raters. Reliabilities were high for both practitioners trained by external technical assistance and internal technical assistance approaches (.909–.995, .915–.997, respectively). 1 and 3-year estimates showed some drift on several scales. High and consistent reliabilities over time and training method has implications for CAFAS training of behavioral health care practitioners, and the maintenance of CAFAS as a global outcome management tool in systems of care.     

体外冲击波与传统方法治疗腰椎间盘突出症疗效比较

比较放散式体外冲击波与传统保守治疗腰椎间盘突出症的临床疗效及生命质量评价。对2010年6月至fi013年6月收治且获得随访的257例腰椎间盘突出症患者进行分析,其中男139例,女118例。根据治疗方式不同随机分为两组,并在治疗前后应用日本骨科学会（JOA）下腰痛评分法及视觉模拟评分（VAS）进行疗效评价,应用健康调查评分量表SF-36进行生活质量评价。对患者进行随访1周～12周,平均（10．0±3．4）周,治疗组治疗后2周和4周VAS评分低于对照组,治疗后1周、2周及4周JOA评分高于对照组,治疗后4周、8周和12周的生理机能、生理职能、总体健康、情感职能和躯体疼痛得分高于对照组,差异有统计学意义（P〈0．05）。放散式体外冲击波治疗腰椎间盘突出症疗效较好,并能够较好地提高患者的生活质量。     

Microsaccades keep the eyes' balance during fixation

During fixation of a stationary target, small involuntary eye movements exhibit an erratic trajectory-a random walk. Two types of these fixational eye movements are drift and microsaccades (small-amplitude saccades). We investigated fixational eye movements and binocular coordination using a statistical analysis that had previously been applied to human posture control. This random-walk analysis uncovered two different time scales in fixational eye movements and identified specific functions for microsaccades. On a short time scale, microsaccades enhanced perception by increasing fixation errors. On a long time scale, microsaccades reduced fixation errors and binocular disparity (relative to pure drift movements). Thus, our findings clarify the role of oculomotor processes during fixation.     

The passion scale: Aspects of reliability and validity of a new 8-item scale assessing passion.

In this article, the psychometric properties of a new scale aimed at quantifying passion are explored, i.e. passion related to becoming good or achieving in some area/theme/skill.The Passion Scale was designed to be quantitative, simple to administer, applicable for large-group testing, and reliable in monitoring passion.A total of 126 participants between 18 and 47 years of age (mean age = 21.65, SD = 3.45) completed an assessment of Passion Scale, enabling us to investigate its feasibility, internal consistency, construct validity and test-retest reliability.FeasibilityThe overall pattern of results suggest that the scale for passion presented here is applicable for the age studied (18–47).Internal consistencyAll individual item scores correlated positively with the total score, with correlations ranging from 0.51 to 0.69. The Cronbach's alpha value for the standardized items was 0.86.Construct validityPearson correlations coefficient between total score passion scale and Grit-S scale were 0.39 for adults, mean age 21.23 (SD = 3.45) (N = 107).Test-retest reliability: Intraclass correlation coefficient (ICCs) between test and retest scores for the total score was 0.92.These promising results warrant further development of the passion scale, including normalization based on a large, representative sample.     

Clinical Utility of the Bem Sex Role Inventory (BSRI) in the Spanish Transsexual and Nontranssexual Population

The aim of the study is to evaluate the usefulness of the Bem Sex Role Inventory (BSRI; Bem, 1974), an overall measurement of the cultural construct of masculinity and femininity, in the psychological assessment of Spanish transsexuals. Seventy male-to-female transsexuals (MF), 51 female-to-male transsexuals (FM), 77 control men, and 79 control women completed the Spanish version of the BSRI. Statistically significant differences between groups were only found on the femininity scale, on which MF transsexuals and control women scored significantly higher than FM transsexuals and control men. The results indicate that (a) only the femininity scale of the BSRI appears to be useful today for evaluating differences in the sex-role identification in Spanish controls and transsexuals; and (b) MF and FM transsexuals score as a function of their gender identity instead of their anatomical sex on the BSRI femininity scale.     

Clinical utility of the Bem Sex Role Inventory (BSRI) in the Spanish transsexual and nontranssexual population

The aim of the study is to evaluate the usefulness of the Bem Sex Role Inventory (BSRI; Bem, 1974), an overall measurement of the cultural construct of masculinity and femininity, in the psychological assessment of Spanish transsexuals. Seventy male-to-female transsexuals (MF), 51 female-to-male transsexuals (FM), 77 control men, and 79 control women completed the Spanish version of the BSRI. Statistically significant differences between groups were only found on the femininity scale, on which MF transsexuals and control women scored significantly higher than FM transsexuals and control men. The results indicate that (a) only the femininity scale of the BSRI appears to be useful today for evaluating differences in the sex-role identification in Spanish controls and transsexuals; and (b) MF and FM transsexuals score as a function of their gender identity instead of their anatomical sex on the BSRI femininity scale.     

Effects of Goal Type on Performance,Task Interest,and Affect Over Time1

The purpose of this study was to increase understanding regarding the effects of goal-setting practices on a wide range of dependent variables over time. Goal type (control group, quality goal, quantity goal, and dual goal) was manipulated across 3 time periods. Goal × Time interactions were predicted such that quantity, effort, task interest, and positive affect would initially be low, but by later trials would be higher in the quantity and dual-goal conditions compared to the quality goal and control groups. Quality was expected to be greatest in the quality goal condition and to be relatively constant over time. Participants (N= 80) were undergraduates at a large midwestern university. Results indicated some support for our hypotheses in that Goal × Time interactions were found for quantity produced, rated effort, task interest, and positive affect.     

Time, action, and consciousness

Time plays a central role in consciousness, at different levels and in different aspects of information processing. Subliminal perception experiments demonstrate that stimuli presented too briefly to enter conscious awareness are nevertheless processed to some extent. Implicit learning, implicit memory, and conditioning studies suggest that the extent to which memory traces are available for verbal report and for cognitive control is likewise dependent on the time available for processing during acquisition. Differences in the time available for processing also determine not only the extent to which one becomes conscious of action, but also provides the basis for making attributions of authorship to experienced acts. In this paper, we offer a brief overview of these different findings and suggest that they can all be understood based on the fact that consciousness takes time. From this perspective, the availability of representations to conscious awareness depends on the quality of these representations - the extent to which they are strong, stable in time, and distinctive. High-quality representations occur when processes of global competition have had sufficient time to operate so as to make the system settle into the best possible interpretation of the input. Such processes implement global constraint satisfaction and critically depend on reentrant processing, through which representations can be further enriched by high-level constraints. We discuss these ideas in light of current theories of consciousness, emphasizing the fact that consciousness should be viewed as a process rather than as a static property associated with some states and not with others.     

Harmonic Regression and Scale Stability

Monitoring a very frequently administered educational test with a relatively short history of stable operation imposes a number of challenges. Test scores usually vary by season, and the frequency of administration of such educational tests is also seasonal. Although it is important to react to unreasonable changes in the distributions of test scores in a timely fashion, it is not a simple matter to ascertain what sort of distribution is really unusual. Many commonly used approaches for seasonal adjustment are designed for time series with evenly spaced observations that span many years and, therefore, are inappropriate for data from such educational tests. Harmonic regression, a seasonal-adjustment method, can be useful in monitoring scale stability when the number of years available is limited and when the observations are unevenly spaced. Additional forms of adjustments can be included to account for variability in test scores due to different sources of population variations. To illustrate, real data are considered from an international language assessment.     

The efficiency of behavior rating scales to assess inattentive-overactive and oppositional-defiant behaviors: applying generalizability theory to streamline assessment

Although the efficiency with which a wide range of behavioral data can be obtained makes behavior rating scales particularly attractive tools for the purposes of screening and evaluation, feasibility concerns arise in the context of formative assessment. Specifically, informant load, or the amount of time informants are asked to contribute to the assessment process, likely has a negative impact on the quality of data over time and the informant's willingness to participate. Two important determinants of informant load in progress monitoring are the length of the rating scale (i.e., the number of items) and how frequently informants are asked to provide ratings (i.e., the number of occasions). The purpose of the current study was to investigate the dependability of the IOWA Conners Teacher Rating Scale (Loney & Milich, 1982), which is used to differentiate inattentive-overactive from oppositional-defiant behaviors. Specifically, the facets of items and occasions were examined to identify combinations of these sources of error necessary to reach an acceptable level of dependability for both absolute and relative decisions. Results from D studies elucidated a variety of possible item-occasion combinations reaching the criteria for adequate dependability. Recommendations for research and practice are discussed.     

Establishing Measurement Equivalence and Invariance in Longitudinal Data With Item Response Theory

If measurement invariance does not hold over 2 or more measurement occasions, differences in observed scores are not directly interpretable. Golembiewski, Billingsley, and Yeager (1976) identified 2 types of psychometric differences over time as beta change and gamma change. Gamma change is a fundamental change in thinking about the nature of a construct over time. Beta change can be described as respondents' change in calibration of the response scale over time. Recently, researchers have had considerable success establishing measurement invariance using confirmatory factor analytic (CFA) techniques. However, the use of item response theory (IRT) techniques for assessing item parameter drift can provide additional useful information regarding the psychometric equivalence of a measure over time that is not attainable with traditional CFA techniques. This article marries the terminology commonly used in CFA and IRT techniques and illustrates real advantages for identifying beta change over time with IRT methods rather than typical CFA methods, utilizing a longitudinal assessment of job satisfaction as an example.     

学习判断与学习时间分配的关系

学习判断与学习时间分配的关系是元认知领域的重要问题, 对学习判断的研究在于使个体了解自己的学习和记忆活动, 从而有效控制学习过程, 合理分配学习时间, 这是元记忆监测影响控制的典型例子。近来有研究者提出学习时间分配不仅表现控制功能, 而且还表现监测功能。于是围绕学习时间的双重功能, 他们提出了元记忆监测与控制至少存在两种关系模型—— MC(监测影响控制)模型和CM(控制影响监测)模型, 后来又提出了两模型相互协同作用的合并动态模型—— 连续波浪型模式和同时发生模型。未来的研究应在继续验证MC模型和完善CM模型的基础上, 尝试检验两模型的合并动态模型。     

Hierarchical Bayes Models for Response Time Data

Human response time (RT) data are widely used in experimental psychology to evaluate theories of mental processing. Typically, the data constitute the times taken by a subject to react to a succession of stimuli under varying experimental conditions. Because of the sequential nature of the experiments there are trends (due to learning, fatigue, fluctuations in attentional state, etc.) and serial dependencies in the data. The data also exhibit extreme observations that can be attributed to lapses, intrusions from outside the experiment, and errors occurring during the experiment. Any adequate analysis should account for these features and quantify them accurately. Recognizing that Bayesian hierarchical models are an excellent modeling tool, we focus on the elaboration of a realistic likelihood for the data and on a careful assessment of the quality of fit that it provides. We judge quality of fit in terms of the predictive performance of the model. We demonstrate how simple Bayesian hierarchical models can be built for several RT sequences, differentiating between subject-specific and condition-specific effects.     

Self‐reported attitude scales: current practice in adequate assessment of reliability,validity, and dimensionality

The development of methods to create self‐reported attitude scales has lost momentum, in part because of increased research focused on implicit measures. This paper reviews 162 papers on methodological approaches applied to the validation and assessment of attitude scales. Assessment of methodological approaches applied indicates that neither reliability, validity, nor dimensionality assessments are consistently used according to standard operating procedures or in accordance with best practice. Within current practices in the field of attitude scale development, the full potential of self‐report scales is not met, in part because of such methodological issues. The improvement of existing practices and adoption of promising new developments in attitude scale construction and evaluation are discussed, together with recommendations for best practice in scale validation.     

论个人的时间管理倾向

时间管理倾向是一种具有动力性的人格特征,它由时间价值感、时间监控观和时间效能感构成。善于管理时间者的生活质量是高的。时间管理技能可以通过训练得到提高。     

Item response theory and the measurement of clinical change

An instrument's sensitivity to detect individual-level change is an important consideration for both psychometric and clinical researchers. In this article, we develop a cognitive problems measure and evaluate its sensitivity to detect change from an item response theory (IRT) perspective. After illustrating assumption checking and model fit assessment, we detail 4 features of IRT modeling: (a) the scale information curve and its relation to the bandwidth of measurement precision, (b) the scale response curve and how it is used to link the latent trait metric with the raw score metric, (c) content-based versus norm-based score referencing, and (d) the level of measurement of the latent trait scale. We conclude that IRT offers an informative, alternative framework for understanding an instrument's psychometric properties and recommend that IRT analyses be considered prior to investigations of change, growth, or the effectiveness of clinical interventions.     

Are ratings in the eye of the beholder? A non-technical primer on many facet Rasch measurement to evaluate rater effects on teacher behavior rating scales

Rater-mediated assessments, such as teacher behavior rating scales, measure student behavior indirectly through the lens of a rater. As a result, scores from rater-mediated assessments can be influenced by rater effects— individual differences in rater perspectives, attitudes, beliefs, and interpretation of rating scale items. Rater effects are a fundamental aspect of all rater-mediated assessments. However, traditional approaches to evaluate rater effects (i.e., classical test theory, generalizability theory, and multilevel modeling) merely estimate how much score variability is due to the rater. These approaches, while informative, do not offer a solution to the problem. In contrast, Many-facet Rasch measurement (MFRM) approaches estimate and control for rater effects in rater-mediated assessments so that scores are adjusted to account for rater variability. Thus, MFRM offers unique insights into individual- and group-level rater effects that can be used to inform a solution. The resultant purpose of this paper is to introduce MFRM, discuss its advantages for evaluating rater effects in rater-mediated assessments, and demonstrate its use through an applied example.     

Investigating Separate and Concurrent Approaches for Item Parameter Drift in 3PL Item Response Theory Equating

This study examines separate and concurrent approaches to combine the detection of item parameter drift (IPD) and the estimation of scale transformation coefficients in the context of the common item nonequivalent groups design with the three-parameter item response theory equating. The study uses real and synthetic data sets to compare the two approaches based on IPD flagging rates, type I error and power rates, and recovery of scale transformation coefficients. Results indicate that the two approaches render similar outcomes with stable anchor sets. However, they can produce dissimilar results with unstable anchor sets because of differences in the performance of their IPD components. Further, the findings of this study caution about working backward from equated cut scores to motivate the selection of an anchor set.     

义务教育阶段学业质量监测题库建设的理论与实践

义务教育阶段学业质量监测是学生学业质量评价的重要方式,在当前国内外教育质量评价领域中发挥着极为重要的作用.加强学业质量监测的题库建设,则是促进学业质量监测工具的命制与实施从经验型向科学化迈进的必由之路.本文在对义务教育阶段学业质量监测题库的性质和功能进行阐述的基础上,重点探讨了学业质量监测题库建设的理论基础、基本流程和关键环节,并进一步分析指出了当前题库建设中有待深入研究的问题.