Assessment of Score Dependability of the Wisconsin Schizotypy Scales Using Generalizability Analysis

To investigate the reliability of the Wisconsin Schizotypy Scales, this study applied generalizability analysis with two college student samples who completed the scales at two time points. The results indicated that the Revised Social Anhedonia Scale had acceptable levels of score dependability, but that the score dependability for the other scales (the Physical Anhedonia Scale, the Perceptual Aberration Scale, and the Magical Ideation Scale) was below an acceptable level of .80. The patterns of variance components suggested that the scales’ items need improvement. Researchers can use the included tables to choose the number of items and occasions needed to get dependable score interpretations.     

A FIELD STUDY OF THE EFFECTS OF RATING PURPOSE ON THE QUALITY OF MULTISOURCE RATINGS

Using a field sample of peers and subordinates, the current study employed generalizability theory to estimate sources of systematic variability associated with both developmental and administrative ratings (variance due to items, raters, etc.) and then used these values to estimate the dependability (i.e., reliability) of the performance ratings under various conditions. Results indicated that the combined rater and rater-by-ratee interaction effect and the residual effect were substantially larger than the person effect (i.e., object of measurement) for both rater sources across both purpose conditions. For subordinates, the person effect accounted for a significantly greater percentage of total variance in developmental ratings than in administrative ratings; however, no differences were observed for peer ratings as a function of rating purpose. These results suggest that subordinate ratings are of significantly better quality when made for developmental than for administrative purposes, but the same is not true for peer ratings.     

Toward a dependable measure of reflection-impulsivity: An application of generalizability theory

Generalizability theory was applied to the Matching Familiar Figures Test (MFF) to analyze the dependability of the MFF as a measure of reflection-impulsivity at four grade levels: second, third, fourth, and fifth. A completely crossed, two-facet random model design was used to provide a multidimensional framework for examining the dependability of the MFF. Components of variance and coefficients of generalizability were derived from this design for the MFF error and latency scores at each grade level. Results showed that the MFF latency score was a more dependable measure than the MFF error score. In addition, the number of testing occasions made a more significant contribution to the generalizability of the MFF than the number of items. Coefficients of generalizability based on extrapolated items and occasions were also computed, providing the basis for improving the dependability of the MFF in future research. Overall, results indicate that the traditional method of allowing multiple trials for each item contributes to the imprecision of the MFF error score. An alternative procedure for administering the MFF is recommended.Portions of this article were presented at the annual convention of the American Educational Research Association, Toronto, Canada, March 1978. At various times during the conduct and completion of the present article, the first author was affiliated with the Los Angeles Unified School District and the University of California, Los Angeles.     

"青少年学生生活满意度量表"的概化理论研究

概化理论是用统计调整技术分析测量误差的一种测量理论,它侧重于从宏观方面讨论实测时的测量条件与结论推广应用范围之间的关系来探讨测量的外部效度问题。本文用概化理论对青少年学生生活满意度量表(CASLSS)进行了研究,得到以下研究结果:(1)对于生活满意度的维度数目,就我国青少年学生而言取6到8个维度较为合适,当对CASLSS取2个维度时,CASLSS只适合作常模参照性解释,而不适合作标准参照性解释;(2)CASLSS的各分量表和总量表的信度较高,且它不仅可以作常模参照性解释,还适合作标准参照性解释;(3)CASLSS的环境满意度因子相对其它五个因子而言,量表特性稍差,是今后改进该量表的主要方向。CASLSS无论是各个因子还是总量表均具有非常优良的量表特性,值得在实际的工作和研究中加以推广应用。     

Does calling it “Morgan's way” reduce student learning? Evaluating the effect of person-presentation during comparison and discussion of worked examples in mathematics classrooms

Mathematics textbooks sometimes present worked examples as being generated by particular fictitious students (i.e., person-presentation). However, there are indicators that person-presentation of worked examples may harm generalization of the presented strategies to new problems. In the context of comparing and discussing worked examples during extended classroom instruction, the current study compared the impact of person-presentation to strategy labels on students' posttest accuracy and ratings of strategy generalizability. Five algebra teachers and their 168 students used worked examples either presented using fictitious students or with a strategy label during a multiweek unit on equation solving, with teachers randomly assigned to condition. All students compared and discussed the worked examples. In this context, we found no effect of condition on student accuracy at posttest, nor on their ratings of the generalizability of the presented strategies. We discuss why previously found negative effects of person-presentation may not have extended to this context.     

Examination performance in lecture-discussion and personalized instruction courses

Students enrolled in a Psychology of Learning course were assigned to either a lecture section, one of two similar personalized instruction sections, or a fourth section that rotated across all three teaching procedures. All students took identical midterms and a final examination. After correcting test performance for differences in the cumulative grade point average of students in the four sections, examination performance of students in the personalized sections was found to be superior to that of students in the lecture section. An analysis of class section examination performance by item type revealed that students in the lecture section scored lower on all item types, but the greatest differences occurred on items that required written responses (essay and fill-in items) rather than recognition responses (multiple choice items). A gross analysis of student performance in the class rotated across the instructional procedures suggests that personalized instruction had its greatest impact on students with "average" to "poor" academic records.     

概化理论缺失数据方差分量估计

各种心理调查、心理实验中, 数据的缺失随处可见。由于数据缺失, 给概化理论分析非平衡数据的方差分量带来一系列问题。基于概化理论框架下, 运用Matlab 7.0软件, 自编程序模拟产生随机双面交叉设计p×i×r缺失数据, 比较和探讨公式法、REML法、拆分法和MCMC法在估计各个方差分量上的性能优劣。结果表明：(1) MCMC方法估计随机双面交叉设计p×i×r缺失数据方差分量, 较其它3种方法表现出更强的优势; (2) 题目和评分者是缺失数据方差分量估计重要的影响因素。     

The efficiency of behavior rating scales to assess inattentive-overactive and oppositional-defiant behaviors: applying generalizability theory to streamline assessment

Although the efficiency with which a wide range of behavioral data can be obtained makes behavior rating scales particularly attractive tools for the purposes of screening and evaluation, feasibility concerns arise in the context of formative assessment. Specifically, informant load, or the amount of time informants are asked to contribute to the assessment process, likely has a negative impact on the quality of data over time and the informant's willingness to participate. Two important determinants of informant load in progress monitoring are the length of the rating scale (i.e., the number of items) and how frequently informants are asked to provide ratings (i.e., the number of occasions). The purpose of the current study was to investigate the dependability of the IOWA Conners Teacher Rating Scale (Loney & Milich, 1982), which is used to differentiate inattentive-overactive from oppositional-defiant behaviors. Specifically, the facets of items and occasions were examined to identify combinations of these sources of error necessary to reach an acceptable level of dependability for both absolute and relative decisions. Results from D studies elucidated a variety of possible item-occasion combinations reaching the criteria for adequate dependability. Recommendations for research and practice are discussed.     

青少年学习倦怠量表的多元概化分析

运用多元概化理论考察青少年学习倦怠量表（ASBI）的测量学属性。结果发现,ASBI各因子以及全量表的概化系数和可靠性指数均达到较好水平,既可用于常模参照测验,也可用于标准参照测验;各因子的分量比例还不尽完善,低成就感对全域总分的贡献略低于在全量表中的分值比重,对相对误差方差贡献最大的是低成就感,学业疏离对绝对误差方差的贡献最大;今后ASBI的修订工作,可考虑对学业疏离和低成就感因子条目质量的改进,同时适当增加身心耗竭因子的条目数以提高其测量信度。     

Between,Within, and Total Group Factor Analyses of Student Ratings of Instruction

The purpose of this study was to compare the factor structures of student ratings of instruction resulting from total group, between group, and within group analyses. Six factors obtained from responses by 9700 students to 31 items were used to approximate the between group covariance matrix based on 43'7 classroom means and the pooled within classroom covariance matrix. A good approximation to both the between and within group covariance matrices was obtained.     

Self-labeling by educably mentally retarded high school students in their mainstream and special education classes

Educably mentally retarded (EMR) students and nonhandicapped students from their mainstream classes completed semantic differential ratings of a stereotypic popular teen-ager, juvenile delinquent, and special education student. Subjects also rated their global self-concepts and situation-specific self-concepts within the mainstream and the special class settings. Ratings were done at the beginning (Time 1) and the end (Time 2) of a semester. Results showed that mainstream classes did not "de-label" EMR students. Within the mainstream class, the number of EMR students who saw themselves as similar to a special education student significantly increased over time. Moreover, at Time 2, EMR students were more likely to think of themselves as similar to a special education student in their mainstream class than in their special education class. However, EMR subjects' global self-concepts did not change. Implications for mainstreaming are discussed.     

Intergenerational transmission of literacy skills among Filipino families

We examined the joint role of parental word reading skills and conventional home literacy environment measures among 320 Filipino low‐ to middle‐income families in Cebu City, Philippines with children aged 5–8 years old. A ranking of parent‐reported ratings of their frequency of engaging in home literacy activities and adult literacy practices revealed that book‐related behaviors were less frequently practiced relative to other behaviors, and mean ratings on the home literacy resources scale suggested a relatively print‐poor environment. Nevertheless, scale items about book reading and direct literacy instruction at home correlated with child's language and literacy skills. Structural equation modeling showed that parent's education and frequency of engaging in home literacy activities uniquely accounted for variance in child's oral language and print knowledge skills. In a second model, parent's word reading skills were significantly related to child's skills, but did not eliminate or attenuate influences from parent's education and home literacy activities. Results are important in relation to theories on the intergenerational transmission of literacy skills and the generalizability of findings from developed countries to developing country contexts.     

高校教师教学水平评价概化理论预算限制下最佳样本量估计

使用“高校教师教学水平评价问卷”,要求566名学生对19名教师进行评价,对收集到的数据作不同的概化设计,包括t×i、（s：t）×i、（s：t）×（i：v）和（s：t）×（i：v）×o四种设计。基于概化理论,结合预算限制,统一LaGrange乘法公式,自行推导不同设计的最佳样本量公式,联合估计的方差分量,计算出不同设计的最佳样本量。结果表明：（1）LaGrange乘法统一公式表现出较强的通用性,能够适用于预算限制下各种概化设计;（2）评价场合是影响高校教师教学水平评价一个相当重要的因素;（3）（s：t）×（i：v）×o是高校教师教学水平评价概化理论预算限制下最优概化设计;（4）高校教师教学水平评价概化理论预算限制下,每位教师最佳评价学生人数为20人,每个维度最佳评价题目数为3题。     

Using response cards to increase student participation in an elementary classroom

The use of response cards during large-group social studies instruction was evaluated in a fourth-grade classroom. The experiment consisted of two conditions, hand raising and write-on response cards, alternated in an ABAB design. During baseline, the teacher called upon 1 student who had raised his or her hand in response to the teacher's question. During the response-card condition, each student in the class was provided with a white laminated board on which to write one- or two-word answers in response to each question asked by the teacher. Rate of active student response during instruction was much higher with response cards than with hand raising. Most students scored higher on daily quizzes following sessions in which response cards were used than they did on quizzes that followed hand-raising sessions. Response cards were preferred over hand raising by 19 of the 20 students in the class.     

Familiarity and sex based stereotypes on instant impressions of male and female faculty

To address the stranger-to-stranger critique of stereotyping research, psychology students $(n= 139)$ and law students $(n = 58)$ rated photographs of familiar or unfamiliar male or female professors on competence. Results from Study 1 indicated that familiar male psychology faculty were rated as more competent than were familiar female faculty, whereas unfamiliar female faculty were rated as more competent than unfamiliar male faculty. By contrast, in Study 2, familiarity had a stronger positive effect on competence ratings of female faculty than it did for male faculty. Among psychology students, familiarity increased sex bias against female faculty, whereas among law students familiarity decreased sex bias. Together, these studies call into question the stranger-to-stranger critique of stereotyping research. Our findings have direct implications for the context of student evaluations. In male-dominated disciplines it is important for students to be exposed to female instructors in order to reduce pre-existing biases against such instructors.     

The Effects of Using Response Cards on Student Participation, Academic Achievement, and On-Task Behavior During Whole-Class, Math Instruction

This study evaluated the effects of using response cards during whole-group math instruction in a fourth-grade classroom, using an ABA research design. During both A and B conditions the classroom teacher conducted her math lessons as planned. However during the A conditions, hand-raising (HR) was in effect. That is, when the teacher asked a question she instructed the students to raise their hands and after each question she called on one student whose hand was raised. During the B condition, response cards (RC) were in effect. Whenever she asked a question, the teacher instructed the entire class to write their answers on individual response cards. After each question, the teacher instructed the entire class to hold up their RC. Data were collected on five students who represented the class range in general level of participation, academic skills, and on-task behavior. Results indicated that student participation, specifically the number of student-initiated opportunities to respond and the number of student responses were higher when using RC than when HR was in effect. Students scored higher on the weekly math quiz after the RC condition than after the HR conditions. The percentage of intervals that students were on-task also was higher when RC were used.     

概化理论在内科学临床技能考核中的应用研究

运用多元概化理论对两届临床医学硕士研究生内科临床实践能力考核进行评价比较。结果表明,两届研究生内科临床实践能力考核的信度均较高,可靠性指数分别为0.78878、0.67985,考核内容较全面。比较发现,01级学生考核的信度要高于02级,考核专家以3-5位为宜。     

A Multivariate Analysis of Selected Class Characteristic and Student Rating of Instruction

This study examined the relationship between selected class characteristics and student ratings of instructors. A large number of classes (N = 1247)and students (over 33,000) a t a large Midwestern University provided the data for this study. The results indicated that the class characteristics that had the strongest influence on the results of instructor ratings were the grades expected by students and the percentage of students in the class taking the course as an elective.     

Boys Don’t Work? On the Psychological Benefits of Showing Low Effort in High School

Male students show less academic effort and lower academic achievement than do female students. The present study aimed to shed more light on the reasons for why male students show low academic effort despite the finding that this undermines their academic achievement. We explored whether students experience psychological benefits from showing low effort or “effortless” achievement in school and whether these benefits are greater for male than for female students. In two experimental vignette studies with independent samples of German ninth graders (N?=?210) and teachers (N?=?176), we systematically varied student targets’ gender, effort, and achievement and tested for effects on targets’ ascribed intelligence, popularity, likeability, masculinity, femininity, and gender-typicality. The “effortless” achiever was rated as more popular than students showing high effort. Teachers perceived the effortless achiever as the most intelligent target. Academic effort further increased students’ ratings of a low-achieving target’s likeability and students’ and teachers’ ratings of all targets’ femininity as well as decreased students’ ratings of all targets’ masculinity. Students and teachers perceived targets showing low (vs. high) effort as more similar to a typical boy, whereas teachers perceived targets showing high (vs. low) effort as more similar to a typical girl. Results indicate a need to understand the psychological benefits of low academic engagement, especially for male students, and to address the feminine stereotyping of (academic) effort.