A strict response criterion yields a mirror effect in the novelty paradigm

According to the novelty/encoding hypothesis (NEH; Tulving & Kroll, 1995), efficacy of encoding information into long-term memory depends on the movelty of the information. Recognition accuracy is higher for novel than for previously familiarized material. This novelty effect is not a mirror effect: the superiority of novel over familiar items is not found in the hit rates but only in the false-alarm rates. The main result in the present replication study was that novel hit rates were higher than familiar ones when the most confident responses were examined separately, and thus a mirror effect could be demonstrated for these data, for both the low- and the high-frequency words. Similarly, the word-frequency effect on hits was stronger when a stricter response criterion was applied. It was concluded that the novelty effect and the word-frequency effect are more similar to one another than has hitherto been thought.     

Interval estimation for the intraclass correlation in dirichlet-multinomial data

When the underlying distribution is discrete with a limited number of categories, methods for interval estimation of the intraclass correlation which assume normality are theoretically inadequate for use. On the basis of large sample theory, this paper develops an asymptotic closed-form interval estimate of the intraclass correlation for the case where there is a natural score associated with each category. This paper employs Monte Carlo simulation to demonstrate that when the underlying intraclass correlation is large, the traditional interval estimator which assumes normality can be misleading. We find that when the number of classes is 20, the interval estimator proposed here can generally perform reasonably well in a variety of situations. This paper further notes that the proposed interval estimator is invariant with respect to a linear transformation. When the data are on a nominal scale, an extension of the proposed method to account for this case, as well as a discussion on the relationship between the intraclass correlation and a kappa-type measure defined here and on the limitation of the corresponding kappa-type estimator are given.The authors wish to thank the Editor, the Associate Editor, and the three referees for many valuable comments and suggestions to improve the clarity of this paper. The works for the first, the third, and the fourth authors were partially supported by grant #R01AR43025-01 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases.     

Approximate interval estimation for a certain intraclass correlation coefficient

When the raters participating in a reliability study are a random sample from a larger population of raters, inferences about the intraclass correlation coefficient must be based on the three mean squares from the analysis of variance table summarizing the results: between subjects, between raters, and error. An approximate confidence interval for the parameter is presented as a function of these three mean squares.Dr. Fleiss is also with the Biometrics Research Unit of the New York State Psychiatric Institute. This work was supported in part by grant DE 04068 from the National Institute of Dental Research.     

认知诊断模型Q矩阵修正：完整信息矩阵的作用

Q矩阵是CDM的核心元素之一,反映了测验的内部结构和内容设计,通常由领域专家根据经验进行主观界定,因此需要对可能存在的错误进行修正。本研究提出了一种新的Q矩阵修正方法——基于完整经验交叉相乘信息矩阵的Wald-XPD方法。采用Monte Carlo模拟检验了新方法的表现,并与同类方法进行了比较。研究表明：新开发的Wald-XPD方法在Q矩阵恢复率、保留正确标定属性的比例以及修正错误标定属性的比例这3个主要指标上均有较好的表现,且整体上优于其他方法,尤其是在修正错误标定的属性方面。通过实证数据展示了Wald-XPD方法在Q矩阵修正中的良好表现。总之,本研究为Q矩阵修正提供了有效的方法。     

Early childhood educator outcomes from online professional development for physical literacy: A randomised controlled trial

IntroductionEarly childhood is recognised as a critical window of opportunity for physical literacy development, however early childhood educators typically lack the training required to effectively provide appropriate physical literacy opportunities for children. We examined the effects of an online physical literacy professional development program—relative to continuing with ‘standard’ practice—on early childhood educators’ physical literacy knowledge and application.MethodsWe conducted a parallel two-arm randomised controlled trial, in which 88 early childhood educators were randomly assigned to an online professional development program designed to support educators’ physical literacy instructional skills (intervention arm; n = 37), or a ‘standard practice’ control condition (n = 51). Data were collected prior to and after the four-week intervention period. We measured educators’ physical literacy knowledge and application (our primary outcome) through independent coding of open-ended survey responses, and educators’ self-reported perceptions of values, confidence, behaviours, and barriers (secondary outcomes). Between-group differences were assessed through analysis of covariance.ResultsOne intervention arm participant withdrew from the study, resulting in 87 participants included in analysis. Educators in the intervention arm scored significantly higher on post-intervention physical literacy knowledge (d = 0.62) and application (d = 0.33) than those in the control arm. Educators in the intervention arm also scored significantly higher than controls on confidence in teaching physical activity (d = 0.42) and significantly lower than controls on perceived personal barriers to physical activity (d = 0.53). Thirteen participants in the intervention arm (36%) did not begin the online professional development program.ConclusionImprovements in physical literacy instructional outcomes indicate the potential for further investigation into broader implementation of online professional development programs of this nature in the future.     

Order effects and memory for evidence in individual versus group decision making in auditing

Sequential processing of evidence may lead to recency effect, a potential bias in judgment. The present research seeks to extend the literature on recency effects by assessing the potential moderating influence of team work: whether group decision making moderates the severity of recency effects predicted by Hogarth and Einhorn (1992), and whether group processing influences the accuracy of, and confidence in memory for evidence. Experienced auditors from a Big‐6 accounting firm made audit judgments, either individually or as groups. They were randomly assigned to one of two levels of evidence presentation order. After performing the judgment task, participants completed two evidence recognition tests. Consistent with prior findings, recency effects on judgments were observed, but only for individuals. Group judgments or audit reports were not affected by recency. Order effects, however, did not translate into different choices of audit reports, and did not persist in memories of either individuals or groups. As expected, group memory was more accurate than individual memory and groups were more confident than individuals. Overall, confidence in accurate memories was greater than in inaccurate ones. Copyright © 1999 John Wiley & Sons, Ltd.     

Computer self efficacy and gender: a cross cultural study of Scotland and Romania

Research has indicated that in West Europe males on average are often found to be more experienced about computing than females, and to have more positive attitudes and self efficacy (self confidence) towards computing than females. Research has also shown that some East Europe countries have had relatively greater female involvement with technology, engineering and computing than West Europe. 200 Romanian and 148 Scottish students completed a computer self efficacy scale. Overall males were more confident than females in advanced and file and software computer skills, but of similar confidence for beginning skills. Scots of both genders were more confident than their Romanian equivalents for beginning computer skills, whilst Romanians were more confident for advanced computer skills. The 3 elements of the self efficacy scale — beginning, advanced and file and software skills — correlated positively with each other. The results were interpreted as showing both similarity and difference in computing self efficacy between Scotland and Romania.     

Stepwise variable selection in factor analysis

It is very important to choose appropriate variables to be analyzed in multivariate analysis when there are many observed variables such as those in a questionnaire. What is actually done in scale construction with factor analysis is nothing but variable selection.In this paper, we take several goodness-of-fit statistics as measures of variable selection and develop backward elimination and forward selection procedures in exploratory factor analysis. Once factor analysis is done for a certain numberp of observed variables (thep-variable model is labeled the current model), simple formulas for predicted fit measures such as chi-square, GFI, CFI, IFI and RMSEA, developed in the field of the structural equation modeling, are provided for all models obtained by adding an external variable (so that the number of variables isp + 1) and for those by deleting an internal variable (so that the number isp – 1), provided that the number of factors is held constant.A programSEFA (Stepwise variable selection in Exploratory Factor Analysis) is developed to actually obtain a list of the fit measures for all such models. The list is very useful in determining which variable should be dropped from the current model to improve the fit of the current model. It is also useful in finding a suitable variable that may be added to the current model. A model with more appropriate variables makes more stable inference in general.The criteria traditionally often used for variable selection is magnitude of communalities. This criteria gives a different choice of variables and does not improve fit of the model in most cases.The URL of the programSEFA is http://koko15.hus.osaka-u.ac.jp/~harada/factor/stepwise/.     

The use of item scores and response times to detect examinees who may have benefited from item preknowledge

According to Wollack and Schoenig (2018, The Sage encyclopedia of educational research, measurement, and evaluation. Thousand Oaks, CA: Sage, 260), benefiting from item preknowledge is one of the three broad types of test fraud that occur in educational assessments. We use tools from constrained statistical inference to suggest a new statistic that is based on item scores and response times and can be used to detect examinees who may have benefited from item preknowledge for the case when the set of compromised items is known. The asymptotic distribution of the new statistic under no preknowledge is proved to be a simple mixture of two χ² distributions. We perform a detailed simulation study to show that the Type I error rate of the new statistic is very close to the nominal level and that the power of the new statistic is satisfactory in comparison to that of the existing statistics for detecting item preknowledge based on both item scores and response times. We also include a real data example to demonstrate the usefulness of the suggested statistic.