Bayesian inference for finite mixtures of generalized linear models with random effects

We present an hierarchical Bayes approach to modeling parameter heterogeneity in generalized linear models. The model assumes that there are relevant subpopulations and that within each subpopulation the individual-level regression coefficients have a multivariate normal distribution. However, class membership is not known a priori, so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions. This approach combines the flexibility of semiparametric, latent class models that assume common parameters for each sub-population and the parsimony of random effects models that assume normal distributions for the regression parameters. The number of subpopulations is selected to maximize the posterior probability of the model being true. Simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub-populations. An application is presented concerning preferences for various aspects of personal computers.     

A Finite Mixture of Nonlinear Random Coefficient Models for Continuous Repeated Measures Data

Nonlinear random coefficient models (NRCMs) for continuous longitudinal data are often used for examining individual behaviors that display nonlinear patterns of development (or growth) over time in measured variables. As an extension of this model, this study considers the finite mixture of NRCMs that combine features of NRCMs with the idea of finite mixture (or latent class) models. The efficacy of this model is that it allows the integration of intrinsically nonlinear functions where the data come from a mixture of two or more unobserved subpopulations, thus allowing the simultaneous investigation of intra-individual (within-person) variability, inter-individual (between-person) variability, and subpopulation heterogeneity. Effectiveness of this model to work under real data analytic conditions was examined by executing a Monte Carlo simulation study. The simulation study was carried out using an R routine specifically developed for the purpose of this study. The R routine used maximum likelihood with the expectation–maximization algorithm. The design of the study mimicked the output obtained from running a two-class mixture model on task completion data.     

Dispersion-weighted kappa: An integrative framework for metric and nominal scale agreement coefficients

The rater agreement literature is complicated by the fact that it must accommodate at east two different properties of rating data: the number of raters (two versus more than two) and the rating scale level (nominal versus metric). While kappa statistics are most widely used for nominal scales, intraclass correlation coefficients have been preferred for metric scales. In this paper, we suggest a dispersion-weighted kappa framework for multiple raters that integrates some important agreement statistics by using familiar dispersion indices as weights for expressing disagreement. These weights are applied to ratings identifying cells in the traditional inter-judge contingency table. Novel agreement statistics can be obtained by applying less familiar indices of dispersion in the same wayThis revised article was published online in August 2005 with the PDF paginated correctly.     

等级反应多水平评分者漂移模型的构建

对于评定耗时较长的测验来说,时间因素对评分精确性的影响不容忽视,因此,评分者漂移方面的研究备受关注。研究基于康春花,孙小坚和曾平飞(2016)提出的等级反应多水平侧面模型建构出可用于检测评分者漂移的等级反应多水平评分者漂移模型,并通过模拟研究对模型性能进行验证。结果表明：模型能够精确估计项目和能力参数;且与固定效应模型相比,评分者随机效应模型能更有效地检测出评分者漂移效应,随机效应模型的有效性和稳定性更佳。     

Rater bias in the EASI temperament scales: a twin study

Under trait theory, ratings may be modeled as a function of the temperament of the child and the bias of the rater. Two linear structural equation models are described, one for mutual self- and partner ratings, and one for multiple ratings of related individuals. Application of the first model to EASI temperament data collected from spouses rating each other shows moderate agreement between raters and little rating bias. Spouse pairs agree moderately when rating their twin children, but there is significantly rater bias, with greater bias for monozygotic than for dizygotic twins. MLE's of heritability are approximately .5 for all temperament scales with no common environmental variance. Results are discussed with reference to trait validity, the person-situation debate, halo effects, and stereotyping. Questionnaire development using ratings on family members permits increased rater agreement and reduced rater bias.     

TRAIT, RATER AND LEVEL EFFECTS IN 360-DEGREE PERFORMANCE RATINGS

Method and trait effects in multitrait-multirater (MTMR) data were examined in a sample of 2,350 managers who participated in a developmental feedback program. Managers rated their own performance and were also rated by two subordinates, two peers, and two bosses. The primary purpose of the study was to determine whether method effects are associated with the level of the rater (boss, peer, subordinate, self) or with each individual rater, or both. Previous research which has tacitly assumed that method effects are associated with the level of the rater has included only one rater from each level; consequently, method effects due to the rater's level may have been confounded with those due to the individual rater. Based on confirmatory factor analysis, the present results revealed that of the five models tested, the best fit was the 10-factor model which hypothesized 7 method factors (one for each individual rater) and 3 trait factors. These results suggest that method variance in MTMR data is more strongly associated with individual raters than with the rater's level. Implications for research and practice pertaining to multirater feedback programs are discussed.     

Interrater reliability for the basic categories of the AO/ASIF's system as a frame of reference for classifying distal radial fractures

This study is about agreement on the assignment into the three basic classes or categories (A, B, C) of the Arbeitsgemeinschaft für Osteosynthesefragen/Association for the Study of Internal Fixation's (AO/ASIF) classification system for distal radial fractures. A random sample of 124 distal radial fractures was classified by two experienced observers. Their agreement was calculated according to Cohen's kappa statistic. To investigate the possible bases for disagreement, all conflicting X-ray assessments were discussed in a consensus meeting. It appeared that the kappa value was .65 (good agreement) before the meeting; kappa rose to .86 (excellent agreement) after the consensus meeting. It appeared that the undisplaced fractures were a major source of disagreement. Further, the presence of articular involvement was an important issue. It was frequently noted that one observer classified the fracture as extraarticular (basic Class A), while the other observer chose classification as an intra-articular fracture (basic Class C) or vice versa. This phenomenon has been called the A/C reversal shift. It is concluded that radiological innovations might enhance agreement on articular involvement, and a separate category for undisplaced fractures should be defined in the Arbeitsgemeinschaft für Osteosynthesefragen (AO) system. However, agreement on relevant distinctive features and discussion of conflicting assessments may also be important in achieving excellent agreement.     

Training raters to assess adult ADHD: reliability of ratings

The standardization of ADHD ratings in adults is important given their differing symptom presentation. The authors investigated the agreement and reliability of rater standardization in a large-scale trial of atomoxetine in adults with ADHD. Training of 91 raters for the investigator-administered ADHD Rating Scale (ADHDRS-IV-Inv) occurred prior to initiation of a large, 31-site atomoxetine trial. Agreement between raters on total scores was established in two ways: (a) by Kappa coefficient (rater agreement for each item with the percentage of raters that had identical item-by-item scores) and (b) intraclass correlation coefficients (reliability). For the ADHDRS-IV-Inv, rater agreement was moderate, and reliability, as measured by Cronbach's alpha, was substantial. The data indicate that clinicians can be trained to reliably evaluate ADHD in adults using the ADHDRS-IV-Inv.     

Twins and the study of rater (dis)agreement

Genetically informative data can be used to address fundamental questions concerning the measurement of behavior in children. The authors illustrate this with longitudinal multiple-rater data on internalizing problems in twins. Valid information on the behavior of a child is obtained for behavior that multiple raters agree upon and for rater-specific perception of the child's behavior. Rater-disagreement variance varsigma2(rd) accounted for 35% of the individual differences in internalizing behavior. Up to 17% of this varsigma2(rd) was accounted for by rater-specific additive genetic variance varsigma2(Au). Thus, the disagreement should not be considered only to be bias/error but also as representing the unique feature of the relationships between that parent and the child. The longitudinal extension of this model helps to make a distinction between measurement error and the raters' unique perception of the child's behavior. For internalizing behavior, the results show large stability across time, which is accounted for by common additive genetic and common shared environmental factors. Rater-specific shared environmental factors show substantial influence on stability. This could mean that rater bias may be persistent and affect longitudinal studies.     

IRT中Logistic模型的c、γ参数对能力估计的改善

设计一个理想测验和被试作答情况,在单、双参数模型下进行能力估计,存在第一、二未契合现象;增加c参数后进行能力估计,则能有效纠正第一未契合现象,仍然存在第二未契合现象,同时存在第三未契合现象;增加γ参数后进行能力估计,则能有效纠正第二未契合现象,仍然存在第一未契合现象,同时存在第四未契合现象;同时增加c、γ参数后进行能力估计,则能有效纠正第一、二、三、四未契合现象。最后概述了c、γ参数的测量含义     

Structural analysis of subjective categorical data

A general approach to the analysis of subjective categorical data is considered, in which agreement matrices of two or more raters are directly expressed in terms of error and agreement parameters. The method provides focused analyses of ratings from several raters for whom ratings have measurement error distributions that may induce bias in the evaluation of substantive questions of interest. Each rater's judgment process is modeled as a mixture of two components: an error variable that is unique for the rater in question as well as an agreement variable that operationalizes the true values of the units of observation. The statistical problems of identification, estimation, and testing of such measurement models are discussed.The general model is applied in several special cases. The most simple situation is that underlying Cohen's Kappa, where two raters place units into unordered categories. The model provides a generalization and systematization of the Kappa-idea to correct for agreement by chance. In applications with typical research designs, including a between-subjects design and a mixed within-subjects, between-subjects design, the model is shown to disentangle structural and measurement components of the observations, thereby controlling for possible confounding effects of systematic rater bias. Situations considered include the case of more than two raters as well as the case of ordered categories. The different analyses are illustrated by means of real data sets.The authors wish to thank Lawrence Hubert and Ivo Molenaar for helpful and detailed comments on a previous draft of this paper. Thanks are also due to Jens Möller und Bernd Strauß for the data from the 1992 Olympic Games. We thank the editor and three anonymous reviewers for valuable comments on an earlier draft.     

Modeling Rater Disagreement for ADHD: Are Parents or Teachers Biased?

The present study is the first to utilize twin modeling to examine whether parent-teacher disagreement for ADHD ratings is due to parent or teacher bias, or due to raters observing different but valid ADHD behaviors. A joint analysis was conducted with 106 twin pairs, including twins selected for ADHD and control twin pairs. Total ADHD scores were analyzed using multiple rater models that estimate genetic and environmental contributions common to both raters and unique to each rater. Results suggest that 1) disagreement in ADHD ratings is strongly due to parents and teachers observing different ADHD behaviors, some of which is valid and some of which is due to bias, and 2) parents may be more biased than teachers in their ADHD ratings.     

The creation of uncertainty in the influence process: The roles of stimulus information and disagreement with similar others

Social and cognitive psychologists have conceptualized judgemental confidence (how strongly a person holds the belief that some judgement is correct) as being proportional to the amount of evidence in favour of a response. Festinger (1950) argued that there are two separate processes by which uncertainty (the inverse of confidence) can be reduced. These two processes are physical reality testing (the perceptual processing of stimulus information) and social reality testing (reliance on other people to resolve particularly ambiguous situations). However, there is surprisingly little direct evidence that uncertainty is either reduced or increased by the responses of other people. In two experimental tests (N = 74 and N = 83) it was found that disagreement increased uncertainty and agreement tended to reduce uncertainty. In a third experiment (N = 63) it was found that disagreement only increased uncertainty when stimulus information was limited, but that agreement generally reduced uncertainty. The results challenge Festinger's model of uncertainty reduction and support a self-categorization theory account.     

等级反应多水平侧面模型及其在主观题评分中的应用

探讨了康春花,孙小坚和曾平飞(2016)提出的等级反应多水平侧面模型(GR-MLFM)在包含被试及评分者层面预测变量(完整模型)下的返真性和适用性。结果表明:(1)GR-MLFM完整模型具有逻辑上和数理上的合理性,可用于主观题的评分情境,能较好地检测出评分者效应、影响因素及其影响程度;(2)在数学问题解决的评分实践中,评分员存在两种类型的评分倾向(宽松和严格效应),但绝大多数评分员的宽严度不明显;评分者的责任心可正向预测其严格程度,自信心可正向预测其宽松程度,而情绪稳定性和评分经验的预测作用不显著。     

基于等级反应模型的多水平多侧面评分者模型

国内外考试改革和大型测评实践越来越强调主观题的作用,则评分者信度研究又重新成为一个备受关注的议题。研究在Wang和Liu（2007）的广义多水平侧面模型基础上,提出并探讨了等级反应多水平侧面模型。结果表明：在评分者固定效应和随机效应两种实验条件下,各偏差值的均值与标准差均较小,说明模型在当前实验条件下,各参数估计值的返真性和稳健性均较好,可以检测出评分者效应,由此,后续可进一步加入评分者效应的影响因素,使其发展为可同时检测评分者效应及其影响因素的完整模型。     

Visual discrimination of shape by octopus. squares and triangles

An experiment was performed upon visual discrimination of shape by the octopus in order to test two predictions derived from a theory of visual discrimination of orientation and shape. Two groups of octopuses were trained to discriminate between a square and a triangle and between a diamond and a triangle. It was found that octopuses discriminate more readily between a square with base horizontal and a triangle than between a diamond and a triangle. Transfer tests showed that: (1) For the octopus, an upright and an inverted triangle have more equivalence than a diamond and a square with base horizontal. (2) Octopuses do not discriminate between the figures used by analysing only differences in one part of the figures (e.g. bases or tops). (3) Having learned the initial discrimination, octopuses transfer to both larger and smaller figures. (4) A pentagon has more equivalence to a square or diamond than to a triangle. (5) A circle is not treated as equivalent to a square. The results are taken to be in agreement with the theory that octopuses analyse the vertical and lateral extents of figures, and that shape discrimination is achieved by analysing the changes in the rates of change in the firing of neurons representing the vertical and lateral extents of the shapes. The results are shown to differ from those obtained with birds and rats, but to agree with results found for higher mammals where these are available for comparison.     

一种改进的分层并行演化算法(英文)

提出了一种改进的分层并行演化算法。针对传统算法中"同构子种群"和"同步通信"所引发的问题,新算法构建了异构模型,并将各子种群充分连接。子种群一旦满足迁移条件,便可按照预设的迁移模式实施异步迁移。仿真实验结果表明,本文提出的新算法有效地解决了"征服问题"和"无效问题",避免了算法的过早收敛,提高了算法的效率。     

Prior distributions for random choice structures

We study various axioms of discrete probabilistic choice, measuring how restrictive they are, both alone and in the presence of other axioms, given a specific class of prior distributions over a complete collection of finite choice probabilities. We do this by using Monte Carlo simulation to compute, for a range of prior distributions, probabilities that various simple and compound axioms hold. For example, the probability of the triangle inequality is usually many orders of magnitude higher than the probability of random utility. While neither the triangle inequality nor weak stochastic transitivity imply the other, the conditional probability that one holds given the other holds is greater than the marginal probability, for all priors in the class we consider. The reciprocal of the prior probability that an axiom holds is an upper bound on the Bayes factor in favor of a restricted model, in which the axiom holds, against an unrestricted model. The relatively high prior probability of the triangle inequality limits the degree of support that data from a single decision maker can provide in its favor. The much lower probability of random utility implies that the Bayes factor in favor of it can be much higher, for suitable data.     

Regime-Switching Bivariate Dual Change Score Model

Mixture structural equation model with regime switching (MSEM-RS) provides one possible way of representing over-time heterogeneities in dynamic processes by allowing a system to manifest qualitatively or quantitatively distinct change processes conditional on the latent “regime” the system is in at a particular time point. Unlike standard mixture structural equation models such as growth mixture models, MSEM-RS allows individuals to transition between latent classes over time. This class of models, often referred to as regime-switching models in the time series and econometric applications, can be specified as regime-switching mixture structural equation models when the number of repeated measures involved is not large. We illustrate the empirical utility of such models using one special case—a regime-switching bivariate dual change score model in which two growth processes are allowed to manifest regime-dependent coupling relations with one another. The proposed model is illustrated using a set of longitudinal reading and arithmetic performance data from the Early Childhood Longitudinal Study, Kindergarten Class of 1998–99 study (ECLS-K; U.S. Department of Education, National Center for Education Statistics, 2010).