Hierarchical models of simple mechanisms underlying confidence in decision making

Choice confidence is a central measure in psychological decision research, often being reported on a probabilistic scale. Simple mechanisms that describe the psychological processes underlying choice confidence, including those based on error and confirmation biases, have typically received support via fits to data averaged over subjects. While averaged data ease model development, they can also destroy important aspects of the confidence data distribution. In this paper, we develop a hierarchical model of raw confidence judgments using the beta distribution, and we implement two simple confidence mechanisms within it. We use Bayesian methods to fit the hierarchical model to data from a two-alternative confidence experiment, and we use a variety of Bayesian tools to diagnose shortcomings of the simple mechanisms that are overlooked when applied to averaged data. Bugs code for estimating the models is also supplied.     

Marginal and Random Intercepts Models for Longitudinal Binary Data With Examples From Criminology

Two models for the analysis of longitudinal binary data are discussed: the marginal model and the random intercepts model. In contrast to the linear mixed model (LMM), the two models for binary data are not subsumed under a single hierarchical model. The marginal model provides group-level information whereas the random intercepts model provides individual-level information including information about heterogeneity of growth. It is shown how a type of numerical averaging can be used with the random intercepts model to obtain group-level information, thus approximating individual and marginal aspects of the LMM. The types of inferences associated with each model are illustrated with longitudinal criminal offending data based on N = 506 males followed over a 22-year period. Violent offending indexed by official records and self-report were analyzed, with the marginal model estimated using generalized estimating equations and the random intercepts model estimated using maximum likelihood. The results show that the numerical averaging based on the random intercepts can produce prediction curves almost identical to those obtained directly from the marginal model parameter estimates. The results provide a basis for contrasting the models and the estimation procedures and key features are discussed to aid in selecting a method for empirical analysis.     

Modeling Rule-Based Item Generation

An application of a hierarchical IRT model for items in families generated through the application of different combinations of design rules is discussed. Within the families, the items are assumed to differ only in surface features. The parameters of the model are estimated in a Bayesian framework, using a data-augmented Gibbs sampler. An obvious application of the model is computerized algorithmic item generation. Such algorithms have the potential to increase the cost-effectiveness of item generation as well as the flexibility of item administration. The model is applied to data from a non-verbal intelligence test created using design rules. In addition, results from a simulation study conducted to evaluate parameter recovery are presented.     

Bayesian modeling of measurement error in predictor variables using item response theory

It is shown that measurement error in predictor variables can be modeled using item response theory (IRT). The predictor variables, that may be defined at any level of an hierarchical regression model, are treated as latent variables. The normal ogive model is used to describe the relation between the latent variables and dichotomous observed variables, which may be responses to tests or questionnaires. It will be shown that the multilevel model with measurement error in the observed predictor variables can be estimated in a Bayesian framework using Gibbs sampling. In this article, handling measurement error via the normal ogive model is compared with alternative approaches using the classical true score model. Examples using real data are given.This paper is part of the dissertation by Fox (2001) that won the 2002 Psychometric Society Dissertation Award.     

我国近十年来心理学研究中HLM方法的应用述评

以2002-2011年中国期刊网收录的50例应用多层线性模型(HLM)的心理学期刊论文为研究对象,从样本描述、模型发展与规范、数据准备、估计方法与假设检验4个角度进行文献计量和内容分析,对我国心理学研究中HLM方法的使用现状进行评估。结果表明,HLM方法主要用于管理、发展和教育心理学,绝大多数应用都是两层模型且层2样本量较大。HLM方法在广泛应用的同时仍存在忽略前提假设检验、分析过程中的重要信息和结果报告不完整等问题,随后提供了4条建议。     

The effect of cultural values on pro-environmental attitude in the context of travel mode choice: A hierarchical approach

The cultural theory explains social behavior through four elementary types of cultural values consisting of hierarchy, individualism, egalitarianism, and fatalism. The knowledge of how these values influence attitudes and behaviors specifically pertain to the environment is limited. Understanding individuals’ values and attitudes should be addressed in travel mode choice based on possible impacts of transportation on the environment. This study investigates the effect of cultural values on pro-environmental attitude and the influence of this attitude on travel mode choice in light of a hierarchical latent choice model. The model is estimated using data from a random sampling in CBD (Central Business District) of Tehran, Iran. The pro-environmental attitude, which is postulated to be affected by cultural values, is considered as the latent variable directly affecting travel mode choice. The cultural values drivers of pro-environmental attitude have been seen in a hierarchical structure. The estimated results show that hierarchical cultural tendency has the strongest and positive effect on being pro-environmental. Also, individualistic culture indicates a positive trend in being pro-environmental. On the other side, people with egalitarianism value tend to report an orientation towards pro-environmental attitude. Moreover, pro-environmental attitude increases the utility of public and active modes of transportation and a negative significant effect is found on the utility of private car and motorcycle.     

Nonlinear Random-Effects Mixture Models for Repeated Measures

A mixture model for repeated measures based on nonlinear functions with random effects is reviewed. The model can include individual schedules of measurement, data missing at random, nonlinear functions of the random effects, of covariates and of residuals. Individual group membership probabilities and individual random effects are obtained as empirical Bayes predictions. Although this is a complicated model that combines a mixture of populations, nonlinear regression, and hierarchical models, it is straightforward to estimate by maximum likelihood using SAS PROC NLMIXED. Many different models can be studied with this procedure. The model is more general than those that can be estimated with most special purpose computer programs currently available because the response function is essentially any form of nonlinear regression. Examples and sample code are included to illustrate the method.     

School System Evaluation by Value Added Analysis Under Endogeneity

Value added is a common tool in educational research on effectiveness. It is often modeled as a (prediction of a) random effect in a specific hierarchical linear model. This paper shows that this modeling strategy is not valid when endogeneity is present. Endogeneity stems, for instance, from a correlation between the random effect in the hierarchical model and some of its covariates. This paper shows that this phenomenon is far from exceptional and can even be a generic problem when the covariates contain the prior score attainments, a typical situation in value added modeling. Starting from a general, model-free definition of value added, the paper derives an explicit expression of the value added in an endogeneous hierarchical linear Gaussian model. Inference on value added is proposed using an instrumental variable approach. The impact of endogeneity on the value added and the estimated value added is calculated accurately. This is also illustrated on a large data set of individual scores of about 200,000 students in Chile.     

Constructing informative model priors using hierarchical methods

Despite their negative reputation, informative priors are very useful in inference. Priors that express psychologically meaningful intuitions damp out random fluctuations in the data due to sampling variability, without sacrificing flexibility. This article focuses on how an intuitively satisfying informative prior distribution can be constructed. In particular, it demonstrates how the hierarchical introduction of a parameterized generative account of the set of models under consideration naturally imposes a non-uniform prior distribution over the models, encoding existing intuitions about the models. The hierarchical approach for constructing informative model priors is made concrete using a worked example, the Varying Abstraction Model (VAM), a family of categorization models including and expanding the exemplar and prototype models. It is shown how psychological intuitions about the relative plausibilities of the models in the VAM can be formally captured in an informative prior distribution over these models, by specifying a theoretically informed process for generating the models in the VAM. The smoothing effect of the informative prior in estimation is demonstrated by considering ten previously published data sets from the category learning literature.     

Improving precision of ability estimation: Getting more from response times

By considering information about response time (RT) in addition to response accuracy (RA), joint models for RA and RT such as the hierarchical model (van der Linden, 2007) can improve the precision with which ability is estimated over models that only consider RA. The hierarchical model, however, assumes that only the person's speed is informative of ability. This assumption of conditional independence between RT and ability given speed may be violated in practice, and ignores collateral information about ability that may be present in the residual RTs. We propose a posterior predictive check for evaluating the assumption of conditional independence between RT and ability given speed. Furthermore, we propose an extension of the hierarchical model that contains cross-loadings between ability and RT, which enables one to take additional collateral information about ability into account beyond what is possible in the standard hierarchical model. A Bayesian estimation procedure is proposed for the model. Using simulation studies, the performance of the model is evaluated in terms of parameter recovery, and the possible gain in precision over the standard hierarchical model and an RA-only model is considered. The model is applied to data from a high-stakes educational test.     

Robust estimation of the hierarchical model for responses and response times

Van der Linden's (2007, Psychometrika, 72, 287) hierarchical model for responses and response times in tests has numerous applications in psychological assessment. The success of these applications requires the parameters of the model to have been estimated without bias. The data used for model fitting, however, are often contaminated, for example, by rapid guesses or lapses of attention. This distorts the parameter estimates. In the present paper, a novel estimation approach is proposed that is robust against contamination. The approach consists of two steps. In the first step, the response time model is fitted on the basis of a robust estimate of the covariance matrix. In the second step, the item response model is extended to a mixture model, which allows for a proportion of irregular responses in the data. The parameters of the mixture model are then estimated with a modified marginal maximum likelihood estimator. The modified marginal maximum likelihood estimator downweights responses of test-takers with unusual response time patterns. As a result, the estimator is resistant to several forms of data contamination. The robustness of the approach is investigated in a simulation study. An application of the estimator is demonstrated with real data.     

Latent growth curve analysis with dichotomous items: Comparing four approaches

A Monte Carlo study was used to compare four approaches to growth curve analysis of subjects assessed repeatedly with the same set of dichotomous items: A two‐step procedure first estimating latent trait measures using MULTILOG and then using a hierarchical linear model to examine the changing trajectories with the estimated abilities as the outcome variable; a structural equation model using modified weighted least squares (WLSMV) estimation; and two approaches in the framework of multilevel item response models, including a hierarchical generalized linear model using Laplace estimation, and Bayesian analysis using Markov chain Monte Carlo (MCMC). These four methods have similar power in detecting the average linear slope across time. MCMC and Laplace estimates perform relatively better on the bias of the average linear slope and corresponding standard error, as well as the item location parameters. For the variance of the random intercept, and the covariance between the random intercept and slope, all estimates are biased in most conditions. For the random slope variance, only Laplace estimates are unbiased when there are eight time points.     

Comparison of Methods for Collecting and Modeling Dissimilarity Data: Applications to Complex Sound Stimuli

Sorting procedures are frequently adopted as an alternative to dissimilarity ratings to measure the dissimilarity of large sets of stimuli in a comparatively short time. However, systematic empirical research on the consequences of this experiment-design choice is lacking. We carried out a behavioral experiment to assess the extent to which sorting procedures compare to dissimilarity ratings in terms of efficiency, reliability, and accuracy, and the extent to which data from different data-collection methods are redundant and are better fit by different distance models. Participants estimated the dissimilarity of either semantically charged environmental sounds or semantically neutral synthetic sounds. We considered free and hierarchical sorting and derived indications concerning the properties of constrained and truncated hierarchical sorting methods from hierarchical sorting data. Results show that the higher efficiency of sorting methods comes at a considerable cost in terms of data reliability and accuracy. This loss appears to be minimized with truncated hierarchical sorting methods that start from a relatively low number of groups of stimuli. Finally, variations in data-collection method differentially affect the fit of various distance models at the group-average and individual levels. On the basis of these results, we suggest adopting sorting as an alternative to dissimilarity-rating methods only when strictly necessary. We also suggest analyzing the raw behavioral dissimilarities, and avoiding modeling them with one single distance model.     

A parametric procedure for ultrametric tree estimation from conditional rank order proximity data

The psychometric and classification literatures have illustrated the fact that a wide class of discrete or network models (e.g., hierarchical or ultrametric trees) for the analysis of ordinal proximity data are plagued by potential degenerate solutions if estimated using traditional nonmetric procedures (i.e., procedures which optimize a STRESS-based criteria of fit and whose solutions are invariant under a monotone transformation of the input data). This paper proposes a new parametric, maximum likelihood based procedure for estimating ultrametric trees for the analysis of conditional rank order proximity data. We present the technical aspects of the model and the estimation algorithm. Some preliminary Monte Carlo results are discussed. A consumer psychology application is provided examining the similarity of fifteen types of snack/breakfast items. Finally, some directions for future research are provided.     

多水平项目反应理论模型在测验发展中的应用

作者简要介绍了多水平项目反应模型,对多水平项目反应理论与通常项目反应理论之间的关系进行了探讨,得到了多水平项目反应模型参数与通常项目反应模型参数之间的关系,并讨论了多水平项目反应模型的推广模型。通过一个实际例子,用多水平项目反应模型对测验中项目的特征进行分析;检验个体水平和组水平预测变量对能力参数的影响;对项目功能差异进行分析。最后文章就多水平项目反应理论模型的优势与不足进行了讨论     

Hierarchical Bayesian parameter estimation for cumulative prospect theory

Cumulative prospect theory (CPT Tversky & Kahneman, 1992) has provided one of the most influential accounts of how people make decisions under risk. CPT is a formal model with parameters that quantify psychological processes such as loss aversion, subjective values of gains and losses, and subjective probabilities. In practical applications of CPT, the model’s parameters are usually estimated using a single-participant maximum likelihood approach. The present study shows the advantages of an alternative, hierarchical Bayesian parameter estimation procedure. Performance of the procedure is illustrated with a parameter recovery study and application to a real data set. The work reveals that without particular constraints on the parameter space, CPT can produce loss aversion without the parameter that has traditionally been associated with loss aversion. In general, the results illustrate that inferences about people’s decision processes can crucially depend on the method used to estimate model parameters.     

Post-traumatic stress disorder symptoms in United Nations peacekeepers: an examination of factor structure in peacekeepers with and without chronic pain

Recent factor analytic investigations of post-traumatic stress disorder in military veterans suggest that symptoms are best described by either a hierarchical 2-factor model or a 4-factor inter-correlated model. Other recent evidence suggests that post-traumatic stress disorder and chronic pain are intricately related; however, the nature of this relationship is not well understood. Factor analysis provides one method for clarifying this relationship. In study 1, we compared competing models of post-traumatic stress disorder symptom structure in a sample of 400 male United Nations peacekeepers using confirmatory factor analysis. Results indicated that both the hierarchical 2-factor and the 4-factor inter-correlated models provided good fit to the data. In study 2, the reliability of these models was assessed in 427 male United Nations peacekeepers with chronic back pain and 341 without. Group comparisons of the confirmatory factor analysis results revealed that the structure of the hierarchical 2-factor and 4-factor inter-correlated models both provided good fit to the data in both the chronic back pain and the group without. However, the structure of the models for the group with chronic back pain group differed in significant ways from that of the group without chronic back pain. Post-traumatic stress disorder symptoms in military veterans can be adequately conceptualized using either a hierarchical 2-factor or 4-factor inter-correlated model. Chronic pain has a minimal influence on overall factor structure. The hierarchical 2-factor model, while parsimonious, does not provide the degree of symptom detail provided by the 4-factor inter-correlated model. Implications for conceptualization of post-traumatic stress disorder symptoms for patients with chronic back pain and significant post-traumatic stress disorder symptomatology are discussed.     

Post-Traumatic Stress Disorder Symptoms in United Nations Peacekeepers: An Examination of Factor Structure in Peacekeepers with and without Chronic Pain

Recent factor analytic investigations of post-traumatic stress disorder in military veterans suggest that symptoms are best described by either a hierarchical 2-factor model or a 4-factor inter-correlated model. Other recent evidence suggests that post-traumatic stress disorder and chronic pain are intricately related; however, the nature of this relationship is not well understood. Factor analysis provides one method for clarifying this relationship. In study 1, we compared competing models of post-traumatic stress disorder symptom structure in a sample of 400 male United Nations peacekeepers using confirmatory factor analysis. Results indicated that both the hierarchical 2-factor and the 4-factor inter-correlated models provided good fit to the data. In study 2, the reliability of these models was assessed in 427 male United Nations peacekeepers with chronic back pain and 341 without. Group comparisons of the confirmatory factor analysis results revealed that the structure of the hierarchical 2-factor and 4-factor inter-correlated models both provided good fit to the data in both the chronic back pain and the group without. However, the structure of the models for the group with chronic back pain group differed in significant ways from that of the group without chronic back pain. Post-traumatic stress disorder symptoms in military veterans can be adequately conceptualized using either a hierarchical 2-factor or 4-factor inter-correlated model. Chronic pain has a minimal influence on overall factor structure. The hierarchical 2-factor model, while parsimonious, does not provide the degree of symptom detail provided by the 4-factor inter-correlated model. Implications for conceptualization of post-traumatic stress disorder symptoms for patients with chronic back pain and significant post-traumatic stress disorder symptomatology are discussed.     

Bootstrapping the estimated latent distribution of the two‐parameter latent trait model

This paper focuses on the two‐parameter latent trait model for binary data. Although the prior distribution of the latent variable is usually assumed to be a standard normal distribution, that prior distribution can be estimated from the data as a discrete distribution using a combination of EM algorithms and other optimization methods. We assess with what precision we can estimate the prior from the data, using simulations and bootstrapping. A novel calibration method is given to check that near optimality is achieved for the bootstrap estimates. We find that there is sufficient information on the prior distribution to be informative, and that the bootstrap method is reliable. We illustrate the bootstrap method for two sets of real data.     

Marginal likelihood inference for a model for item responses and response times

Marginal maximum‐likelihood procedures for parameter estimation and testing the fit of a hierarchical model for speed and accuracy on test items are presented. The model is a composition of two first‐level models for dichotomous responses and response times along with multivariate normal models for their item and person parameters. It is shown how the item parameters can easily be estimated using Fisher's identity. To test the fit of the model, Lagrange multiplier tests of the assumptions of subpopulation invariance of the item parameters (i.e., no differential item functioning), the shape of the response functions, and three different types of conditional independence were derived. Simulation studies were used to show the feasibility of the estimation and testing procedures and to estimate the power and Type I error rate of the latter. In addition, the procedures were applied to an empirical data set from a computerized adaptive test of language comprehension.