The relationship between multilevel models and non‐parametric multilevel mixture models: Discrete approximation of intraclass correlation,random coefficient distributions,and residual heteroscedasticity

Multilevel data structures are common in the social sciences. Often, such nested data are analysed with multilevel models (MLMs) in which heterogeneity between clusters is modelled by continuously distributed random intercepts and/or slopes. Alternatively, the non‐parametric multilevel regression mixture model (NPMM) can accommodate the same nested data structures through discrete latent class variation. The purpose of this article is to delineate analytic relationships between NPMM and MLM parameters that are useful for understanding the indirect interpretation of the NPMM as a non‐parametric approximation of the MLM, with relaxed distributional assumptions. We define how seven standard and non‐standard MLM specifications can be indirectly approximated by particular NPMM specifications. We provide formulas showing how the NPMM can serve as an approximation of the MLM in terms of intraclass correlation, random coefficient means and (co)variances, heteroscedasticity of residuals at level 1, and heteroscedasticity of residuals at level 2. Further, we discuss how these relationships can be useful in practice. The specific relationships are illustrated with simulated graphical demonstrations, and direct and indirect interpretations of NPMM classes are contrasted. We provide an R function to aid in implementing and visualizing an indirect interpretation of NPMM classes. An empirical example is presented and future directions are discussed.     

FOOLING YOURSELF WITH CROSS-VALIDATION: SINGLE SAMPLE DESIGNS

The most commonly used cross-validation design involves drawing a single sample and partitioning that sample into derivation and holdout subsamples. This type of design allows one to adjust for random sampling error, but like formula estimates of cross-validity, is insensitive to violations of sampling assumptions. As is shown in a small Monte Carlo study, results obtained in non-representative samples, which are known to be invalid in the population, will nonetheless hold up well under cross-validation when single-sample designs are employed. It is suggested that single-sample cross-validation estimates possess no clear-cut advantages over formula estimates, and thus are not worth the effort or the loss of degrees of freedom.     

A Comparison of Inverse-Wishart Prior Specifications for Covariance Matrices in Multilevel Autoregressive Models

Multilevel autoregressive models are especially suited for modeling between-person differences in within-person processes. Fitting these models with Bayesian techniques requires the specification of prior distributions for all parameters. Often it is desirable to specify prior distributions that have negligible effects on the resulting parameter estimates. However, the conjugate prior distribution for covariance matrices—the Inverse-Wishart distribution—tends to be informative when variances are close to zero. This is problematic for multilevel autoregressive models, because autoregressive parameters are usually small for each individual, so that the variance of these parameters will be small. We performed a simulation study to compare the performance of three Inverse-Wishart prior specifications suggested in the literature, when one or more variances for the random effects in the multilevel autoregressive model are small. Our results show that the prior specification that uses plug-in ML estimates of the variances performs best. We advise to always include a sensitivity analysis for the prior specification for covariance matrices of random parameters, especially in autoregressive models, and to include a data-based prior specification in this analysis. We illustrate such an analysis by means of an empirical application on repeated measures data on worrying and positive affect.     

A Method of Correcting Estimation Failure in Latent Differential Equations with Comparisons to Kalman Filtering

Abstract

Studies have used the latent differential equation (LDE) model to estimate the parameters of damped oscillation in various phenomena, but it has been shown that correct, non-zero parameter estimates are only obtained when the latent series exhibits little or no process noise. Consequently, LDEs are limited to modeling deterministic processes with measurement error rather than those with random behavior in the true latent state. The reasons for these limitations are considered, and a piecewise deterministic approximation (PDA) algorithm is proposed to treat process noise outliers as functional discontinuities and obtain correct estimates of the damping parameter. Comprehensive, random-effects simulations were used to compare results with those obtained using a state-space model (SSM) based on the Kalman filter. The LDE with the PDA algorithm (LDEPDA) successfully recovered the simulated damping parameter under a variety of conditions when process noise was present in the latent state. The LDEPDA had greater precision and accuracy than the SSM when estimating parameters from data with sparse jump discontinuities, but worse performance for diffusion processes overall. All three methods were applied to a sample of postural sway data. The basic LDE estimated zero damping, while the LDEPDA and SSM estimated moderate to high damping. The SSM estimated the smallest standard errors for both frequency and damping parameter estimates.     

Nonparametric Estimation of Standard Errors in Covariance Analysis Using the Infinitesimal Jackknife

The infinitesimal jackknife provides a simple general method for estimating standard errors in covariance structure analysis. Beyond its simplicity and generality what makes the infinitesimal jackknife method attractive is that essentially no assumptions are required to produce consistent standard error estimates, not even the requirement that the population sampled has the covariance structure assumed. Commonly used covariance structure analysis software uses parametric methods for estimating parameters and standard errors. When the population sampled has the covariance structure assumed, but fails to have the distributional form assumed, the parameter estimates usually remain consistent, but the standard error estimates do not. This has motivated the introduction of a variety of nonparametric standard error estimates that are consistent when the population sampled fails to have the distributional form assumed. The only distributional assumption these require is that the covariance structure be correctly specified. As noted, even this assumption is not required for the infinitesimal jackknife. The relation between the infinitesimal jackknife and other nonparametric standard error estimators is discussed. An advantage of the infinitesimal jackknife over the jackknife and the bootstrap is that it requires only one analysis to produce standard error estimates rather than one for every jackknife or bootstrap sample.     

The limits of definition: Gadamer’s critique of Aristotle’s ethics

ABSTRACT

There is a recent scholarly trend drawing similarities between Aristotle’s conceptions of ethics and demonstrative science. One such similarity has become widely and rightly recognized: for Aristotle both ethics and demonstrative science seek essential definitions of phenomena. The task of the paper is to show that German philosopher and classicist Hans-Georg Gadamer not only prefigured this interpretative trend, he also identified a problematic feature of Aristotle’s method so construed. The problematic feature is semantic. For Aristotle essential definitions must consist of univocal terms, but such univocity cannot be maintained if the definienda and definientes are concepts to be usefully employed in everyday ethical judgements. Gadamer therefore points out a problematic tension in Aristotle’s ethics. On the one hand, ethical inquiry should have practical application, but, on the other hand, the semantic assumptions underlying inquiry imply that it cannot.     

MCMC estimation for the p2 network regression model with crossed random effects

The p₂ model is a statistical model for the analysis of binary relational data with covariates, as occur in social network studies. It can be characterized as a multinomial regression model with crossed random effects that reflect actor heterogeneity and dependence between the ties from and to the same actor in the network. Three Markov chain Monte Carlo (MCMC) estimation methods for the p₂ model are presented to improve iterative generalized least squares (IGLS) estimation developed earlier, two of which use random walk proposals. The third method, an independence chain sampler, and one of the random walk algorithms use normal approximations of the binary network data to generate proposals in the MCMC algorithms. A large‐scale simulation study compares MCMC estimates with IGLS estimates for networks with 20 and 40 actors. It was found that the IGLS estimates have a smaller variance but are severely biased, while the MCMC estimates have a larger variance with a small bias. For networks with 20 actors, mean squared errors are generally comparable or smaller for the IGLS estimates. For networks with 40 actors, mean squared errors are the smallest for the MCMC estimates. Coverage rates of confidence intervals are good for the MCMC estimates but not for the IGLS estimates.     

Finite Mixtures of Hidden Markov Models for Longitudinal Responses Subject to Drop out

Abstract

Drop out is a typical issue in longitudinal studies. When the missingness is non-ignorable, inference based on the observed data only may be biased. This paper is motivated by the Leiden 85+ study, a longitudinal study conducted to analyze the dynamics of cognitive functioning in the elderly. We account for dependence between longitudinal responses from the same subject using time-varying random effects associated with a heterogeneous hidden Markov chain. As several participants in the study drop out prematurely, we introduce a further random effect model to describe the missing data mechanism. The potential dependence between the random effects in the two equations (and, therefore, between the two processes) is introduced through a joint distribution specified via a latent structure approach. The application of the proposal to data from the Leiden 85+ study shows its effectiveness in modeling heterogeneous longitudinal patterns, possibly influenced by the missing data process. Results from a sensitivity analysis show the robustness of the estimates with respect to misspecification of the missing data mechanism. A simulation study provides evidence for the reliability of the inferential conclusions drawn from the analysis of the Leiden 85+ data.     

Investigating the Impact of Noneffortful Responses on Individual-Level Scores: Can the Effort-Moderated IRT Model Serve as a Solution?

Suboptimal effort is a major threat to valid score-based inferences. While the effects of such behavior have been frequently examined in the context of mean group comparisons, minimal research has considered its effects on individual score use (e.g., identifying students for remediation). Focusing on the latter context, this study addressed two related questions via simulation and applied analyses. First, we investigated how much including noneffortful responses in scoring using a three-parameter logistic (3PL) model affects person parameter recovery and classification accuracy for noneffortful responders. Second, we explored whether improvements in these individual-level inferences were observed when employing the Effort Moderated IRT (EM-IRT) model under conditions in which its assumptions were met and violated. Results demonstrated that including 10% noneffortful responses in scoring led to average bias in ability estimates and misclassification rates by as much as 0.15 SDs and 7%, respectively. These results were mitigated when employing the EM-IRT model, particularly when model assumptions were met. However, once model assumptions were violated, the EM-IRT model’s performance deteriorated, though still outperforming the 3PL model. Thus, findings from this study show that (a) including noneffortful responses when using individual scores can lead to potential unfounded inferences and potential score misuse, and (b) the negative impact that noneffortful responding has on person ability estimates and classification accuracy can be mitigated by employing the EM-IRT model, particularly when its assumptions are met.     

Constructs to describe individual knowledge / Constructos para describir el conocimiento individual

Abstract

The basic unit of any cognitive model that attempts to describe individual knowledge and its manifestations must be chosen carefully, as the resulting position on learning, teaching and, in general, of the psychological research subject depends largely on this choice. The objective is to assess and contrast two constructs of enormous significance in cognitive psychology that have been used as cognitive units under different theoretical contexts: the concept in its classical sense, and the scheme in its organismic version. Arguments are presented to show that the latter is more consistent with constructivist assumptions and is more respectful of psychological data such as the Gestalt and prototype effects or the existence of procedural and implicit knowledge.     

Bias in longitudinal data analysis with missing data using typical linear mixed‐effects modelling and pattern‐mixture approach: An analytical illustration

We analytically derive the fixed‐effects estimates in unconditional linear growth curve models by typical linear mixed‐effects modelling (TLME) and by a pattern‐mixture (PM) approach with random‐slope‐dependent two‐missing‐pattern missing not at random (MNAR) longitudinal data. Results showed that when the missingness mechanism is random‐slope‐dependent MNAR, TLME estimates of both the mean intercept and mean slope are biased because of incorrect weights used in the estimation. More specifically, the estimate of the mean slope is biased towards the mean slope for completers, whereas the estimate of the mean intercept is biased towards the opposite direction as compared to the estimate of the mean slope. We also discuss why the PM approach can provide unbiased fixed‐effects estimates for random‐coefficients‐dependent MNAR data but does not work well for missing at random or outcome‐dependent MNAR data. A small simulation study was conducted to illustrate the results and to compare results from TLME and PM. Results from an empirical data analysis showed that the conceptual finding can be generalized to other real conditions even when some assumptions for the analytical derivation cannot be met. Implications from the analytical and empirical results were discussed and sensitivity analysis was suggested for longitudinal data analysis with missing data.     

Human Psychophysical Functions,an Update: Methods for Identifying their form; Estimating their Parameters; and Evaluating the Effects of Important Predictors

Stevens’ power law for the judgments of sensation has a long history in psychology and is used in many psychophysical investigations of the effects of predictors such as group or condition. Stevens’ formulation \(\varPsi = {aP}^{n}\), where \(\varPsi \) is the psychological judgment, P is the physical intensity, and \(n\) is the power law exponent, is usually tested by plotting log \((\varPsi )\) against log (P). In some, but by no means all, studies, effects on the scale parameter, \(a\), are also investigated. This two-parameter model is simple but known to be flawed, for at least some modalities. Specifically, three-parameter functions that include a threshold parameter produce a better fit for many data sets. In addition, direct non-linear computation of power laws often fit better than regressions of log-transformed variables. However, such potentially flawed methods continue to be used because of assumptions that the approximations are “close enough” as to not to make any difference to the conclusions drawn (or possibly through ignorance the errors in these assumptions). We investigate two modalities in detail: duration and roughness. We show that a three-parameter power law is the best fitting of several plausible models. Comparison between this model and the prevalent two parameter version of Stevens’ power law shows significant differences for the parameter estimates with at least medium effect sizes for duration.     

Premature conclusions about the signal-to-noise ratio in structural equation modeling research: A commentary on Yuan and Fang (2023)

In a recent article published in this journal, Yuan and Fang (British Journal of Mathematical and Statistical Psychology, 2023) suggest comparing structural equation modeling (SEM), also known as covariance-based SEM (CB-SEM), estimated by normal-distribution-based maximum likelihood (NML), to regression analysis with (weighted) composites estimated by least squares (LS) in terms of their signal-to-noise ratio (SNR). They summarize their findings in the statement that “[c]ontrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the [SNR].” In our commentary, we show that Yuan and Fang have made several incorrect assumptions and claims. Consequently, we recommend that empirical researchers not base their methodological choice regarding CB-SEM and regression analysis with composites on the findings of Yuan and Fang as these findings are premature and require further research.     

Reduced emotional stress reactivity to a real-life academic examination stressor in students participating in a 20-week aerobic exercise training: A randomised controlled trial using Ambulatory Assessment

ObjectivesTo examine if a preventive 20-week aerobic exercise intervention (AET) can improve emotional stress reactivity during real-life stress.DesignRandomised controlled trial; within-subject design.MethodSixty-one inactive students were randomly assigned to a waiting control and an AET group. To capture the situation-specific, intra-individual data in real life, electronic diaries were used. Participants reported their moods and perceived stress (PS) repeatedly over two days during their daily routines pre- and post-intervention. The pre-intervention baseline assessment was scheduled at the beginning of the semester, and the post-intervention assessment was scheduled at a real-life stressful episode, an academic examination. For the aerobic fitness assessment, both groups completed a cardiopulmonary exercise test on the treadmill before and after the intervention. Multilevel models (MLMs) were conducted to compare within- and between-subject associations.ResultsSignificant emotional stress reactivity was evident in both groups during all assessment periods. However, participants in the AET group showed lower emotional stress reactivity compared with their control counterparts after the 20-week training programme during the real-life stress episode (the academic examination).ConclusionsAET conferred beneficial effects on emotional stress reactivity during an academic examination, which is likely an extremely stressful real-life situation for students.AET appears to be a promising strategy against the negative health effects of accumulated emotional stress reactivity.     

A Computationally More Efficient and More Accurate Stepwise Approach for Correcting for Sampling Error and Measurement Error

Over the last decade or two, multilevel structural equation modeling (ML-SEM) has become a prominent modeling approach in the social sciences because it allows researchers to correct for sampling and measurement errors and thus to estimate the effects of Level 2 (L2) constructs without bias. Because the latent variable modeling software Mplus uses maximum likelihood (ML) by default, many researchers in the social sciences have applied ML to obtain estimates of L2 regression coefficients. However, one drawback of ML is that covariance matrices of the predictor variables at L2 tend to be degenerate, and thus, estimates of L2 regression coefficients tend to be rather inaccurate when sample sizes are small. In this article, I show how an approach for stabilizing covariance matrices at L2 can be used to obtain more accurate estimates of L2 regression coefficients. A simulation study is conducted to compare the proposed approach with ML, and I illustrate its application with an example from organizational research.     

In the Shadow of Terror

Summary

This study examined the effects of terror on world assumptions in Israeli youth. The sample comprised 2,999 adolescents aged 13-16 who were exposed to different levels of terror. Relations of objective and subjective exposure to terror, life events, ideological, religious commitment, and social support with world assumptions were assessed. Results show that personal and social resources made a more substantial contribution to the explained variance of world assumptions than exposure to terror. Implications of the associations between religious and ideological commitment and social support with world assumptions are discussed.     

Multilevel Modeling with Correlated Effects

When there exist omitted effects, measurement error, and/or simultaneity in multilevel models, explanatory variables may be correlated with random components, and standard estimation methods do not provide consistent estimates of model parameters. This paper introduces estimators that are consistent under such conditions. By employing generalized method of moments (GMM) estimation techniques in multilevel modeling, the authors present a series of estimators along a robust to efficient continuum. This continuum depends on the assumptions that the analyst makes regarding the extent of the correlated effects. It is shown that the GMM approach provides an overarching framework that encompasses well-known estimators such as fixed and random effects estimators and also provides more options. These GMM estimators can be expressed as instrumental variable (IV) estimators which enhances their interpretability. Moreover, by exploiting the hierarchical structure of the data, the current technique does not require additional variables unlike traditional IV methods. Further, statistical tests are developed to compare the different estimators. A simulation study examines the finite sample properties of the estimators and tests and confirms the theoretical order of the estimators with respect to their robustness and efficiency. It further shows that not only are regression coefficients biased, but variance components may be severely underestimated in the presence of correlated effects. Empirical standard errors are employed as they are less sensitive to correlated effects when compared to model-based standard errors. An example using student achievement data shows that GMM estimators can be effectively used in a search for the most efficient among unbiased estimators. This research was supported by the National Academy of Education/Spencer Foundation and the National Science Foundation, grant number SES-0436274. We thank the editor, associate editor, and referees for detailed feedback that helped improve the paper.     

Assumptions in research in sport and exercise psychology

ObjectivesThe aim of this article is to outline how certain key assumptions affect the quality and interpretation of research in quantitative sport and exercise psychology.MethodsA review of three common assumptions made in the sport and exercise psychology literature was conducted. The review focused on three assumptions relating to research validity and the treatment and interpretation of observations. A central theme to this discussion is the assumption that research observations reflect true effects in a population.ResultsAssumptions often made in sport and exercise psychology research were identified in three key areas: (1) validity, (2) inferences of causality, and (3) effect size and the “practical significance” of research findings. Findings indicated that many studies made assumptions about the validity of the self-report psychological measures adopted and few provided a comprehensive evaluation of the validity of these measures. Researchers adopting correlational designs in sport and exercise psychology often infer causality despite such conclusions being based on theory or speculation rather than empirical evidence. Research reports still do not include effect size statistics as standard and confine the discussion of findings to statistical significance alone rather than commenting on “practical significance”.ConclusionResearch quality can only be evaluated with due consideration of the common assumptions that limits empirical investigation in sport and exercise psychology. We offer some practical advice for researchers, reviewers, and journal editors to minimise the impact of these assumptions and enhance the quality of research findings in sport and exercise psychology.     

Probabilistic argumentation systems: A new way to combine logic with probability

Probability is usually closely related to Boolean structures, i.e., Boolean algebras or propositional logic. Here we show, how probability can be combined with non-Boolean structures, and in particular non-Boolean logics. The basic idea is to describe uncertainty by (Boolean) assumptions, which may or may not be valid. The uncertain information depends then on these uncertain assumptions, scenarios or interpretations. We propose to describe information in information systems, as introduced by Scott into domain theory. This captures a wide range of systems of practical importance such as many propositional logics, first order logic, systems of linear equations, inequalities, etc. It covers thus both symbolic as well as numerical systems. Assumption-based reasoning allows then to deduce supporting arguments for hypotheses. A probability structure imposed on the assumptions permits to quantify the reliability of these supporting arguments and thus to introduce degrees of support for hypotheses. Information systems and related information algebras are formally introduced and studied in this paper as the basic structures for assumption-based reasoning. The probability structure is then formally represented by random variables with values in information algebras. Since these are in general non-Boolean structures some care must be exercised in order to introduce these random variables. It is shown that this theory leads to an extension of Dempster–Shafer theory of evidence and that information algebras provide in fact a natural frame for this theory.     

Asymptotic bias of normal-distribution-based maximum likelihood estimates of moderation effects with data missing at random

Moderation analysis is useful for addressing interesting research questions in social sciences and behavioural research. In practice, moderated multiple regression (MMR) models have been most widely used. However, missing data pose a challenge, mainly because the interaction term is a product of two or more variables and thus is a non-linear function of the involved variables. Normal-distribution-based maximum likelihood (NML) has been proposed and applied for estimating MMR models with incomplete data. When data are missing completely at random, moderation effect estimates are consistent. However, simulation results have found that when data in the predictor are missing at random (MAR), NML can yield inaccurate estimates of moderation effects when the moderation effects are non-null. Simulation studies are subject to the limitation of confounding systematic bias with sampling errors. Thus, the purpose of this paper is to analytically derive asymptotic bias of NML estimates of moderation effects with MAR data. Results show that when the moderation effect is zero, there is no asymptotic bias in moderation effect estimates with either normal or non-normal data. When the moderation effect is non-zero, however, asymptotic bias may exist and is determined by factors such as the moderation effect size, missing-data proportion, and type of missingness dependence. Our analytical results suggest that researchers should apply NML to MMR models with caution when missing data exist. Suggestions are given regarding moderation analysis with missing data.