The Latent Change Score Model: A More Flexible Approach to Modeling Time in Self-Regulated Learning

Purpose

This research advances understanding of empirical time modeling techniques in self-regulated learning research. We intuitively explain several such methods by situating their use in the extant literature. Further, we note key statistical and inferential assumptions of each method while making clear the inferential consequences of inattention to such assumptions.

Design/Methodology/Approach

Using a population model derived from a recent large-scale review of the training and work learning literature, we employ a Monte Carlo simulation fitting six variations of linear mixed models, seven variations of latent common factor models, and a single latent change score model to 1500 simulated datasets.

Findings

The latent change score model outperformed all six of the linear mixed models and all seven of the latent common factor models with respect to (1) estimation precision of the average learner improvement, (2) correctly rejecting a false null hypothesis about such average improvement, and (3) correctly failing to reject true null hypothesis about between-learner differences (i.e., random slopes) in average improvement.

Implications

The latent change score model is a more flexible method of modeling time in self-regulated learning research, particularly for learner processes consistent with twenty-first-century workplaces. Consequently, defaulting to linear mixed or latent common factor modeling methods may have adverse inferential consequences for better understanding self-regulated learning in twenty-first-century work.

Originality/Value

Ours is the first study to critically, rigorously, and empirically evaluate self-regulated learning modeling methods and to provide a more flexible alternative consistent with modern self-regulated learning knowledge.

     

Review of Issues About Classical Change Scores: A Multilevel Modeling Perspective on Some Enduring Beliefs

Change scores obtained in pretest–posttest designs are important for evaluating treatment effectiveness and for assessing change of individual test scores in psychological research. However, over the years the use of change scores has raised much controversy. In this article, from a multilevel perspective, we provide a structured treatise on several persistent negative beliefs about change scores and show that these beliefs originated from the confounding of the effects of within-person change on change-score reliability and between-person change differences. We argue that psychometric properties of change scores, such as reliability and measurement precision, should be treated at suitable levels within a multilevel framework. We show that, if examined at the suitable levels with such a framework, the negative beliefs about change scores can be renounced convincingly. Finally, we summarize the conclusions about change scores to dispel the myths and to promote the potential and practical usefulness of change scores.     

Modeling Not-Reached Items in Timed Tests: A Response Time Censoring Approach

Psychometrika - Time limits are imposed on many computer-based assessments, and it is common to observe examinees who run out of time, resulting in missingness due to not-reached items. The present...     

Eavesdropping on Health: A Naturalistic Observation Approach for Social Health Research

This article provides an overview of a novel ecological momentary assessment method called the electronically activated recorder or EAR. The EAR is a portable audio recorder that periodically records snippets of ambient sounds from participants' momentary environments. In tracking moment-to-moment ambient sounds, the EAR yields an acoustic log of a person's day as it naturally unfolds. As a naturalistic observation method, it provides an observer's account of daily life and is optimized for the assessment of audible aspects of participants' daily social environments and interactions. The article discusses the EAR method conceptually and methodologically and identifies three important ways in which it can enrich social health research. Specifically, it can help cross-validate research findings independent of self-reports, calibrate psychological measures against behavioral markers of real-world social functioning, and further our understanding of the role that people's mundane social interactions and language use play in coping and health.     

A Multivariate Multilevel Approach to the Modeling of Accuracy and Speed of Test Takers

Response times on test items are easily collected in modern computerized testing. When collecting both (binary) responses and (continuous) response times on test items, it is possible to measure the accuracy and speed of test takers. To study the relationships between these two constructs, the model is extended with a multivariate multilevel regression structure which allows the incorporation of covariates to explain the variance in speed and accuracy between individuals and groups of test takers. A Bayesian approach with Markov chain Monte Carlo (MCMC) computation enables straightforward estimation of all model parameters. Model-specific implementations of a Bayes factor (BF) and deviance information criterium (DIC) for model selection are proposed which are easily calculated as byproducts of the MCMC computation. Both results from simulation studies and real-data examples are given to illustrate several novel analyses possible with this modeling framework. The authors thank Steven Wise, James Madison University, and Pere Joan Ferrando, Universitat Rovira i Virgili, for generously making available their data sets for the empirical examples in this paper.     

A Joint Modeling Approach for Reaction Time and Accuracy in Psycholinguistic Experiments

In the psycholinguistic literature, reaction times and accuracy can be analyzed separately using mixed (logistic) effects models with crossed random effects for item and subject. Given the potential correlation between these two outcomes, a joint model for the reaction time and accuracy may provide further insight. In this paper, a Bayesian hierarchical framework is proposed that allows estimation of the correlation between time intensity and difficulty at the item level, and between speed and ability at the subject level. The framework is shown to be flexible in that reaction times can follow a (log-) normal or (shifted) Weibull distribution. A simulation study reveals the reduction in bias gains possible when using joint models, and an analysis of an example from a Dutch–English word recognition study illustrates the proposed method.     

A Multilevel Approach for Reducing Mental Health and Substance Use Disparities Affecting Bisexual Individuals

Despite bisexual individuals being at increased risk for mental health and substance use problems, clinicians’ ability to provide affirmative and competent care to bisexual clients is limited by their lack of bisexual-specific training. To address this common gap in training, this article provides a brief review of bisexual health disparities and the factors that influence them. Then, we describe a multi-level approach for improving the health and well-being of bisexual individuals. This approach addresses factors that influence health at the micro-level (e.g., strategies that clinicians can use to help bisexual clients cope with stigma-related stressors), mezzo-level (e.g., adaptations to clinical environments and training programs that promote bisexual-affirmative care), and macro-level (e.g., advocating for political change and implementing strategies to reduce prejudice against bisexual individuals at the population-level). Specifically, we describe how clinicians can adapt evidence-based interventions to tailor them to the needs of their bisexual clients. Additionally, we discuss the need for bisexual-affirmative clinical training and provide recommendations for how clinical training can be adapted to prepare clinicians to work effectively with bisexual clients. Finally, we describe how population-level interventions can be used to reduce prejudice against bisexual individuals in order to reduce bisexual health disparities. Given the striking health disparities affecting bisexual individuals, there is a critical need to develop, test, and disseminate interventions to improve the health of this population and to prepare clinicians to provide bisexual-affirmative care.     

Modeling Person-Situation Correspondence over Time: A Study of 103 Evangelical Disciple-Makers

ABSTRACT Recent Studies of person-situation correspondence demonstrate that people actively select environments that are congruent with their personality, attitudes, motives, and goals (cf. Emmons, Diener, & Larsen, 1986). But do these individual difference variables also influence a person's propensity to remain in an environment over time? To answer this question, we administered the Multidimensional Personality Questionnaire (MPQ; Tellegen, 1982; Tellegen & Waller, 1990) and the Age Universal I-E scale (Gorsuch & Venable, 1983) to a sample of 103 lay-ministers from a large, evangelical church in the upper Midwest. Subjects were participants in a church-sponsored disciple-making program, and our dependent variable was length of involvement as a disciple-maker. Results indicated that the lay-ministers endorsed an intrinsic religious orientation and had high scores on the MPQ dimensions of Control, Harmavoidance, and Traditionalism. Using survival analysis, a statistical technique for modeling event durations, we found that disciple-makers who were both low on Aggression and Harmavoidance remained in the program for the longest period. Our findings are discussed in terms of a temporally oriented model of person-situation correspondence.     

Methods in Sales Research: Scaling and Measurement Section: A Time for Change

The Fall 1991 issue of Journal of Personal Selling & Sales Management announced the establishment of the special topic section “Scaling and Measurement”. The primary purpose of the section was “.to provide an authoritative centralized source for dissemination of information related to measurement and scaling issues germane to personal selling and sales management research.”     

Community Gender Norms Change as a Part of a Multilevel Approach to Sexual Health Among Married Women in Mumbai,India

Inequitable gender norms in societies and communities negatively contribute to women’s sexual and reproductive health. While the need for change in gender norms is well recognized, the task is highly challenging in terms of intervention design, implementation and assessment of impact. This paper describes a methodology for identification of gender norms, the design of community level intervention, community participation and the assessment of intervention impact in a low income, predominately Muslim community of 600,000 people in Mumbai, India. Formative research focused on in‐depth interviews with women, men and couples yielding gender normative statements and assessment of community resources to facilitate change. A Gender Equity Scale (GES) based on this formative research was developed and administered annually for a three‐year period to random, cross‐sectional samples in the intervention and control communities, and to community based, non‐governmental organizations (NGO) staff and Imams (religious leaders) in the intervention community. NGO staff disseminated gender oriented messages to their female constituency through their regular outreach activities and through special events and festivals in the community. Imams disseminated gender messages through lectures on social issues for men attending Friday prayers. The results showed that the NGO staff and Imams, assumed more gender equitable attitudes across time. The intervention was associated with a significant improvement in attitudes towards gender equity in the intervention relative to the control community. Men showed a dramatic change in more positive gender attitudes, while women lagged behind in their GES scores. The meaning of these results are explored and the implications assessed for the generalizability of the methodology for other countries, cultures and communities.     

Thinking and Modeling at Multiple Levels: The Potential Contribution of Multilevel Modeling to Communication Theory and Research

In this introduction to the special issue on applications of multilevel modeling (MLM) to communication research, we provide a conceptual overview of the benefits of MLM—the ability to simultaneously analyze data collected at multiple levels, the ease with which it can be used to assess trends and change over time, and its incorporation of the nested structure of data in the estimation process. We highlight ways in which MLM can be used to further theory and research in communication. In addition, we comment on the applications of MLM highlighted in this special issue and echo past calls for more multilevel theorizing and analysis in the field of communication.     

Stress Prevention and Management: A Workshop Approach

This article presents a theoretical rationale for a workshop on stress prevention and management and describes the “Stress and the Classroom Teacher” workshop as an approach that has applicability in various counseling settings, including those outside the educational field. The goals of this workshop include reducing participants' sense of isolation, increasing their self-awareness of the causes and effects of distress, identifying job-related successes and strengths, and formulating a tenative plan and action steps to prevent or alleviate the distress: workshop activities are outlined. Several conclusions are drawn regarding the value and significance of this intervention mold.     

Modeling Health Behavior Change: How to Predict and Modify the Adoption and Maintenance of Health Behaviors

Health‐compromising behaviors such as physical inactivity and poor dietary habits are difficult to change. Most social‐cognitive theories assume that an individual's intention to change is the best direct predictor of actual change. But people often do not behave in accordance with their intentions. This discrepancy between intention and behavior is due to several reasons. For example, unforeseen barriers could emerge, or people might give in to temptations. Therefore, intention needs to be supplemented by other, more proximal factors that might compromise or facilitate the translation of intentions into action. Some of these postintentional factors have been identified, such as perceived self‐efficacy and strategic planning. They help to bridge the intention–behavior gap. The Health Action Process Approach (HAPA) suggests a distinction between (a) preintentional motivation processes that lead to a behavioral intention, and (b) postintentional volition processes that lead to the actual health behavior. In this article, seven studies are reported that examine the role of volitional mediators in the initiation and adherence to five health behaviors: physical exercise, breast self‐examination, seat belt use, dietary behaviors, and dental flossing. The general aim is to examine the applicability of the HAPA and its universality by replicating it across different health behaviors, based on various measures, time spans, and samples from different countries. Les comportements nuisibles à la santé tels que le manque d’exercice physique et les mauvaises habitudes alimentaires sont difficiles à modifier. La plupart des théories socio‐cognitives admettent que le meilleur prédicteur d’un réel changement est l’intention que la personne a de changer. Mais la conduite est souvent en contradiction avec les intentions. La divergence entre intention et comportement est due à plusieurs raisons. Par exemple, des obstacles imprévus peuvent apparaître, ou encore les intéressés peuvent succomber à la tentation. L’intention doit donc être associée à d’autres facteurs plus immédiats susceptibles de compromettre ou de faciliter le passage de l’intention à l’action. Certains de ces facteurs ont été identifiés, à savoir la planification stratégique et la perception de l’efficacité personnelle. Ils aident à combler le fossé entre l’intention et le comportement. L’Approche des Processus d’Action en faveur de la Santé (HAPA) établit une distinction entre les processus motivationnels préalables à l’intention qui conduisent à une intention comportementale et les processus volontaires secondaires à l’intention qui débouchent sur un comportement effectivement favorable à la santé. On expose dans cet article sept recherches qui examinent le rôle médiateur de la volition dans l’adoption de et la persévérance dans cinq comportements utiles à la santé: l’exercice physique, l’inspection personnelle de ses seins, l’usage de la ceinture de sécurité, un bon régime alimentaire et l’utilisation du fil dentaire. Il s’agit de mettre à l’épreuve le caractère opérationnel de l’HAPA et son universalité en l’appliquant à différents comportements liés à la santéà partir de mesures et de périodes diversifiées sur des échantillons provenant de différents pays.     

Effectively Obtaining Informed Consent for Child and Adolescent Participation in Mental Health Research

With the recent expansion of child mental health research, more attention is being paid to the process of informed consent for research participation. For the consent to be truly informed, it is necessary that the relevant information be both disclosed and actually understood. Traditionally, much effort has gone to ensuring the comprehensiveness of consent/assent documents, which have progressively increased in length and complexity, whereas less attention has been paid to the comprehensibility of these documents. Available data indicate that many parent and children have difficulties appreciating the research nature of treatment studies and that a higher level of formal education among the parents is associated with a greater degree of understanding. Promising approaches to achieving truly informed research participation have emerged, such as additional time for parents to meet with the researchers and using postexplanation questionnaires for identifying issues in need of further clarification. Research is needed to develop and test strategies for improving the effectiveness of the informed consent process in child mental health.     

Health Care Reform and Psychologists in Medical Schools: Stress, Challenge, and Change

Psychologists from 45 academic health science centers throughout the United States and Canada gathered at Georgetown University for a national conference November 2–5, 1995 organized by the Association of Medical School Psychologists. This paper introduces the proceedings of that conference described in the next four articles in this issue of the Journal of Clinical Psychology in Medical Settings[Volume 4, number 1]. Papers prepared by the working groups focus upon clinical services, education and training, research, and governance and administration within the academic medical setting.     

Organizational Cultural Competence in Mental Health Service Delivery: A Multilevel Analysis

Statistical associations between individual and organizational dimensions of cultural competence were examined. Participants were 350 employees of 12 public mental health agencies in Atlanta. Results from hierarchical linear modeling indicated that agencies with culturally competent mission statements and training had significantly higher member perceptions of cultural competence within the organization. Las asociaciones estadísticas entre individuo y dimensiones de la organización de la competencia cultural se examinaron. Los participantes eran 350 empleados de 12 agencias mentales públicas de la salud en Atlanta. Los resultados del modelo linear jerárquico indicó que las agencias con declaraciones de objetivos culturalmente competentes y entrenamiento tenia las percepciones apreciablemente más altas del miembro de la competencia cultural dentro de la organización.     

Some Burning Issues in Research on Health Behavior Change

This response to five excellent commentaries is intended to clarify some issues in research on health behavior change that appear to be ambiguous or controversial, such as the debate about stage models versus continuum models or the search for moderators and mediators. The assumption of stages can be useful, but the quest for truly existing stages is considered fruitless because stage is a scientific construct, not nature. The Health Action Process Approach (HAPA) is designed as an implicit or explicit stage model, based on the distinction between a motivational and a volitional phase. As a template for targeted interventions, it suggests grouping individuals into preintenders, intenders, and actors. Due to indistinct boundaries between stages and to unstable social‐cognitive constructs, the validity of stage assessment becomes the foremost problem. However, if stage‐tailored interventions turn out to be superior to nontailored interventions, then the choice of the corresponding stage model is justified. When analysing the mechanisms of health behavior change or when predicting behaviors, the HAPA is also in line with nonstage (continuum) models. In the latter case, it examines moderators and mediators within a path‐analytic research design. It is suggested that more research be conducted on moderated mediation. Cette réponse aux cinq excellents commentaires tente de clarifier certains des résultats des recherches sur la modification des comportements relatifs à la santé, résultats qui semblent ambigus ou sujets à controverse, ce qui est le cas de la discussion sur les modèles en stades ou en continuum ou encore de la recherche de modérateurs et de médiateurs. L’hypothèse des stades peut se révéler fructueuse, mais il est inutile de considérer que les stades existent réellement puisque le stade, concept scientifique, ne relève pas de la nature. L’Approche des Processus d’Action en faveur de la Santé (HAPA) est par construction un modèle en stades implicites ou explicites basé sur la distinction entre une étape motivationnelle et une étape volitionnelle. Comme cadre pour des interventions ciblées, il propose de regrouper les individus en trois catégories: ceux qui n’ont pas encore l’intention d’agir, ceux qui ont cette intention et enfin ceux qui agissent. En raison des frontières fluctuantes entre les stades et de l’instabilité des concepts socio‐cognitifs, la validité de l’évaluation des stades devient le problème principal. Cependant, si les interventions programmées à partir des stades se révèlent plus efficaces que les autres, le choix de ce modèle est défendable. Quand on analyse les mécanismes des modifications du comportement en faveur de la santé ou quand on prédit ces comportements, l’HAPA accepte aussi le modèle en continuum. En ce cas, elle aborde les modérateurs et les médiateurs avec un plan de recherche en pistes causales. Il faut développer les recherches sur modérateurs et médiateurs.