Estimating Between-Person and Within-Person Subscore Reliability with Profile Analysis

Subscores are of increasing interest in educational and psychological testing due to their diagnostic function for evaluating examinees' strengths and weaknesses within particular domains of knowledge. Previous studies about the utility of subscores have mostly focused on the overall reliability of individual subscores and ignored the fact that subscores should be distinct and have added value over the total score. This study introduces a profile reliability approach that partitions the overall subscore reliability into within-person and between-person subscore reliability. The estimation of between-person reliability and within-person reliability coefficients is demonstrated using subscores from number-correct scoring, unidimensional and multidimensional item response theory scoring, and augmented scoring approaches via a simulation study and a real data study. The effects of various testing conditions, such as subtest length, correlations among subscores, and the number of subtests, are examined. Results indicate that there is a substantial trade-off between within-person and between-person reliability of subscores. Profile reliability coefficients can be useful in determining the extent to which subscores provide distinct and reliable information under various testing conditions.     

Ecological Momentary Assessment Research in Behavioral medicine

Behavioral medicine research often seeks to examine relatively complex theoretical models and clinical processes. Traditional designs, based on retrospective self-report in the natural environment or laboratory-based studies, cannot assess complex and temporally dynamic psychological, behavioral, and physiological processes in the natural environment. An alternate research approach, Ecological Momentary Assessment (EMA), provides a method to collect such data. This research approach and these data allow investigation of a number of research questions not amenable to other data collection methods. Examples of research questions uniquely suited for EMA data include, but are not limited to: (1) the comparison of retrospective data to momentary data; (2) evaluation of physiological and psychosocial processes in the natural environment; (3) the relation of symptoms and other disease processes to characteristics of the natural environment; and (4) examination of dynamic changes in psychosocial variables that may mediate improvement following treatment. Despite the strengths of EMA data, a number of challenges exist – including logistic, analytic, and interpretation problems – to the easy implementation of EMA approaches in behavioral medicine.     

Conceptualizing and Estimating Process Speed in Studies Employing Ecological Momentary Assessment Designs: A Multilevel Variance Decomposition Approach

Researchers have been making use of ecological momentary assessment (EMA) and other study designs that sample feelings and behaviors in real time and in naturalistic settings to study temporal dynamics and contextual factors of a wide variety of psychological, physiological, and behavioral processes. As EMA designs become more widespread, questions are arising about the frequency of data sampling, with direct implications for participants' burden and researchers' ability to capture and study dynamic processes. Traditionally, spectral analytic techniques are used for time series data to identify process speed. However, the nature of EMA data, often collected with fewer than 100 measurements per person, sampled at randomly spaced intervals, and replete with planned and unplanned missingness, precludes application of traditional spectral analytic techniques. Building on principles of variance partitioning used in the generalizability theory of measurement and spectral analysis, we illustrate the utility of multilevel variance decompositions for isolating process speed in EMA-type data. Simulation and empirical data from a smoking-cessation study are used to demonstrate the method and to evaluate the process speed of smoking urges and quitting self-efficacy. Results of the multilevel variance decomposition approach can inform process-oriented theory and future EMA study designs.     

Conceptualizing and Estimating Process Speed in Studies Employing Ecological Momentary Assessment Designs: A Multilevel Variance Decomposition Approach

Researchers have been making use of ecological momentary assessment (EMA) and other study designs that sample feelings and behaviors in real time and in naturalistic settings to study temporal dynamics and contextual factors of a wide variety of psychological, physiological, and behavioral processes. As EMA designs become more widespread, questions are arising about the frequency of data sampling, with direct implications for participants' burden and researchers' ability to capture and study dynamic processes. Traditionally, spectral analytic techniques are used for time series data to identify process speed. However, the nature of EMA data, often collected with fewer than 100 measurements per person, sampled at randomly spaced intervals, and replete with planned and unplanned missingness, precludes application of traditional spectral analytic techniques. Building on principles of variance partitioning used in the generalizability theory of measurement and spectral analysis, we illustrate the utility of multilevel variance decompositions for isolating process speed in EMA-type data. Simulation and empirical data from a smoking-cessation study are used to demonstrate the method and to evaluate the process speed of smoking urges and quitting self-efficacy. Results of the multilevel variance decomposition approach can inform process-oriented theory and future EMA study designs.     

An Ecological Momentary Assessment Study Investigating the Function of Hoarding

This study examined the function of hoarding behaviors and the relations between hoarding and a series of cognitive and affective processes in the moment using ecological momentary assessment. A matched-groups design was used to compare college students with higher hoarding symptoms (n = 31) and matched controls (n = 29). The two groups did not differ in what function they reported acquiring served, and positive automatic reinforcement was the most commonly reported function in both groups. Engaging in hoarding-relevant behaviors did not predict change in positive or negative affect when controlling for previous affect. Emotional reactivity and experiential avoidance in the moment were both elevated in the higher hoarding group compared to controls, while momentary mindfulness and negative affect differentiation were lower. Overall, these findings support the importance of emotion regulation processes in hoarding. They also suggest individuals may not be successfully regulating affect in the moment with hoarding behaviors, despite efforts to do so. It may be useful to evaluate processes such as striving for positive affect in hoarding disorder in the future.     

Multilevel Heterogeneous Factor Analysis and Application to Ecological Momentary Assessment

Psychometrika - Ansari et al. (Psychometrika 67:49–77, 2002) applied a multilevel heterogeneous model for confirmatory factor analysis to repeated measurements on individuals. While the mean...     

应对方式的生态瞬时评估法及其他测量方法简介

生态瞬时评估法（EMA）是当前对应对方式进行测量的一种较为先进的方法。它也广泛适用于其它社会科学和行为科学领域,因此在西方发达国家得到了较为广泛的应用。文章重点介绍了EMA的特点、使用方法、使用情况、适用条件和优缺点等。由于EMA特别适合于研究过程性的问题,因此用它来研究应对是非常恰当的。另外,文章还简单介绍并评价了其它的应对方式测量方法,并指出将来对应对方式进行测量的趋势。     

A Mixed Model to Disentangle Variance and Serial Autocorrelation in Affective Instability Using Ecological Momentary Assessment Data

Affective instability, the tendency to experience emotions that fluctuate frequently and intensively over time, is a core feature of several mental disorders including borderline personality disorder. Currently, affect is often measured with Ecological Momentary Assessment protocols, which yield the possibility to quantify the instability of affect over time. A number of linear mixed models are proposed to examine (diagnostic) group differences in affective instability. The models contribute to the existing literature by estimating simultaneously both the variance and serial dependency component of affective instability when observations are unequally spaced in time with the serial autocorrelation (or emotional inertia) declining as a function of the time interval between observations. In addition, the models can eliminate systematic trends, take between subject differences into account and test for (diagnostic) group differences in serial autocorrelation, short-term as well as long-term affective variability. The usefulness of the models is illustrated in a study on diagnostic group differences in affective instability in the domain of eating disorders. Limitations of the model are that they pertain to group (and not individual) differences and do not focus explicitly on circadian rhythms or cycles in affect.     

Latent Variable Mixture Modeling of Ecological Momentary Assessment Data: Implications for Screening and Adolescent Mood Profiles

Ecological momentary assessment (EMA) studies typically rely on arbitrary decision rules for identifying and excluding invalid responses from the data. In addition, most studies treat independent variables as separate from each other even if their combinations might have importance above the independent contribution of each. Our study aimed to conduct an exploratory latent profile analysis of EMA data to demonstrate an empirical method of identifying invalid responses, and to provide a preliminary investigation of mood profiles. We recruited 20 adolescents between the ages of 13–18 to complete 4 surveys about their internal states each day for 20 days. Participants provided responses on study smartphones using an Android app developed by the study team. Our profile analysis revealed 9 independent profiles. We determined that 3 of these profiles consisted of invalid responses because the integers provided by the participant were nearly invariant. The invalid responses comprised 24.9% of the sample. We also identified 6 valid profiles that were labeled: fatigued (8.7%), good mood/energetic (19.9%), angry/depressed (2.3%), good mood (37.1%), angry (5.7%), and depressed (1.4%). One important implication of the current study is that researchers and clinicians should screen electronic diary data, especially for invariant responding. In addition, it is important for clinicians to note that more than one internal state may drive the mood of an adolescent patient.     

The (Non)Impact of Differential Test Taker Engagement on Aggregated Scores

Disengaged test taking tends to be most prevalent with low-stakes tests. This has led to questions about the validity of aggregated scores from large-scale international assessments such as PISA and TIMSS, as previous research has found a meaningful correlation between the mean engagement and mean performance of countries. The current study, using data from the computer-based version of the PISA-Based Test for Schools, examined the distortive effects of differential engagement on aggregated school-level scores. The results showed that, although there was considerable differential engagement among schools, the school means were highly stable due to two factors. First, any distortive effects of disengagement in a school were diluted by a high proportion of the students exhibiting no non-effortful behavior. Second, and most interestingly, disengagement produced both positive and negative distortion of individual student scores, which tended to cancel out much of the net distortive effect on the school’s mean.     

Motivational Text Message Intervention for Eating Disorders: A Single-Case Alternating Treatment Design Using Ecological Momentary Assessment

ObjectiveThis study tested a motivational text message treatment adjunct for individuals with eating disorders (EDs) who exhibited high dietary restraint/restriction. Method: A replicated single-case alternating treatment design was used to examine (a) the feasibility of combining a brief motivational interview with subsequent text messages and (b) the influence of the text messages on eating behaviors and motivation to change in individuals with EDs (N = 12). The protocol was 8 weeks and the text messages were adjunctive to cognitive-behavioral therapy. Results: The intervention was well accepted (mean rating = 7/10) and feasible within the context of monetary compensation (mean daily monitoring compliance = 91%). Text messages did not impact behavioral outcomes: dietary restraint and kilocalorie intake. They had mixed effects on motivation to change dietary restraint, measured by the Readiness and Motivation Questionnaire (RMQ). When receiving text messages, RMQ precontemplation scores (desire to restrict) significantly increased, indicating decreased motivation; however, action scores (effort toward reducing dietary restraint) significantly increased, indicating increased motivation. These effects were moderated by weight status. Underweight individuals (n = 4; body mass index [BMI] < 19.0) reported increased ambivalence—that is, an increased desire to restrict and increased action toward reducing restriction—in response to the text messages. Normal weight participants (n = 8; BMI > 19.0) reported only increased action toward reducing restriction in response to the text messages. Discussion: These data demonstrate text messages are a potentially feasible and acceptable treatment adjunct and may be effective at increasing motivation to change for normal weight individuals, while their influence on underweight patients is more complex. These findings provide a foundation for future research in technology-based motivational interventions for EDs and offer preliminary evidence for using these methods among normal weight individuals.     

The Effects of Appearance-Based Comments and Non-Appearance-Based Evaluations on Body Dissatisfaction and Disordered Eating Urges: An Ecological Momentary Assessment Study

The current study used ecological momentary assessment (EMA) to investigate whether appearance-based comments, social and performance-based evaluations affected levels of body dissatisfaction (BD) and urges to engage in disordered eating behaviors (DE) throughout daily life. A total of 620 participants completed a baseline questionnaire assessing sociodemographic variables. Participants then downloaded a mobile app which alerted them to complete short surveys assessing their levels of BD, DE urges, and experiences of receiving comments and evaluations six times per day for 7 days. Negative appearance-based comments predicted greater levels of state BD, while positive appearance comments predicted lower levels of state BD. Negative social and performance-based evaluations predicted an increase in state BD, while positive evaluations predicted a decrease in this outcome variable. No significant predictor was found for the DE urge outcomes. The present findings suggest that receiving negative and positive feedback in various domains of one’s life may predict opposite outcomes for body image. However, these effects do not necessarily associate with urges to engage in DE in a nonclinical population.     

Reliability and Validity of Scores From the Inventory of Drug Use Consequences

Psychometric properties of the Inventory of Drug Use Consequences (InDUC; W. M. Miller, J. S. Tonigan, & R. Longabaugh, 1995), a self‐report assessment of negative consequences associated with alcohol and other drug use, were investigated. The InDUC demonstrated sound psychometric characteristics and can be a valuable clinical tool for chemical dependency counselors.