Pooling data from multiple longitudinal studies: the role of item response theory in integrative data analysis

There are a number of significant challenges researchers encounter when studying development over an extended period of time, including subject attrition, the changing of measurement structures across groups and developmental periods, and the need to invest substantial time and money. Integrative data analysis is an emerging set of methodologies that allows researchers to overcome many of the challenges of single-sample designs through the pooling of data drawn from multiple existing developmental studies. This approach is characterized by a host of advantages, but this also introduces several new complexities that must be addressed prior to broad adoption by developmental researchers. In this article, the authors focus on methods for fitting measurement models and creating scale scores using data drawn from multiple longitudinal studies. The authors present findings from the analysis of repeated measures of internalizing symptomatology that were pooled from three existing developmental studies. The authors describe and demonstrate each step in the analysis and conclude with a discussion of potential limitations and directions for future research.     

Experience Sampling Methods: A Modern Idiographic Approach to Personality Research

Experience sampling methods are essential tools for building a modern idiographic approach to understanding personality. These methods yield multiple snapshots of people's experiences over time in daily life and allow researchers to identify patterns of behavior within a given individual, rather than strictly identify patterns of behavior across individuals, as with standard nomothetic approaches. In this article, we discuss the origin and evolution of idiographic methods in the field of personality and explain how experience sampling methods function as modern day idiographic methods in this field. We then review four primary ways in which experience sampling methods have been used to foster idiographic approaches in personality research. Specifically, we highlight approaches that examine individual differences in temporal and behavioral distributions, situation–behavior contingencies, daily processes, and the structure of daily experience. Following a brief methodology primer, we end by discussing future directions for idiographic experience sampling approaches in personality psychology and beyond.     

Idiographic network analysis of discrete mood states prior to treatment

Idiographic network models based on time‐series data have received recent attention for their ability to model relationships among symptoms and behaviours as they unfold in time within a single individual (cf. Epskamp, Borsboom, & Fried, 2018; Fisher, Medaglia, & Jeronimus, 2018). Rather than examine the correlational relationships between variables in a sample of individuals, an idiographic network examines correlations within a single person, averaged over many time points. Because the approach averages over time, the data must be stationary (i.e. relatively consistent over time). If individuals experience varying states over time—different mixtures of symptoms and behaviours in one moment or another—then averaging over categorically different moments may undermine model accuracy. Fisher and Bosley (2019) address these concerns via the application of Gaussian finite mixture modelling to identify latent classes of time points in intraindividual time‐series data from a sample of adults with major depressive disorder and/or generalised anxiety disorder (n = 45). The present paper outlines an extension of this work, wherein network analysis is used to model within‐class covariation of symptoms. To illustrate this approach, network models were constructed for each intraindividual class identified by Fisher and Bosley (137 networks across the 45 participants, mean classes/person = ~3, range = 2–4 classes/person). We examine the relative consistency in symptom organisation between each individual's multiple mood state networks and assess emergent group‐level patterns. We highlight opportunities for enhanced treatment personalisation and review nomothetic patterns relevant to transdiagnostic conceptualisations of psychopathology. We address opportunities for integrating this approach into clinical practice and outline potential shortcomings.     

Effects of setting and number of observations on generalizability of parent-child interactions in childhood obesity treatment

The generalizability of behaviors across observational conditions is a critical issue in behavioral assessment. Generalizability theory was used to examine two aspects of audio recorded parent-child interactions recorded over 6 days of home measurement and 1 day of laboratory measurement in a behavioral treatment program for childhood obesity. Families audiotaped parent-child home meetings during which they reviewed self-monitored diet and exercise records that were coded for the following types of interactions: praise statements, negative statements, prompts for new behaviors, and statements promoting problem solving. A similar meeting was audiotaped in our laboratory. The first question explored was the number of measurements needed to generalize to the universe of the six home measures. Results showed an increase in generalizability over measurements for each behavioral category. Using generalizability coefficients of .60 or more, praise, negative comments and prompts, respectively, could be reliably observed based on 1, 4, or 4 days of measurement. Second, the effects of setting (laboratory versus home) were assessed for 1 day of measurement in each environment. Again using generalizability coefficients of .60, generalizability analysis showed that the lab setting could not be generalized to the home setting based on 1 day of measurement, with generalizability coefficients ranging from .27 for negative comments to .57 for praise. Results suggest that 4 days of behavioral assessment in the home can be used to establish generalizable data for all the dependent measures studied. However, generalizability coefficients suggested that 1 day of laboratory measurement was not adequate to generalize to typical home behavior.This research was supported in part by Grant NIH HD 23713 awarded to the third author.     

A Survey of the Sources of Noise in fMRI

Functional magnetic resonance imaging (fMRI) is a noninvasive method for measuring brain function by correlating temporal changes in local cerebral blood oxygenation with behavioral measures. fMRI is used to study individuals at single time points, across multiple time points (with or without intervention), as well as to examine the variation of brain function across normal and ill populations. fMRI may be collected at multiple sites and then pooled into a single analysis. This paper describes how fMRI data is analyzed at each of these levels and describes the noise sources introduced at each level.     

Implicit psychophysiology: Effects of common beliefs and idiosyncratic physiological responses symptom reporting

Every individual exhibits unique perceptual, behavioral, and physiological responses within and across a variety of settings. Despite the idiosyncratic nature of responses, we seek to establish theories that generalize across a large number of individuals. A strict idiographic method intensively examines the response patterns of a small number of individuals, whereas a nomothetic approach focuses on common responses across a large number of individuals. In the present investigation, we seek to learn how individuals perceive and report physical symptoms and sensations. We offer a methodology that capitalizes on the unique physiological responses of individuals but, at the same time, assumes that the underlying perceptual processes relevant to symptom reporting are comparable across individuals. Our approach, then, is both idiographic and nomothetic. As will be discussed, this integrative approach has the potential to be applied to a multitude of behaviors and processes that are of interest to social and personality psychologists.     

Intraindividual Dynamic Network Analysis – Implications for Clinical Assessment

A network analysis approach to psychopathology regards symptoms as mutually interacting components of a multifaceted system (Borsboom & Cramer, 2013). Although several studies using this approach have examined comorbidity between disorders using cross-sectional samples, a direct application of the network analysis approach to intraindividual dynamic relations between symptoms in a complex, comorbid case has not been reported. The current article describes an intraindividual dynamic network analysis (IDNA) approach to understanding the psychopathology of an individual using dynamic (over time) lead-lag interrelations between symptoms. Multivariate time series data were utilized to create and examine an intraindividual, lag-1 network of the partial, day-to-day relations of symptoms in an individual with comorbid mood and anxiety disorders. Characteristics of the network, including centrality indices, stability, dynamic processes between symptoms, and their implications for clinical assessment are described. Additional clinical implications and future directions for IDNA, including the potential incremental validity of this assessment approach for empirically-based idiographic assessment and personalized treatment planning, are discussed. This person-specific IDNA approach may be especially useful in complex and comorbid cases.     

Idiographic outcome analyses of the clinical significance of two interventions for patients with musculoskeletal pain

This study adopted a perspective of the individual to define domains of everyday life for the analysis of clinically meaningful change. The purpose was to compare the clinical significance of two interventions for patients with musculoskeletal pain, applying an idiographic outcome measure, The Patient Goal Priority Questionnaire, in combination with the Jacobson and Truax methodology [(1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 67 (3), 300-307] for determination of clinical significance. The concurrent validity of the outcome variables behavioral performance, satisfaction with behavioral performance, and fulfilled pre-treatment expectations was also studied. Eighty-two patients, randomized to either individually tailored behavioral medicine treatment (experimental group) or physical exercise therapy (control group) were evaluated at baseline and 3 months post-treatment regarding behavioral treatment goals. The experimental intervention had high impact on participants' performance of their highest ranked everyday life activities, and resulted in larger proportions of clinically significant outcomes compared with controls. The concurrent validity of the outcomes was high for those reporting clinically significant changes, but more generally, there was a moderate agreement across outcome categories. The individual should be the unit for analyses of clinical significance to enhance the ecological validity of the construct. Further development of idiographic outcome measures is necessary, as is the inclusion in pain intervention research.     

Multilevel analysis of matching behavior

Multilevel modeling has been considered a promising statistical tool in the field of the experimental analysis of behavior and may serve as a convenient statistical analysis for matching behavior because it structures data in groups (or levels) to account simultaneously for the within‐subject and between‐subject variances. Heretofore, researchers have sometimes pooled data erroneously from different subjects in a single analysis by using average ratios, average response and reinforcer rates, aggregation of subjects, etc. Unfortunately, this leads to loss of information and biased estimations, which can severely undermine generalization of the results. Instead, a multilevel approach is advocated to combine several subjects' matching behavior. A reanalysis of previous data on matching behavior is provided to illustrate the method and point out its advantages. It illustrates that multilevel regression leads to better estimations, is more convenient, and offers more behavioral information. We hope this paper will encourage the use of multilevel modeling in the statistical practices of behavior analysts.     

A Moderated Nonlinear Factor Model for the Development of Commensurate Measures in Integrative Data Analysis

Integrative data analysis (IDA) is a methodological framework that allows for the fitting of models to data that have been pooled across 2 or more independent sources. IDA offers many potential advantages including increased statistical power, greater subject heterogeneity, higher observed frequencies of low base-rate behaviors, and longer developmental periods of study. However, a core challenge is the estimation of valid and reliable psychometric scores that are based on potentially different items with different response options drawn from different studies. In Bauer and Hussong (2009) we proposed a method for obtaining scores within an IDA called moderated nonlinear factor analysis (MNLFA). Here we move significantly beyond this work in the development of a general framework for estimating MNLFA models and obtaining scale scores across a variety of settings. We propose a 5-step procedure and demonstrate this approach using data drawn from n = 1,972 individuals ranging in age from 11 to 34 years pooled across 3 independent studies to examine the factor structure of 17 binary items assessing depressive symptomatology. We offer substantive conclusions about the factor structure of depression, use this structure to compute individual-specific scale scores, and make recommendations for the use of these methods in practice.     

A Clinician’s Primer for Idiographic Research: Considerations and Recommendations

Theorists and clinicians have long noted the need for idiographic (i.e., individual-level) designs within clinical psychology. Results from idiographic work may provide a possible resolution of the therapist’s dilemma—the problem of treating an individual using information gathered via group-level research. Due to advances in data collection and time series methodology, there has been increasing interest in using idiographic designs to answer clinical questions. Although time series methods have been well-studied outside the field of clinical psychology, there is limited direction on how clinicians can use such models to inform their clinical practice. In this primer, we collate decades of published and word-of-mouth information on idiographic designs, measurement, and modeling. We aim to provide an initial guide on the theoretical and practical considerations that we urge interested clinicians to consider before conducting idiographic work of their own.     

AMAB: Automated measurement and analysis of body motion

Technologies that measure human nonverbal behavior have existed for some time, and their use in the analysis of social behavior has become more popular following the development of sensor technologies that record full-body movement. However, a standardized methodology to efficiently represent and analyze full-body motion is absent. In this article, we present automated measurement and analysis of body motion (AMAB), a methodology for examining individual and interpersonal nonverbal behavior from the output of full-body motion tracking systems. We address the recording, screening, and normalization of the data, providing methods for standardizing the data across recording condition and across subject body sizes. We then propose a series of dependent measures to operationalize common research questions in psychological research. We present practical examples from several application areas to demonstrate the efficacy of our proposed method for full-body measurements and comparisons across time, space, body parts, and subjects.     

EXPERIMENTAL ANALYSIS AND TREATMENT OF MULTIPLY CONTROLLED SELF-INJURY

A functional analysis of the self-injurious behavior (SIB) of 3 adults with profound developmental disabilities showed that each engaged in SIB in more than one assessment condition. Such outcomes may result from a failure to isolate the variable maintaining SIB, or they may reflect multiple sources of control over SIB. In order to identify more clearly the determinants of SIB, each subject was exposed to a series of treatments appropriate to one or both of the apparent functions of SIB. These treatments, applied sequentially on baselines appropriate to each behavioral function, identified the maintaining variables for SIB through differential outcomes across baselines. Results indicated that the SIB of 2 subjects was multiply controlled, confirming the outcomes of the functional analysis. However, the SIB of the 3rd subject was eliminated using a treatment designed for a single function, suggesting spurious results of the original assessment. Alternative interpretations of undifferentiated assessment data are discussed, as are analysis and treatment issues related to multiply determined behavior disorders.     

Conceptualizing and Estimating Process Speed in Studies Employing Ecological Momentary Assessment Designs: A Multilevel Variance Decomposition Approach

Researchers have been making use of ecological momentary assessment (EMA) and other study designs that sample feelings and behaviors in real time and in naturalistic settings to study temporal dynamics and contextual factors of a wide variety of psychological, physiological, and behavioral processes. As EMA designs become more widespread, questions are arising about the frequency of data sampling, with direct implications for participants' burden and researchers' ability to capture and study dynamic processes. Traditionally, spectral analytic techniques are used for time series data to identify process speed. However, the nature of EMA data, often collected with fewer than 100 measurements per person, sampled at randomly spaced intervals, and replete with planned and unplanned missingness, precludes application of traditional spectral analytic techniques. Building on principles of variance partitioning used in the generalizability theory of measurement and spectral analysis, we illustrate the utility of multilevel variance decompositions for isolating process speed in EMA-type data. Simulation and empirical data from a smoking-cessation study are used to demonstrate the method and to evaluate the process speed of smoking urges and quitting self-efficacy. Results of the multilevel variance decomposition approach can inform process-oriented theory and future EMA study designs.     

Statistical Issues in the Study of Temporal Data: Daily Experiences

This article reviews statistical issues that arise in temporal data, particularly with respect to daily experience data. Issues related to nonindependence of observations, the nature of data structures, and claims of causality are considered. Through the analysis of data from a single subject, we illustrate concomitant time-series analysis, a general method of examining relationships between two or more series having 50 or more observations. We also discuss detection of and remedies for the problems of trend, cycles, and serial dependency that frequently plague temporal data, and present methods of combining the results of concomitant time series across subjects. Issues that arise in pooling cross-sectional and time-series data and statistical models for addressing these issues are considered for the case in which there are appreciably fewer than 50 observations and a moderate number of subjects. We discuss the possibility of using structural equation modeling to analyze data structures in which there are a large number (e.g., 200) of subjects, but relatively few time points, emphasizing the different causal status of synchronous and lagged effects and the types of models that can be specified for longitudinal data structures. Our conclusion highlights some of the issues raised by temporal data for statistical models, notably the important roles of substantive theory, the question being addressed, the properties of the data, and the assumptions underlying each technique in determining the optimal approach to statistical analysis.     

Conceptualizing and Estimating Process Speed in Studies Employing Ecological Momentary Assessment Designs: A Multilevel Variance Decomposition Approach

Researchers have been making use of ecological momentary assessment (EMA) and other study designs that sample feelings and behaviors in real time and in naturalistic settings to study temporal dynamics and contextual factors of a wide variety of psychological, physiological, and behavioral processes. As EMA designs become more widespread, questions are arising about the frequency of data sampling, with direct implications for participants' burden and researchers' ability to capture and study dynamic processes. Traditionally, spectral analytic techniques are used for time series data to identify process speed. However, the nature of EMA data, often collected with fewer than 100 measurements per person, sampled at randomly spaced intervals, and replete with planned and unplanned missingness, precludes application of traditional spectral analytic techniques. Building on principles of variance partitioning used in the generalizability theory of measurement and spectral analysis, we illustrate the utility of multilevel variance decompositions for isolating process speed in EMA-type data. Simulation and empirical data from a smoking-cessation study are used to demonstrate the method and to evaluate the process speed of smoking urges and quitting self-efficacy. Results of the multilevel variance decomposition approach can inform process-oriented theory and future EMA study designs.     

Thematic moment analysis: A didactic application of a procedure for phenomenological analysis of narrative data

Abstract

Following in the theoretical and methodological roots of the ‘Duquesne school,’ this work presents a procedure for the phenomenological analysis of narrative data. After describing its ‘approach,’ this work lays out the procedure for ‘thematic moment analysis’ using the work of Giorgi as a point of reference. Idiographic results of single datum of being afraid are presented along with the steps through which this ‘idiographic thematic narrative’ was developed. Procedures for arriving at ‘comparative analysis’ of more than one piece of data are presented and distinguished from claims of generality sought by works drawing on larger data pools. Finally, this work discusses issues of internal and external validity both specifically in regards to the ‘thematic moments’ procedure and in terms of phenomenological research in general.     

Comparison of Nomothetic Versus Idiographic-Oriented Methods for Making Predictions About Distal Outcomes From Time Series Data

We illustrate the idiographic/nomothetic debate by comparing 3 approaches to using daily self-report data on affect for predicting relationship quality and breakup. The 3 approaches included (a) the first day in the series of daily data; (b) the mean and variability of the daily series; and (c) parameters from dynamic factor analysis, a statistical model that uses all measurement occasions to estimate the structure and dynamics of the data. Our results indicated that data from the first measurement occasion does not provide information about the couples' relationship quality or breakup 1 to 2 years later. The mean and variability of the time series, however, were more informative: females' average positive and negative affect across time was related to relationship quality, whereas males' variability in negative affect across time was predictive of breakup. The dynamic factor analysis, in turn, allowed us to extract information central to the dyadic dynamics. This information proved useful to predict relationship quality but not breakup. The importance of examining intraindividual variability and couple dynamics is highlighted.     

空气污染的不良效应及理论模型:环境心理学的视角

空气污染的不良效应不仅限于生理健康损害,还涉及认知功能、情绪和行为等多方面的消极影响。针对空气污染不良效应的形成机制,以往研究提出了环境应激模型和社会-环境交互模型,这些理论模型强调空气污染的不良效应不仅涉及污染的直接暴露程度,也取决于个体的身心状况以及他对空气污染的主观评价。基于以往研究的不足,尤其是环境心理学介入空气污染研究的可行性,未来研究可着眼于改进空气污染的心理学研究方法,开展空气污染的环境应激和风险感知研究。     

Time Series Analysis in Historiometry: A Comment on Simonton

Time series analysis (TSA) is one of a number of new methods of data analysis appropriate for longitudinal data. Simonton (1998) applied TSA to an analysis of the causal relationship between two types of stress and both the physical and mental health of George III. This innovative application demonstrates both the strengths and weaknesses of time series analysis. Time series is applicable to a unique class of problems, can use information about temporal ordering to make statements about causation, and focuses on patterns of change over time, all strengths of the Simonton study. Time series analysis also suffers from a number of weaknesses, including problems with generalization from a single study, difficulty in obtaining appropriate measures, and problems with accurately identifying the correct model to represent the data. While careful attempts are made to minimize these problems, each is present in the Simonton study, although sometimes in a subtle manner. Changes in how the data could be gathered are suggested that might help to solve some of these problems in future studies. Finally, the advantages and disadvantages of employing alternative methods for analyzing multivariate time series data, including dynamic factor analysis, are discussed.