Psychometric network models from time-series and panel data

Researchers in the field of network psychometrics often focus on the estimation of Gaussian graphical models (GGMs)—an undirected network model of partial correlations—between observed variables of cross-sectional data or single-subject time-series data. This assumes that all variables are measured without measurement error, which may be implausible. In addition, cross-sectional data cannot distinguish between within-subject and between-subject effects. This paper provides a general framework that extends GGM modeling with latent variables, including relationships over time. These relationships can be estimated from time-series data or panel data featuring at least three waves of measurement. The model takes the form of a graphical vector-autoregression model between latent variables and is termed the ts-lvgvar when estimated from time-series data and the panel-lvgvar when estimated from panel data. These methods have been implemented in the software package psychonetrics, which is exemplified in two empirical examples, one using time-series data and one using panel data, and evaluated in two large-scale simulation studies. The paper concludes with a discussion on ergodicity and generalizability. Although within-subject effects may in principle be separated from between-subject effects, the interpretation of these results rests on the intensity and the time interval of measurement and on the plausibility of the assumption of stationarity.

     

The linear regression line as a judgmental aid in visual analysis of serially dependent A-B time-series data

The purpose of this study was to investigate the effects of different types and magnitudes of serial dependence (first-order moving average and autoregression) and of linear regression lines within experimental phases on the agreement between results of visual and results of statistical data analyses. The stimulus material consisted of computer-simulated A-B-design data graphs. The time series were generated with a constant variance, varying degrees of treatment effects (changes in level), five conditions of serial dependency, and with or without linear regression lines. The material was presented to three groups of student raters (n₁=52, n₂=14, n₃=17) who rated the treatment effect in the graphs on a five-point scale. These ratings were compared with statistical results (time-series analyses). Each group had to interpret 70 graphs, 35 of which had regression lines. Data were analyzed by means of two three-factor and one four-factor ANOVA and by graphic display. The linear regression lines generally enhanced the agreement between the raters' estimations and the statistical results. Serial dependency also increased the agreement between the two analysis methods. However, with strong autoregression processes in the data, the raters tended to overestimate treatment effects relative to time-series analysis.Parts of this study were presented at the World Congress on Behavior Therapy, Washington, DC, December 11, 1983. The authors wish to express their appreciation to Christoph Bonk and Willi Ecker for their extensive collaboration in data analysis and for their assistance in carrying out the study.     

Time-series analysis in operant research

A time-series method is presented, nontechnically, for analysis of data generated in individual-subject operant studies, and is recommended as a supplement to visual analysis of behavior change in reversal or multiple-baseline experiments. The method can be used to identify three kinds of statistically significant behavior change: (a) changes in score levels from one experimental phase to another, (b) reliable upward or downward trends in scores, and (c) changes in trends between phases. The detection of, and reliance on, serial dependency (autocorrelation among temporally adjacent scores) in individual-subject behavioral scores is emphasized. Examples of published data from the operant literature are used to illustrate the time-series method.     

Visual inspection of data revisited: Do the eyes still have it?

In behavior analysis, visual inspection of graphic information is the standard by which data are evaluated. Efforts to supplement visual inspection using inferential statistical procedures to assess intervention effects (e.g., analysis of variance or time-series analysis) have met with opposition. However, when serial dependence is present in the data, the use of visual inspection by itself may prove to be problematic. Previously published reports demonstrate that autocorrelated data influence trained observers' ability to identify level treatment effects and trends that occur in the intervention phase of experiments. In this report, four recent studies are presented in which autoregressive equations were used to produce point-to-point functions to simulate experimental data. In each study, various parameters were manipulated to assess trained observers' responses to changes in point-to-point functions from the baseline condition to intervention. Level shifts over baseline behavior (treatment effect), as well as no change from baseline (no treatment effect or trend), were most readily identified by observers, but trends were rarely recognized. Furthermore, other factors previously thought to augment and improve observers' responses had no impact. Results are discussed in terms of the use of visual inspection and the training of behavior analysts.     

Statistical inference in behavior analysis: Experimental control is better

Statistical inference promises automatic, objective, reliable assessments of data, independent of the skills or biases of the investigator, whereas the single-subject methods favored by behavior analysts often are said to rely too much on the investigator's subjective impressions, particularly in the visual analysis of data. In fact, conventional statistical methods are difficult to apply correctly, even by experts, and the underlying logic of null-hypothesis testing has drawn criticism since its inception. By comparison, single-subject methods foster direct, continuous interaction between investigator and subject and development of strong forms of experimental control that obviate the need for statistical inference. Treatment effects are demonstrated in experimental designs that incorporate replication within and between subjects, and the visual analysis of data is adequate when integrated into such designs. Thus, single-subject methods are ideal for shaping-and maintaining-the kind of experimental practices that will ensure the continued success of behavior analysis.     

Statistical Issues in the Study of Temporal Data: Daily Experiences

This article reviews statistical issues that arise in temporal data, particularly with respect to daily experience data. Issues related to nonindependence of observations, the nature of data structures, and claims of causality are considered. Through the analysis of data from a single subject, we illustrate concomitant time-series analysis, a general method of examining relationships between two or more series having 50 or more observations. We also discuss detection of and remedies for the problems of trend, cycles, and serial dependency that frequently plague temporal data, and present methods of combining the results of concomitant time series across subjects. Issues that arise in pooling cross-sectional and time-series data and statistical models for addressing these issues are considered for the case in which there are appreciably fewer than 50 observations and a moderate number of subjects. We discuss the possibility of using structural equation modeling to analyze data structures in which there are a large number (e.g., 200) of subjects, but relatively few time points, emphasizing the different causal status of synchronous and lagged effects and the types of models that can be specified for longitudinal data structures. Our conclusion highlights some of the issues raised by temporal data for statistical models, notably the important roles of substantive theory, the question being addressed, the properties of the data, and the assumptions underlying each technique in determining the optimal approach to statistical analysis.     

Time-series analysis of the relationship between dizziness and stress

Ten individuals with dizziness participated in a longitudinal study on the relationship between stress and dizziness. The participants rated dizziness and stress on visual analogue scales twice daily for a period of 28 days (in all 56 data points). The ratings of stress included physical stress, mental stress, emotional stress, and the presence of stressful events. The data was analysed by means of time-series analysis (ARIMA), and the temporal associations investigated by lagged correlations. Results showed concurrent associations between dizziness and mental and emotional stress. However, individual differences were observed indicating complex and diverse patterns of association between different forms of stress and dizziness.     

A generalized least squares regression approach for computing effect sizes in single-case research: application examples

A new method for deriving effect sizes from single-case designs is proposed. The strategy is applicable to small-sample time-series data with autoregressive errors. The method uses Generalized Least Squares (GLS) to model the autocorrelation of the data and estimate regression parameters to produce an effect size that represents the magnitude of treatment effect from baseline to treatment phases in standard deviation units. In this paper, the method is applied to two published examples using common single case designs (i.e., withdrawal and multiple-baseline). The results from these studies are described, and the method is compared to ten desirable criteria for single-case effect sizes. Based on the results of this application, we conclude with observations about the use of GLS as a support to visual analysis, provide recommendations for future research, and describe implications for practice.     

The deterrent effect of capital punishment during the 1950s

This investigation examines the deterrence hypothesis of an inverse relationship between state execution rates and homicides. Although this question has received some attention in recent studies, the findings of these investigations are mixed. Cross-sectional analyses of states have typically shown execution and homicide rates to be positively associated, while at least two national time-series studies report support for the deterrence hypothesis. To test whether these divergent findings are result of the two different methodologies employed (cross-sectional vs. time-series), a methodology that combines the strengths of each is used in the present study. For the period 1950 to 1960, we examine cross-sectionally for states the relationship between changes in execution rates and changes in murder rates. This analysis does not find support for the deterrence argument for the certainty of the death penalty when a number of models of the execution rate--murder rate relationship are considered, and when a variety of imprisonment and socio-demographic factors are considered as control variables.     

心理学研究数据类型与统计方法——谈函数型数据分析的引入

心理学研究数据大致分为截面数据、时间序列数据和面板数据, 三种数据类型的分析方法及使用前提因数据属性不同而有所不同。心理学截面数据的统计方法过于依赖模型的线性结构和假设条件等, 在处理心理学面板数据中难以充分发挥统计方法的功用。函数型数据分析方法主要适用于面板数据处理, 特别适宜ERP、fMRI、发展心理等心理实验中存在时间序列的面板数据的统计分析, 为心理学研究提供了有力的新工具。     

Interpreting functional analysis outcomes using automated nonparametric statistical analysis

Current methods employed to interpret functional analysis data include visual analysis and post-hoc visual inspection (PHVI). However, these methods may be biased by dataset complexity, hand calculations, and rater experience. We examined whether an automated approach using nonparametric rank-based statistics could increase the accuracy and efficiency of functional analysis data interpretation. We applied Automated Nonparametric Statistical Analysis (ANSA) to a sample of 65 published functional analyses for which additional experimental evidence was available to verify behavior function. Results showed that exact behavior function agreement between ANSA and the publications authors was 83.1%, exact agreement between ANSA and PHVI was 75.4%, and exact agreement across all 3 methods was 64.6%. These preliminary findings suggest that ANSA has the potential to support the data interpretation process. A web application that incorporates the calculations and rules utilized by ANSA is accessible at https://ansa.shinyapps.io/ansa/     

Data‐driven Methods for Modeling Social Perception

How do we model the complexity of social perception? A major methodological problem is that the space of possible variables driving social perceptions is infinitely large, thus posing an insurmountable hurdle for conventional approaches. Here, we describe a set of data‐driven methods whose objective is to identify quantitative relationships between high‐dimensional variables (e.g., visual images) and behaviors (e.g., perceptual decisions) with as little bias as possible. We focus on social perception of faces, although the methods could be applied to other visual and nonvisual categories. We review two reverse correlation approaches: (a) psychophysical methods based on judgments of images altered with randomly generated noise, where the analysis relates the random variations of the images to judgments; and (b) methods based on judgments of randomly generated faces from a statistical, multidimensional face space model, where the analysis relates the dimensions of the face model to judgments.     

The need for contextual and technical knowledge in judgmental forecasting

The purpose of this research is to assess the extent to which judgmental forecasts are improved by having more contextual and technical knowledge. Contextual information is knowledge gained by practitioners through experience on the job, consisting of general forecasting experience in the industry as well as specific product knowledge. Technical knowledge is knowledge about data analysis and formal forecasting procedures, including information on how to analyze data judgmentally. We directly compared judgmental forecasts of business practitioners with those generated by students, using 22 real-world time series. The practitioners had considerable contextual but no technical knowledge. The students had no contextual but two different levels of technical knowledge. We also generated forecasts with statistical methods to benchmark performance. Results show that contextual knowledge is particularly important in making good judgmental forecasts, while technical knowledge has little value. Practitioner forecasts are better than student forecasts in almost all comparisons. A decisive factor affecting forecast performance appears to be data variability, measured by the coefficient of variation of the time-series data. As the variability of a time series increases, the performance of all forecasts deteriorates, but judgmental forecasts by practitioners become more preferable. Statistical methods have difficulty achieving reasonable forecasts when the data are more variable, whereas judgemental forecasts reinforced by contextual information do relatively well. Data variability is one explanation for the mixed findings of past studies, relative to how well statistical techniques compare with judgment as a forecasting method.     

The impact of age on useful visual field deterioration and risk evaluation in a simulated driving task

The primary goal of this study is to evaluate whether the driver can estimate his performance and the deterioration of his state of alertness during a simulated driving task. The second goal is to study the relation between useful visual field (UVF) deterioration and the capacity to estimate performance in a visual task and the decrease of level of alertness as a function of age. In our experiment, two groups of subjects: 10 drivers between 21 and 34 years old and nine drivers between 46 and 57 years old were required to follow a vehicle in a simulated road traffic situation for 2 hours. In addition, the driver had to detect the change of colour of a signal located in the central part of his visual field or a peripheral signal appeared on the rear light of one of the vehicles in the traffic. The analysis of data collected during this visual task confirms that UVF deteriorates with the duration of the driving task and with the driver's age. The analysis of subjective data related to the state of alertness highlights an effect both of age and of the moment when this self-evaluation was carried out. However, self-evaluation of the subject's performance does not depend on driver's age. Finally, the study shows that the correlation between objective data (performance of visual task) and subjective data (state of drowsiness and self-evaluation of performance of the visual task) is low, and the implications with regard to road safety are discussed.     

The Gaussian Graphical Model in Cross-Sectional and Time-Series Data

We discuss the Gaussian graphical model (GGM; an undirected network of partial correlation coefficients) and detail its utility as an exploratory data analysis tool. The GGM shows which variables predict one-another, allows for sparse modeling of covariance structures, and may highlight potential causal relationships between observed variables. We describe the utility in three kinds of psychological data sets: data sets in which consecutive cases are assumed independent (e.g., cross-sectional data), temporally ordered data sets (e.g., n = 1 time series), and a mixture of the 2 (e.g., n > 1 time series). In time-series analysis, the GGM can be used to model the residual structure of a vector-autoregression analysis (VAR), also termed graphical VAR. Two network models can then be obtained: a temporal network and a contemporaneous network. When analyzing data from multiple subjects, a GGM can also be formed on the covariance structure of stationary means—the between-subjects network. We discuss the interpretation of these models and propose estimation methods to obtain these networks, which we implement in the R packages graphicalVAR and mlVAR. The methods are showcased in two empirical examples, and simulation studies on these methods are included in the supplementary materials.     

Are there different methods of lexical access for words presented in the left and right visual fields?

Visual field differences can arise from hemispheric specializations or perceptual asymmetries. Deciding which of the two is responsible for a particular visual field difference is a recurrent problem for researchers concerned with lateral asymmetries. In the present paper, the difficulties involved in interpreting visual field asymmetries are discussed as they apply to the Young and Ellis (1985) research on the interactive effects of word length and visual hemifield on the recognition of English words. We show that one of their critical results disappears when small changes are made to their experimental procedure. Our data demonstrate that the visual field differences Young and Ellis reported were the result of preceptual asymmetries rather than different methods of lexical access in the two cerebral hemispheres.     

The Analysis of Single-Case Research Data: Current Instructional Practices

Visual analysis is the predominant method of analysis in single-case research (SCR). However, most research suggests that agreement between visual analysts is poor, which may be due to a lack of clear guidelines and criteria for visual analysis, as well as variability in how individuals are trained. We developed a survey containing questions about the content and methods used to teach visual and statistical analysis of SCR data in verified course sequences (VCS) and distributed it via the VCS Coordinator Listserv. Thirty-seven instructors completed the survey. Results suggest that there is variability across instructors in some fundamental aspects of data analysis (e.g., number of effects required for a functional relation) but a great deal of consistency in others (e.g., emphasizing visual over statistical analysis). We discuss our results along with their implications both for teaching students to analyze SCR data and for conducting additional research on behavior-analytic training programs.

     

Home-based reinforcement: Behavioral covariation between academic performance and inappropriate behavior

This article describes the effects of a home-based reinforcement program on the academic performance and inappropriate behaviors of 3 fourth-grade boys. Home contingencies were placed on only academic performance. The home-based system resulted in marked increases in academic performance and large decreases in disruptive behavior for all three subjects. Throughout baseline and treatment conditions, there was a strong negative correlation between academic performance and disruptive behavior. The intervention was evaluated using both visual inspection (multiple baseline across subjects) and interrupted time-series analysis.     

The capacity of visual short-term memory within and between hemifields

Visual short-term memory (VSTM) and attention are both thought to have a capacity limit of four items [e.g. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 309, 279-281; Pylyshyn, Z. W., & Storm, R. W. (1988). Tracking multiple independent targets: evidence for a parallel tracking mechanism. Spatial Vision, 3, 179-197.]. Using the multiple object visual tracking paradigm (MOT), it has recently been shown that twice as many items can be simultaneously attended when they are separated between two visual fields compared to when they are all presented within the same hemifield [Alvarez, G. A., & Cavanagh, P. (2004). Independent attention resources for the left and right visual hemifields (Abstract). Journal of Vision, 4(8), 29a.]. Does VSTM capacity also increase when the items to be remembered are distributed between the two visual fields? The current paper investigated this central issue in two different tasks, namely a color and spatial location change detection task, in which the items were displayed either in the two visual fields or in the same hemifield. The data revealed that only memory capacity for spatial locations and not colors increased when the items were separated between the two visual fields. These findings support the view of VSTM as a chain of capacity limited operations where the spatial selection of stimuli, which dominates in both spatial location VSTM and MOT, occupies the first place and shows independence between the two fields.     

Trend and rhythm analysis of time-series data using complex demodulation

Biological time-series data collected over long intervals generally show combined systematic and periodic fluctuations. Comprehensive analysis of such data requires separation of the trend and rhythmic components. Most available time-series analytic techniques do not explicitly extract the trend, and do implicitly assume the underlying rhythms are simple symmetrical sinusoids, whose amplitude and phase values remain constant throughout the recorded interval. Neither assumption is very accurate when dealing with biological data, and the stationarity assumption in particular becomes harder to defend as experiments extend over days or even weeks. Complex demodulation (CD) is described here as a technique for separation of trend from cyclic components, and multiple complex demodulation (MCD) as a technique for extraction of all possible frequencies in the data set, along with their moment-by-moment amplitude and phase values.