A Measure for the Reliability of a Rating Scale Based on Longitudinal Clinical Trial Data

A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the covariance parameters obtained from a linear mixed model. The method provides a single number to express the reliability of the scale, but allows for the study of the reliability’s time evolution. The method is illustrated using a case study in schizophrenia. The authors are grateful to J&J PRD for kind permission to use their data. We gratefully acknowledge support from the Belgian IUAP/PAI network “Statistical Techniques and Modeling for Complex Substantive Questions with Complex Data.”     

Selective attention to threat in the dot probe paradigm: differentiating vigilance and difficulty to disengage

The dot probe task [J Abnorm Psychol 95 (1986) 15] is an often-used paradigm to investigate selective attention to threat. A facilitated response to probes that appear at the same location of threat information in comparison with responses to probes at the opposite location of threat information is interpreted as vigilance for threat. We argue that the findings in the dot probe paradigm are ambiguous evidence for the vigilance to threat hypothesis. Results can also be interpreted as a difficulty to disengage from threat. In this study, 44 undergraduates performed a pictorial version of the probe detection task. Taking into account the reaction times on neutral trials, we found no evidence for a facilitated detection of threatening information. It was found that the dot probe effects are at least partially due to disengagement effects. The implications of these results for the understanding of attentional bias in normal and anxious individuals are discussed.     

Return of fear in a human differential conditioning paradigm caused by a return to the original acquistion context

In a differential human fear conditioning paradigm evidence for ABA-renewal was obtained manipulating the lighting in the experimental room. During acquisition in either a dark or illuminated room, one neutral slide was sometimes paired with a loud aversive noise whereas another slide was not. Subsequently, extinction took place in the opposite lighting context. When afterwards the participants were tested again in the original acquisition context, measurements revealed a recovery of the conditioned electrodermal response and an increase in the retrospective verbal US-expectancy ratings. No response recovery was obtained in an AAA-group that received acquisition, extinction and test trials in one and the same context. Several theoretical explanations for this type of return of fear as well as implications for clinical practice are discussed.     

Proactive and retroactive transfer of middle age adults in a sequential motor learning task

We assessed the effects of aging in the transfer of motor learning in a sequential manual assembly task that is representative for real working conditions. On two different days, young (18–30 years) and middle-aged adults (50–65 years) practiced to build two products that consisted of the same six components but which had to be assembled in a partly different order. Assembly accuracy and movement time during tests, which were performed before and after the practice sessions, were compared to determine proactive and retroactive transfer.     

Reliability measures in item response theory: Manifest versus latent correlation functions

For item response theory (IRT) models, which belong to the class of generalized linear or non‐linear mixed models, reliability at the scale of observed scores (i.e., manifest correlation) is more difficult to calculate than latent correlation based reliability, but usually of greater scientific interest. This is not least because it cannot be calculated explicitly when the logit link is used in conjunction with normal random effects. As such, approximations such as Fisher's information coefficient, Cronbach's α, or the latent correlation are calculated, allegedly because it is easy to do so. Cronbach's α has well‐known and serious drawbacks, Fisher's information is not meaningful under certain circumstances, and there is an important but often overlooked difference between latent and manifest correlations. Here, manifest correlation refers to correlation between observed scores, while latent correlation refers to correlation between scores at the latent (e.g., logit or probit) scale. Thus, using one in place of the other can lead to erroneous conclusions. Taylor series based reliability measures, which are based on manifest correlation functions, are derived and a careful comparison of reliability measures based on latent correlations, Fisher's information, and exact reliability is carried out. The latent correlations are virtually always considerably higher than their manifest counterparts, Fisher's information measure shows no coherent behaviour (it is even negative in some cases), while the newly introduced Taylor series based approximations reflect the exact reliability very closely. Comparisons among the various types of correlations, for various IRT models, are made using algebraic expressions, Monte Carlo simulations, and data analysis. Given the light computational burden and the performance of Taylor series based reliability measures, their use is recommended.     

On using multiple imputation for exploratory factor analysis of incomplete data

A simple multiple imputation-based method is proposed to deal with missing data in exploratory factor analysis. Confidence intervals are obtained for the proportion of explained variance. Simulations and real data analysis are used to investigate and illustrate the use and performance of our proposal.     

Social behaviour and social cognition in high-functioning adolescents with autism spectrum disorder (ASD): two sides of the same coin?

Of the triad of symptoms found in autism spectrum disorder (ASD), that is, social impairments, communication difficulties and repetitive interests and behaviour, the social impairments are the most stable and common throughout the lifespan. They typically manifest themselves in abnormalities as reciprocal interactions and difficulties in the expression and recognition of emotions. Although peer interactions become especially important during adolescence, little is known about the mentalizing abilities of high-functioning adolescents with ASD. Here, we compared the mentalizing skills and emotion recognition abilities of 21 high-functioning adolescents with ASD and 21 matched controls. All adolescents had estimated above-average verbal intelligence levels. Spontaneous social abilities and task-related social abilities were measured using questionnaires, tasks and the Autism Diagnostic Observation Schedule. Results confirm social impairment in daily life situations in adolescents with ASD, but were not found on experimental tasks of social cognition. The use of more explicit cognitive or verbally mediating reasoning techniques and a lesser tendency of high-functioning adolescents with ASD to search for and use social information in natural environments are further discussed.     

Detecting concealed information with reaction times: Validity and comparison with the polygraph

The Concealed Information Test is used to assess recognition of information that the examinee cannot or does not want to reveal. Physiological measures recorded with the polygraph have shown to be highly valid measures of concealed information. Research suggests that a reaction‐time (RT) based test may also successfully reveal concealed information. Due to its simplicity, an RT‐based test has great advantages for applied testing. In the present study, we assessed the validity of the RT‐based test for concealed information detection, and compared its discriminative power with the polygraph. Thirty two participants in a feigned amnesia study were promised a financial reward when successfully concealing autobiographical information. Participants performed an RT‐based test, and a polygraph test. The data support the validity of the RT‐based test for concealed information detection, and indicate its discriminative power is similar to the polygraph. Our data confirm the potential of the RT‐based test. Copyright © 2009 John Wiley & Sons, Ltd.     

Development and learning: a TASC-based perspective of the acquisition of perceptual-motor behaviors

The concepts of development and learning are discussed in terms of different theoretical orientations to the study of motor behavior. It is argued that although these concepts are commonly used to discuss various aspects of change across the life span, there is little agreement regarding the definition of these terms and even whether these are useful concepts for researchers studying change over time. A theoretical approach, the TASC-based approach is presented as an alternative account for examining change over time. The TASC label stands for a focus on particular tasks, adaptation and selection of behaviors as a function of constraints. This account is grounded in evolutionary theory and assumes that variability, selection, and adaptation are central to change over time within individuals. Emphasis is placed on the tasks individuals attempt to solve in achieving particular goals given the constraints of the local environment and the organism. An alternative manner to conceptualize the concepts of development and learning are presented within the TASC-based approach.