A Measure for the Reliability of a Rating Scale Based on Longitudinal Clinical Trial Data

A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the covariance parameters obtained from a linear mixed model. The method provides a single number to express the reliability of the scale, but allows for the study of the reliability’s time evolution. The method is illustrated using a case study in schizophrenia. The authors are grateful to J&J PRD for kind permission to use their data. We gratefully acknowledge support from the Belgian IUAP/PAI network “Statistical Techniques and Modeling for Complex Substantive Questions with Complex Data.”     

Prosocial behavior from early to middle childhood: genetic and environmental influences on stability and change

Prosocial behavior is important for the functioning of society. This study investigates the extent to which environment shared by family members, nonshared environment, and genetics account for children's prosocial behavior. The prosocial behavior of twins (9,424 pairs) was rated by their parents at the ages of 2, 3, 4, and 7 and by their teachers at age 7. For parent ratings, shared environmental effects decreased from .47 on average at age 2 to .03 at age 7, and genetic effects increased from .32 on average to .61. The finding of weak shared environmental effects and large heritability at age 7 was largely confirmed through the use of teacher ratings. Using longitudinal genetic analyses, the authors conclude that genetic effects account for both change and continuity in prosocial behavior and nonshared environment contributes mainly to change.     

A "Jekyll and Hyde" within: aggressive versus friendly interactions in REM and non-REM dreams

We hypothesized that representations of social interactions in REM and non-REM (NREM) dreams would reflect differing regional brain activation patterns associated with the two sleep states, and that levels of aggressive interactions would be higher in REM than in NREM dreams. One hundred REM, 100 NREM, and 100 wake reports were collected in the home from 8 men and 7 women using the Nightcap sleep-wake mentation-monitoring system and scored for number and variety of social interactions. We found that (a) social interactions were more likely to be depicted in dream than in wake reports, (b) aggressive social interactions were more characteristic of REM than NREM or wake reports, and (c) dreamer-initiated friendliness was more characteristic of NREM than REM reports. We conclude that processing of, or simulations about, selected social interactions is preferentially performed while "off-line" during the dream state, with the REM state specializing in simulation of aggressive interactions and the NREM state specializing in simulation of friendly interactions.     

Masculine girls and feminine boys: genetic and environmental contributions to atypical gender development in early childhood

In this genetic study of atypical gender role development, parents of 5,799 twin pairs, ages 3 and 4, rated their twin children's masculinity and femininity. Boys were selected as gender atypical if they were highly feminine (top 5%, 10%, or 15%) relative to other boys, and girls were selected if they were highly masculine relative to other girls. Gender-atypical boys and girls were each divided into 2 groups: fully gender atypical (e.g., feminine boys also low on masculinity) and partially gender atypical (e.g., feminine boys who are not low on masculinity). DeFries-Fulker (DF; J. C. DeFries & D. W. Fulker, 1985, 1988) extremes analysis yielded moderate group heritability and substantial shared environment effects for atypical gender role behavior. However, for fully gender-atypical girls, group heritability accounted for most of the variance, and shared environment had no effect. The results are discussed in light of past studies and potential implications for atypical gender development.     

Analysis of Recruitment Strategies: Enrolling Veterans With PTSD Into a Clinical Trial

This analysis evaluated the efficacy of a variety of recruitment methods employed in a medication trial in a veteran population. Recruitment and demographic data were collected over the course of a randomized controlled trial (RCT) assessing the efficacy of an experimental medication for posttraumatic stress disorder (PTSD). Multivariate logistic regression was used to identify demographic and recruitment variables predictive of consenting to participation in the trial. Data were collected for 1,050 potential study participants. As compared with individuals contacted by mail based on chart review, potential participants who were referred by a clinician were 3 times more likely to consent to participate, and individuals who responded to an advertisement were 5 times more likely to consent to participate. Females were more likely to respond to advertisements, whereas males were more likely to be referred. Those who were referred were significantly older than potential participants identified via the directed mailing. Clinician referral was the method most likely to lead to identifying an eligible and willing participant. When designing a recruitment plan for a clinical trial, factors such as the target sample and available staff should be carefully considered.     

Reliability measures in item response theory: Manifest versus latent correlation functions

For item response theory (IRT) models, which belong to the class of generalized linear or non‐linear mixed models, reliability at the scale of observed scores (i.e., manifest correlation) is more difficult to calculate than latent correlation based reliability, but usually of greater scientific interest. This is not least because it cannot be calculated explicitly when the logit link is used in conjunction with normal random effects. As such, approximations such as Fisher's information coefficient, Cronbach's α, or the latent correlation are calculated, allegedly because it is easy to do so. Cronbach's α has well‐known and serious drawbacks, Fisher's information is not meaningful under certain circumstances, and there is an important but often overlooked difference between latent and manifest correlations. Here, manifest correlation refers to correlation between observed scores, while latent correlation refers to correlation between scores at the latent (e.g., logit or probit) scale. Thus, using one in place of the other can lead to erroneous conclusions. Taylor series based reliability measures, which are based on manifest correlation functions, are derived and a careful comparison of reliability measures based on latent correlations, Fisher's information, and exact reliability is carried out. The latent correlations are virtually always considerably higher than their manifest counterparts, Fisher's information measure shows no coherent behaviour (it is even negative in some cases), while the newly introduced Taylor series based approximations reflect the exact reliability very closely. Comparisons among the various types of correlations, for various IRT models, are made using algebraic expressions, Monte Carlo simulations, and data analysis. Given the light computational burden and the performance of Taylor series based reliability measures, their use is recommended.     

Family size and psychological functioning in the Orthodox Jewish community

Caring for children is a known psychosocial stressor; however, its effects on psychological functioning may have substantial cross-cultural variance. We explored relationships between family size and a variety of psychological outcomes among Orthodox Jews in four separate studies: (1) an international treatment-seeking sample (n?=?82), (2) a community sample from Canada (n?=?226), (3) an out-patient clinical sample from greater New York (n?=?82), and (4) a large dyadic sample of Israeli couples (n?=?789). Surprisingly, results suggested that family size was not associated with greater stress, anxiety, depression, global functioning, family functioning, family communication, family satisfaction, or even parenting stress. It is possible that the high religious value placed on family life as well as structural adaptions in families buffer against potential stressors associated with child rearing, and further research on these potential effects is warranted.     

Verbal working memory predicts co-speech gesture: Evidence from individual differences

Gesture facilitates language production, but there is debate surrounding its exact role. It has been argued that gestures lighten the load on verbal working memory (VWM; Goldin-Meadow, Nusbaum, Kelly, & Wagner, 2001), but gestures have also been argued to aid in lexical retrieval (Krauss, 1998). In the current study, 50 speakers completed an individual differences battery that included measures of VWM and lexical retrieval. To elicit gesture, each speaker described short cartoon clips immediately after viewing. Measures of lexical retrieval did not predict spontaneous gesture rates, but lower VWM was associated with higher gesture rates, suggesting that gestures can facilitate language production by supporting VWM when resources are taxed. These data also suggest that individual variability in the propensity to gesture is partly linked to cognitive capacities.     

Reputation-like inference in domestic dogs (<Emphasis Type="Italic">Canis familiaris</Emphasis>)

Humans frequently interact with strangers absent prior direct experience with their behavior. Some conjecture that this may have favored evolution of a cognitive system within the hominoid clade or perhaps the primate order to assign reputations based on third-party exchanges. However, non-primate species’ acquisition of skills from experienced individuals, attention to communicative cues, and propensity to infer social rules suggests reputation inference may be more widespread. We utilized dogs’ sensitivity to humans’ social and communicative cues to explore whether dogs evidenced reputation-like inference for strangers through third-party interactions. Results indicated dogs spontaneously show reputation-like inference for strangers from indirect exchanges. Further manipulations revealed that dogs continued to evidence this ability despite reduction of specific components of the observed interactions, including reduction of visual social cues (i.e., face-to-face contact between the participants in the interaction) and the nature of the recipient (i.e., living, animate agent versus living, inanimate self-propelled agent). Dogs also continued to demonstrate reputation-like inference when local enhancement was controlled and in a begging paradigm. However, dogs did not evidence reputation-like inference when the observed interaction was inadvertent.