An empirical Q-matrix validation method for the sequential generalized DINA model

As a core component of most cognitive diagnosis models, the Q-matrix, or item and attribute association matrix, is typically developed by domain experts, and tends to be subjective. It is critical to validate the Q-matrix empirically because a misspecified Q-matrix could result in erroneous attribute estimation. Most existing Q-matrix validation procedures are developed for dichotomous responses. However, in this paper, we propose a method to empirically detect and correct the misspecifications in the Q-matrix for graded response data based on the sequential generalized deterministic inputs, noisy ‘and’ gate (G-DINA) model. The proposed Q-matrix validation procedure is implemented in a stepwise manner based on the Wald test and an effect size measure. The feasibility of the proposed method is examined using simulation studies. Also, a set of data from the Trends in International Mathematics and Science Study (TIMSS) 2011 mathematics assessment is analysed for illustration.     

Psychometric properties of the revised Norwegian dispositional resilience (hardiness) scale

Hystad, S. W., Eid, J., Johnsen, B. H., Laberg, J. C. & Bartone, P. T. (2010) Psychometric properties of the revised Norwegian dispositional resilience (hardiness) scale. Scandinavian Journal of Psychology, 51, 237–245. In the 30 years that have elapsed since it was first introduced, the concept of hardiness has continued to attract the attention and interest of researchers from all over the world. The purpose of this study was to examine the reliability and factor structure of a revised Norwegian hardiness scale (Dispositional Resilience Scale 15; DRS‐15). Using exploratory and confirmatory factor analytic strategies with a large sample of working adults (N = 7,280), support was found for a hierarchical structure comprising a general hardiness dimension and three sub‐dimensions (commitment, control, and challenge). Overall, the results support the reliability and validity of the revised DRS‐15 and underscore the importance of examining the psychometric properties and cultural appropriateness of translated scales.     

Development and initial validation of the job loss grief scale

Background: Research on complicated grief (CG) symptoms following job loss is surprisingly rare. Involuntary job loss can turn someone’s world upside down and can result in loss of identity, social contacts, and self-worth. In this study, we drew on the literature on major life events in conceptualizing involuntary job loss as a significant and potentially devastating life event.

Objectives: The aim of this study was to develop and evaluate an instrument that measures job loss-related CG symptoms, the Job Loss Grief Scale (JLGS). The purpose of the JLGS is to foster systematic research on CG symptoms following job loss.

Design: A cross-sectional study

Methods: We recruited Dutch workers who had lost their job, 130 men and 158 women with an average age of 49.6 years. To examine the psychometric properties of the JLGS and its associations with other concepts we conducted correlational and confirmatory factor analyses.

Results: CFA revealed that the JLGS was a one-dimensional instrument, and that CG symptoms were distinguishable from depression and anxiety symptoms.

Conclusion: The JLGS is a reliable and valid instrument to measure job loss-related CG symptoms. The availability of the JLGS could stimulate systematic research on the antecedents and consequences of involuntary job loss.     

A novel in vivo measure of cyberaggression

Cyberaggression (CA), or the use of information communication technologies to inflict harm on others, is an emerging public health crisis. Unfortunately, our current ability to assess CA in a research context remains limited, curtailing efforts to address this important issue. We sought to fill this gap in the literature by developing an adapted “chat” version of the Taylor aggression paradigm (TAP) that would more closely resemble a social gaming format (hereafter referred to as the TAP-Chat). In the TAP-Chat, participants have a chat function available to communicate with their (fictitious) co-player. Following loss trials in a competitive reaction time task, they receive a “mean chat” from their co-player. Participant messages to their (fictitious) co-player are then coded for aggressive content by a team of trained research assistants, and via automated linguistic analysis software (Linguistic Inquiry and Word Count). The current study evaluated the predictive utility of the TAP-Chat task in independent discovery and replication samples (N = 843 and N = 350, respectively). Participants’ publicly available tweets served as an important external criterion variable, along with a handful of self-report questionnaires assessing CA and related constructs. Analyses suggest that, although it can be completed in ∼13 min, the TAP-Chat predicts CA on Twitter and, to a lesser extent, as reported on questionnaires. Although there are still several issues to address, it is our hope that the research community will benefit from this straightforward behavioral assessment of CA.     

Eliciting emotion ratings for a set of film clips: A preliminary archive for research in emotion

ABSTRACT

Film clips are commonly used to elicit subjectively experienced emotional states for many research purposes, but film clips currently available in databases are out of date, include a limited set of emotions, and/or pertain to only one conceptualization of emotion. This work reports validation data from two studies aimed to elicit basic and complex emotions (amusement, anger, anxiety, compassion, contentment, disgust, fear, happiness/joy, irritation, neutrality, pride, relief, sadness, surprise), equally distributed according to valence (positive, negative) and intensity (high, low). Participants rated film clips according to the degree of experienced emotion, and for valence and arousal. Our findings initiate an iterative archive of film clips shown here to discretely elicit 11 different emotions. Although further validation of these film clips is needed, ratings provided here should assist researchers in selecting potential film clips to meet the aims of their work.     

Measuring associations between social anxiety and use of different types of social media using the Swedish Social Anxiety Scale for Social Media Users: A psychometric evaluation and cross-sectional study

Research on the association between social anxiety and social media usage remains inconclusive: despite the preference for computer-mediated communication there is currently no clear empirical support for social anxiety being associated with longer duration of social media use. Self-report measures for social anxiety that are adapted for the context of social media could facilitate further research. The current study aimed to develop a Swedish version of the recently developed Social Anxiety Scale for Social Media Users (SAS-SMU), evaluate its psychometric properties, and explore associations between different uses of social media and social anxiety. Three factors were retained for SAS-SMU with excellent internal consistency. SAS-SMU evidenced convergent validity with measures of social anxiety, negative convergent validity with satisfaction with life, and divergent validity with measures of obsessive-compulsive disorder, depression and generalized anxiety disorder. Results indicated that higher levels of social anxiety were associated with passive and active use as well as longer duration of social media use in general, which is at odds with a previous study where passive use remained the only significant predictor for social anxiety.     

Do overall dimension ratings from assessment centres show external construct-related validity?

ABSTRACT

There have been repeated calls for an external construct validation approach to advance our understanding of the construct-related validity of assessment centre dimension ratings beyond existing internal construct-related validity findings. Following an external construct validation approach, we examined whether linking assessment centre overall dimension ratings to ratings of the same dimensions that stem from sources external to the assessment centre provides evidence for construct-related validity of assessment centre ratings. We used data from one laboratory assessment centre sample and two field samples. External ratings of the same dimensions stemmed from assessees, assessees’ supervisors, and customers. Results converged across all three samples and showed that different dimension-same source correlations within the assessment centres were larger than same dimension-different source correlations. Moreover, confirmatory factor analyses revealed source factors but no dimension factors in the latent factor structure of overall dimension ratings from the assessment centre and from external sources. Hence, consistent results across the three samples provide no support that assessment centre overall dimension ratings and ratings of the same dimensions from other sources can be attributed to dimension factors. This questions arguments that assessment centre overall dimension ratings should have construct-related validity.