Modeling error variance in job specification ratings: the influence of rater, job, and organization-level factors

The authors modeled sources of error variance in job specification ratings collected from 3 levels of raters across 5 organizations (N=381). Variance components models were used to estimate the variance in ratings attributable to true score (variance between knowledge, skills, abilities, and other characteristics [KSAOs]) and error (KSAO-by-rater and residual variance). Subsequent models partitioned error variance into components related to the organization, position level, and demographic characteristics of the raters. Analyses revealed that the differential ordering of KSAOs by raters was not a function of these characteristics but rather was due to unexplained rating differences among the raters. The implications of these results for job specification and validity transportability are discussed.     

The impact of enhanced incentives on vocational rehabilitation outcomes for dually diagnosed veterans

This study evaluated the efficacy of adding contingency management techniques to vocational rehabilitation (VR) to improve treatment outcome as measured by entry into competitive employment. Nineteen dually diagnosed veterans who entered VR in the Veterans' Administration's compensated work therapy (CWT) program were randomly assigned to CWT (n = 8) or to CWT with enhanced incentives (n = 11). Over the first 16 weeks of rehabilitation, those in the incentives condition could earn up to dollar 1,006 in cash for meeting two sets of clinical goals: (a) remaining abstinent from drugs and alcohol and (b) taking steps to obtain and maintain a competitive job. Results indicate that relative to participants in the CWT-only group, those in the incentives condition engaged in more job-search activities, were more likely to remain abstinent from drugs and alcohol, were more likely to obtain competitive employment, and earned an average of 68% more in wages. These results suggest that rehabilitation outcomes may be enhanced by restructuring traditional work-for-pay contingencies to include direct financial rewards for meeting clinical goals.     

Psychometric Modeling of response speed and accuracy with mixed and conditional regression

Human performance in cognitive testing and experimental psychology is expressed in terms of response speed and accuracy. Data analysis is often limited to either speed or accuracy, and/or to crude summary measures like mean response time (RT) or the percentage correct responses. This paper proposes the use of mixed regression for the psychometric modeling of response speed and accuracy in testing and experiments. Mixed logistic regression of response accuracy extends logistic item response theory modeling to multidimensional models with covariates and interactions. Mixed linear regression of response time extends mixed ANOVA to unbalanced designs with covariates and heterogeneity of variance. Related to mixed regression is conditional regression, which requires no normality assumption, but is limited to unidimensional models. Mixed and conditional methods are both applied to an experimental study of mental rotation. Univariate and bivariate analyzes show how within-subject correlation between response and RT can be distinguished from between-subject correlation, and how latent traits can be detected, given careful item design or content analysis. It is concluded that both response and RT must be recorded in cognitive testing, and that mixed regression is a versatile method for analyzing test data.I am grateful to Rogier Donders for putting his data at my disposal.     

Distinctive processing underlies skilled memory

Research on skilled memory has focused on organizational processes to the exclusion of item-specific processes, although theories of skilled memory do acknowledge the importance of both kinds of processes. Using the isolation methodology, we presented lists of American football team names to participants who had either a high or a low level of knowledge about American football. An isolation effect (greater recall of the target in the isolate list than in a homogeneous control list) was observed only with high-knowledge participants. When standard lists were used, an isolation effect was observed with both groups. These findings empirically validate the importance of both organizational and item-specific processing as the basis of distinctive processing underlying skilled memory performance.     

Human cognition and 1/f scaling

Ubiquitous 1/f scaling in human cognition and physiology suggests a mind-body interaction that contradicts commonly held assumptions. The intrinsic dynamics of psychological phenomena are interaction dominant (rather than component dominant), and the origin of purposive behavior lies with a general principle of self-organization (rather than a special neurocognitive mechanism). E.-J. Wagenmakers, S. Farrell, and R. Ratcliff (2005) raised concerns about the kinds of data and analyses that support generic 1/f scaling. This reply is a defense that furthermore questions the model that Wagenmakers and colleagues endorse and their strategy for addressing complexity.     

Individual pattern representations are context independent, but their collective representation is context dependent.

We studied context dependency of the representations underlying perceptual "goodness". Three experiments used a same-different task with classical Garner 5-dot patterns presented with an interstimulus interval (ISI) of 500 ms. Same patterns were allowed to be rotated or reflected versions of each other. Pattern goodness was varied according to rotation and reflection equivalence, using Garner's equivalence set size (ESS) measure. The ESS of both first and second patterns affected reaction time and accuracy. A model based on assumptions that Garner's equivalence sets constitute the generic representation of these patterns and that items within these sets are accessed serially was fitted to the data. Excellent fits were obtained, which were robust against frequency-induced bias at the level of the individual pattern, but sensitive to such bias at the level of the equivalence set. It was concluded that individual pattern representations are context independent, whereas their collective representations are context dependent. Simplicity and likelihood principles, therefore, seem to apply to different levels of a representation hierarchy.     

Compatibility effects based on stimulus and response numerosity

Four choice reaction time experiments documented a stimulus-response (S-R) compatibility effect involving the numbers of stimuli and responses. In Experiment 1, the stimulus consisted of one or two tones, and the correct response was either one or two taps of a response key. Responses were much faster with a compatible S-R assignment, in which the number of taps matched the number of tones, than with an incompatible assignment in which these numbers mismatched. Experiments 2 and 3 replicated this effect, using visual stimuli and bimodal stimuli, respectively, suggesting that auditory/manual rhythmic compatibility is not essential to it. Experiment 4 showed that an analogous but smaller effect is obtained when stimuli are the digits 1 and 2. This new numerosity-based compatibility effect has general theoretical implications regarding the mechanisms responsible for compatibility effects and practical implications for interface design.     

From interpretation to segmentation

In visual perception, part segmentation of an object is considered to be guided by image-based properties, such as occurrences of deep concavities in the outer contour. However, object-based properties can also provide information regarding segmentation. In this study, outer contours and interpretations of object configurations were manipulated to examine differences between image-based and object-based segmentation in a visual search task. We found that locating a two-dimensional object configuration with deep concavities in the outer contour depends on the type of outer contour of the surrounding distractors. In addition, locating a three-dimensional object configuration was harder when it was surrounded by object-based-disconnected distractors, as compared with object-based-connected distractors, regardless of image-based connections in these distractors. We conclude that segmentation based on the outer contours of a target facilitates its localization. However, when three-dimensional information is available, segmentation strongly depends on object-based properties, rather than on image-based properties.