AN EXAMINATION OF THE EFFECTS OF USING BEHAVIOR CHECKLISTS ON THE CONSTRUCT VALIDITY OF ASSESSMENT CENTER DIMENSIONS

Although research has established the criterion-related validity of assessment centers for selection purposes, the construct validity of dimension ratings has not been demonstrated. A quasi-experimental design was used to investigate the influence of retranslated behavior checklists on the construct validity of dimension ratings for two assessment center exercises. Assessor use of behavior checklists increased the average convergent (i.e., same dimension across exercise) validity from .24 to .43 while decreasing the average discriminant (i.e., different dimension within exercise) validity (.47 to .41). Behavior checklist sums were moderately correlated with corresponding dimension ratings and demonstrated a comparable level of construct validity. It is suggested that using behavior checklists may improve dimension construct validity by reducing the cognitive demands placed on raters.     

A dilemma for internal realism

Conclusion Given my dilemma, then, we should think twice before following Putnam away from metaphysical realism. Internal realism can offer only a false sense of philosophical security. And now we may doubt that it can offer even that.     

EEG performance spectral mapping during drug therapy in a patient with schizophrenic psychosis

In a paranoid-hallucinatoric patient with chronic course and acute exacerbation a EEG mapping study during neuroleptic treatment was carried out. The typical changes of the alpha power spectra in schizophrenic psychoses as well the normalization after neuroleptic treatment could be found in the alpha map. The changes were in correlation to the clinical state.     

Psychoses in epilepsy

The heterogenous psychoses in epilepsies, caused by well known conditions, are not rare but associated with regularly a few of seizure-types not with the nature and development of attacks. Polar transitional ranks and converging courses of schizophrenic (accentuated) syndromes in epilepsies and idiopathic schizophrenias are rather frequent. Also (sub-)acute schizophrenic psychoses are corresponding to the complete palette of first and second rank symptoms (K. Schneider) of idiopathic schizophrenias. After manifestations of epilepsy these syndromes can appear at any time. It is given a profile of risks. Progressive avoidance of a. phenylaceturea, b. mixtures of antiepileptics did not put an end to psychotic syndromes: Long-term therapies with 1. Polytherapy, 2. Primidone and Phenytoin (dosedependant) as well as 3. Ethosuximide (-monotherapy) cause a disorder of feed back mechanisms, especially a disturbed regulation of vigilance and sleeping-waking-cycle and their psychological correlates. Carbamazepine and Sodium Valproate are, plasma-level-controlled of preventive antipsychotic effect. Selected neuroleptics of rather slight epileptogenic potency are of going down importance. Benzodiazepines are required mostly in prepsychotic syndromes, Lithium compounds in selected cases. There is no more alternative seizures or psychosis.     

Information search following damage to the frontal lobes

Five adult patients with focal brain damage in different areas of the frontal lobes and five matched controls were administered a 20 Questions test. Other tests of planning, problem-solving, and decision-making were also administered to frontal lobe patients. The results indicate that damage to the frontal lobe impairs the ability to impose and utilize hierarchical organization in mental representation.     

Psychotherapists' religious beliefs and their perception of the psychotherapy process

In a comparison of religious and nonreligious psychotherapists' perceptions of a portion of a psychotherapeutic session no significant differences were found on the Vanderbilt Psychotherapy Process Scale for four groups of therapists designated according to their scores on three religious questionnaires. While the results suggest attitudes toward religion are not reflected in therapists' practice of psychotherapy, limitations of the study indicate the need for more rigorous testing of this possibility.     

The many null distributions of person fit indices

This paper deals with the situation of an investigator who has collected the scores ofn persons to a set ofk dichotomous items, and wants to investigate whether the answers of all respondents are compatible with the one parameter logistic test model of Rasch. Contrary to the standard analysis of the Rasch model, where all persons are kept in the analysis and badly fittingitems may be removed, this paper studies the alternative model in which a small minority ofpersons has an answer strategy not described by the Rasch model. Such persons are called anomalous or aberrant. From the response vectors consisting ofk symbols each equal to 0 or 1, it is desired to classify each respondent as either anomalous or as conforming to the model. As this model is probabilistic, such a classification will possibly involve false positives and false negatives. Both for the Rasch model and for other item response models, the literature contains several proposals for a person fit index, which expresses for each individual the plausibility that his/her behavior follows the model. The present paper argues that such indices can only provide a satisfactory solution to the classification problem if their statistical distribution is known under the null hypothesis that all persons answer according to the model. This distribution, however, turns out to be rather different for different values of the person's latent trait value. This value will be called ability parameter, although our results are equally valid for Rasch scales measuring other attributes.As the true ability parameter is unknown, one can only use its estimate in order to obtain an estimated person fit value and an estimated null hypothesis distribution. The paper describes three specifications for the latter: assuming that the true ability equals its estimate, integrating across the ability distribution assumed for the population, and conditioning on the total score, which is in the Rasch model the sufficient statistic for the ability parameter.Classification rules for aberrance will be worked out for each of the three specifications. Depending on test length, item parameters and desired accuracy, they are based on the exact distribution, its Monte Carlo estimate and a new and promising approximation based on the moments of the person fit statistic. Results for the likelihood person fit statistic are given in detail, the methods could also be applied to other fit statistics. A comparison of the three specifications results in the recommendation to condition on the total score, as this avoids some problems of interpretation that affect the other two specifications.The authors express their gratitude to the reviewers and to many colleagues for comments on an earlier version.     

Integration and cue overlap in recognition failure of recallable words

A highly systematic relation between the conditional probability of recognition given recall and the overall recognition hit rate has been demonstrated in a wide variety of experiments. A function describing this relationship was developed by Tulving & Wiseman (1975). Exceptions to this function have, in retrospect, been interpreted in terms of (a) a low integration between cue and target items, or (b) a high cue overlap between the two tests involved: recognition and recall. The experiment reported was designed to evaluate the joint and the separate contributions from integration and cue overlap for obtaining conformity with and exception from the Tulving-Wiseman function. In line with the predictions made, the results showed that these two factors in combination can account for data that fit the function and for exceptions above the function. In relative terms, the contribution from integration was somewhat more pronounced than that from cue overlap.     

On the sampling theory roundations of item response theory models

Item response theory (IT) models are now in common use for the analysis of dichotomous item responses. This paper examines the sampling theory foundations for statistical inference in these models. The discussion includes: some history on the stochastic subject versus the random sampling interpretations of the probability in IRT models; the relationship between three versions of maximum likelihood estimation for IRT models; estimating versus estimating -predictors; IRT models and loglinear models; the identifiability of IRT models; and the role of robustness and Bayesian statistics from the sampling theory perspective.A presidential address can serve many different functions. This one is a report of investigations I started at least ten years ago to understand what IRT was all about. It is a decidedly one-sided view, but I hope it stimulates controversy and further research. I have profited from discussions of this material with many people including: Brian Junker, Charles Lewis, Nicholas Longford, Robert Mislevy, Ivo Molenaar, Donald Rock, Donald Rubin, Lynne Steinberg, Martha Stocking, William Stout, Dorothy Thayer, David Thissen, Wim van der Linden, Howard Wainer, and Marilyn Wingersky. Of course, none of them is responsible for any errors or misstatements in this paper. The research was supported in part by the Cognitive Science Program, Office of Naval Research under Contract No. Nooo14-87-K-0730 and by the Program Statistics Research Project of Educational Testing Service.