An ecological study of the reliability of clinical judgment in executive appraisal

This research was a study of the reliability of clinical judgment findings (multitrait) across three different information sources (psychometric tests, structured interview, and psychometric tests and interview). Subjects (N = 74) were middle and senior executives of Western Canadian technical companies; clinicians (N = 3) were trained and experienced industrial psychologists. The study investigated the similarity of clinical evaluation of personological characteristics (based on an 18-factor multitrait paradigm) across the three different information sources. Subjects were independently rated by a single clinician on 18 criterion factors in each of the three information source categories. Test information source categories required the administration of approximately 12 hrs of standardized psychological assessment questionnaires to each of the 74 subjects. Interview source category involved a 1.5-hr structured interview per subject. Combined condition pooled both test and interview conditions. Generalizability of the findings was maximized by the undertaking of the experiment in a natural situation thus increasing ecological validity. Statistical treatments used were designed to assess the similarity of a clinician's evaluation of a subject based on the different category of information available about that client. Convergence (intrarater reliability) indexes range from a high of .64 to a low of .05. Results indicate a varying degree of convergence of multitrait clinical ratings dependent on clinician and trait being rated. Results are discussed in terms of implications for practitioners involved in executive personnel selection.     

Individualized behavior therapy for alcoholics: a third year independent double-blind follow-up.

An independently conducted double-blind study providing third year follow-up data for 53 of 70 male alcoholics who served as subjects in an experiment evaluating “Individualized Behavior Therapy (IBT)” techniques is reported. In the original study, subjects were initially assigned to either a controlled drinking or non-drinking (abstinence) treatment goal, and were then randomly assigned to either an experimental group receiving IBT or a control group receiving conventional state hospital treatment oriented towards abstinence. Previously reported results showed that IBT subjects functioned significantly better than control subjects throughout the first year of follow-up. Second year follow-up results found that IBT subjects treated with a controlled drinking goal functioned significantly better than their respective control subjects on a variety of measures, including drinking behavior. Differences between IBT subjects treated with a non-drinking goal and their control subjects did not retain statistical significance during the second year of follow-up. The independently determined follow-up data showed that during the third post-treatment year, subjects in the experimental groups generally functioned better than their respective control subjects. Comparison of the controlled drinking experimental group with its control group showed the significantly better functioning of subjects in the experimental condition on a number of drinking and other life functioning measures. Comparison of the non-drinking groups indicated only one possible difference on the drinking related measures involving a trend for subjects in the experimental condition to abstain more than those in the non-drinker control group. On other life functioning measures, however, subjects in the non-drinking experimental condition showed consistent improvement over their respective control subjects. Both of the controlled drinking groups reported more controlled drinking days than the non-drinking groups throughout the third year follow-up period.