Forced-Choice Personality Measures and Academic Dishonesty: a Comparative Study

Extant research (e.g., Wilks et al. 2016; Williams et al. 2010) has shown personality to be a predictor of engagement in academic dishonesty. The current study seeks to determine whether the type of personality measure affects predictive efficacy by comparing single stimulus and forced-choice measures of personality using a sample of 278 undergraduate students in two U.S. universities. Students scoring high on conscientiousness reported as engaging in fewer academic cheating behaviors than those scoring low on conscientiousness regardless of whether conscientiousness was measured using the forced-choice or single stimulus scale format. In addition, the forced-choice and single stimulus measures each contributed significant unique variance to prediction of academic dishonesty. For agreeableness, scores on the single stimulus measure were negatively correlated with academic dishonesty whereas there was a positive relationship found for the forced-choice measure. Overall, the forced-choice format of the Occupational Personality Questionnaire 32r (OPQ32r) did not show higher validities than the single stimulus IPIP counterpart in predicting self-reported academic dishonesty. Implications for future research and management education are discussed.     

The Impact of Faking on Employment Tests: Does Forced Choice Offer a Solution?

We evaluated the effects of faking on mean scores and correlations with self-reported counterproductive behavior of integrity-related personality items administered in single-stimulus and forced-choice formats. In laboratory studies, we found that respondents instructed to respond as if applying for a job scored higher than when given standard or "straight-take" instructions. The size of the mean shift was nearly a full standard deviation for the single-stimulus integrity measure, but less than one third of a standard deviation for the same items presented in a forced-choice format. The correlation between the personality questionnaire administered in the single-stimulus condition and self-reported workplace delinquency was much lower in the job applicant condition than in the straight-take condition, whereas the same items administered in the forced-choice condition maintained their substantial correlations with workplace delinquency.     

On the use of the WISC-R supplementary subtests as alternates

If a regularly administered WISC-R subtest cannot be administered properly or is invalidated, the manual suggests that a supplementary subtest, either Digit Span or Mazes, can serve as a substitute, but perhaps prorating the sum of scaled scores on the remaining subtests would be better. Data from the WISC-R standardization sample were analyzed to investigate the psychometric properties of these two variants of the standard procedure. Proration consistently resulted in higher validities (i.e., correlations between scores on the variant Verbal, Performance, or Full Scale and scores on the corresponding original scale), whereas substitution almost as consistently resulted in higher reliabilities, but in both cases the differences were small. Examiners may decide for themselves whether or not to use the supplementary subtests as alternates.     

The Moderating Effect of Raters’ Opportunities to Observe Ratees' Job Performance on the Validity of an Assessment Centre

Previous research has suggested that the quality of judgmental validation criteria may improve depending on the increase of opportunity for the rater to observe ratees’ job performances. This study investigated whether the validity of an assessment centre increases with opportunity to observe. Subjects were engineers and applied scientists from the Research and Development (R&D) units of seven German high-technology companies. A total of 6 personality scales, 10 cognitive ability tests and 9 work samples was administered to 155 subjects. Performance appraisals were collected from their supervisors. Assessors were trained organizational psychologists who were not members of the participating organizations. A total score (‘assessment centre score’) was computed by averaging the 25 predictors after standardization of each predictor. The (uncorrected) validity of this assessment centre was r = 0.37. In addition, opportunity to observe (i.e. amount of time the performance rater was the ratee’s supervisor) emerged as a moderator: for example validities were r = 0.09 (opportunity to observe ≤ 2 years) and r = 0.50 (opportunity to observe> 2 years).     

VALIDITY OF SELF-ASSESSMENT: A FIELD EVALUATION

This study evaluated the validity of self-assessment in a natural setting. It also addressed the possibility of improving the utility of self-assessment by cautioning subjects that their responses could be compared with other existing data on them. Subjects were 357 males, aged 18–19 years, who were being screened for a prestigious military course. During the screening, they participated in small groups for nine days of tests, class exercises, and field simulations supervised by veteran commanders. On the ninth day, ratings were collected from commanders, peer group members, and the subjects themselves on dimensions related to eventual success in the training course. The experimental group was told their reports would be compared with those from other sources; the control group was not. Predictive and convergent validities were examined on three criteria: course success, commander ratings, and peer evaluations. Self-assessments from both experimental and control group were valid; however, those of the experimental group did not yield consistently higher validities. Findings are discussed in regard to their practical ramifications.     

Utilizing probabilities as decision weights in closed and open information boards: A comparison of children and adults

Decisions in preschoolers (6 years), elementary schoolers (9.7 years), and adults (21 years) were studied with an information board crossing three probabilistic cues (validities: .83, .67, .50) with two options. Experiment 1 (n = 215) applied a standard version of the information board (closed presentation format), in which information must be searched sequentially and kept in mind for the decision. Experiment 2 (n = 217) applied an open format (Glöckner & Betsch, 2008), in which all information was visible during decision making. Elementary schoolers but not preschoolers benefited from an open format — indicated by an increase in using probabilities as decision weights. In the open but not closed format, choices were biased by normatively irrelevant information (the lure). Variations in the prediction patterns of the cues influenced decisions in all age groups. Effects for presentation format, pattern, and lure jointly indicate that even children are capable of considering multiple information in their decisions.     

Self-instructions as discriminative cues for rule-breaking or rule-following

This experiment examined the relationship between self-instructions and subsequent instruction-following performance, as a function of the history of the consequences for correspondence or noncorrespondence. Subjects were two groups of elementary school children (ages 7 and 8 years). In training, the two groups were exposed to different histories of congruence or noncongruence between self-instructions and consequences for performance. In the Congruent group, subjects were exposed to a series of stimuli to which they were trained to give self-instructions to make, or not to make, a key-pressing response. Reinforcement was given for self-instructing, and responding in accord with the self-instruction. Subjects in the Noncongruent group were trained to give the same self-instructions for the same stimuli, but reinforcement was given for counter-compliance. Subsequently, in a temptation situation, the experimenter gave an instruction not to press for a test stimulus; and subjects were trained to give the inhibitory self-instruction for this stimulus. Subjects in the Noncongruent group showed a high rate of rule-breaking. The effects on behavior of self-delivered instructions depend on the past relations between these events and the consequences for instruction-following. Implications are discussed for moral behavior and cognitively oriented therapy.     

A PRELIMINARY COMPARISON OF REINFORCER ASSESSMENT METHODS FOR CHILDREN WITH ATTENTION DEFICIT HYPERACTIVITY DISORDER

We evaluated the relative treatment utility of a verbal forced-choice questionnaire, child nomination, and direct observation for identifying the most potent reinforcers for children with attention deficit hyperactivity disorder. Results demonstrated that all three methods were more likely to disagree than to agree, that a forced-choice format may enhance verbal reinforcer assessment, and that further development and evaluation of verbal reinforcer-assessment methods are needed.     

Psychometric evaluation of the Geriatric Depression Scale and the Zung Self-Rating Depression Scale using an elderly community sample

In this study the psychometric properties of the Geriatric Depression Scale (GDS) and the Zung Self-Rating Depression Scale (SDS) were evaluated and compared, using a relatively large elderly community sample. The GDS generally performed well, replicating earlier findings from a different population. Also, as hypothesized, the SDS, which has a multiple-choice format, had a higher non-completion rate than the GDS, which has a true-false format. Finally, no significant differences between the responses of young-old and old-old subjects were observed.     

Establishing discriminative stimulus control within generalized oddity performance,without reinforcement

In two oddity learning studies with children, subjects were reinforced for oddity (or nonoddity) responding on line-tilt or dot numerosity problems. Interposed form and color problems were not reinforced. No instructions to make oddity choices were given. In Experiment I it was found that reinforcement for oddity or nonoddity responding on tilt and numerosity problems produced the corresponding tendency toward oddity or nonoddity performance on these problems and also on the nonreinforced form and color problems. These results show a generalized oddity phenomenon similar to generalized imitation. In Experiment II a third type of nonreinforced problem was presented in this same format: compound stimuli permitting either a color or a form solution. It was found that immediate prior training with nonreinforced form problems, interpolated among the reinforced tilt and numerosity problems, led to form-oddity choices in the compound problems. Similarily, color pretraining produced color-oddity choices. These results show that selective discriminative stimulus control can be obtained in oddity learning, without reinforcement for choices on either of the two dimensions involved.     

Reduction of faking with the use of a forced-choice personality test: Cross-cultural comparisons between South Korea and the United States

Forced-choice format tests have been suggested as an alternative to Likert-scale measures for personnel selection due to robustness to faking and response styles. This study compared degrees of faking occurring in Likert-scale and forced-choice five-factor personality tests between South Korea and the United States. Also, it was examined whether the forced-choice format was effective at reducing faking in both countries. Data were collected from 396 incumbents participating in both honest and applicant conditions (N_SK = 179, N_US = 217). Cohen's d values for within-subjects designs (ds_within) for between the two conditions were utilized to measure magnitudes of faking occurring in each format and country. In both countries, the degrees of faking occurring in the Likert-scale were larger than those from the forced-choice format, and the magnitudes of faking across five personality traits were larger in South Korea by from 0.07 to 0.12 in ds_within. The forced-choice format appeared to successfully reduce faking for both countries as the average ds_within decreased by 0.06 in both countries. However, the patterns of faking occurring in the forced-choice format varied between the two countries. In South Korea, degrees of faking in Openness and Conscientiousness increased, whereas those in Extraversion and Agreeableness were substantially decreased. Potential factors leading to trait-specific faking under the forced-choice format were discussed in relation to cultural influence on the perception of personality traits and score estimation in Thurstonian item response theory (IRT) models. Finally, the adverse impact of using forced-choice formats on multicultural selection settings was elaborated.     

Field articulation and linguistic structures.

The relation of field articulation to sensitivity to linguistic structures was explored. Subjects (23 males and 23 females) were given the rod-and-frame test and then made true-false judgments of sentences varying in grammatical and semantic complexity. While increased complexity did increase the difficulty of true-false judgments of sentences, the latencies of these judgments were unrelated to the scores on the rod-and-frame test. These results were discussed in terms of the generality of previous studies reporting a relationship between field articulation and linguistic abilities.     

The forced-choice paradigm and the perception of facial expressions of emotion

The view that certain facial expressions of emotion are universally agreed on has been challenged by studies showing that the forced-choice paradigm may have artificially forced agreement. This article addressed this methodological criticism by offering participants the opportunity to select a none of these terms are correct option from a list of emotion labels in a modified forced-choice paradigm. The results show that agreement on the emotion label for particular facial expressions is still greater than chance, that artifactual agreement on incorrect emotion labels is obviated, that participants select the none option when asked to judge a novel expression, and that adding 4 more emotion labels does not change the pattern of agreement reported in universality studies. Although the original forced-choice format may have been prone to artifactual agreement, the modified forced-choice format appears to remedy that problem.     

Forced-Choice Personality Tests: A Measure of Personality and Cognitive Ability?

This study examined the effects of item format (single-stimulus vs. forced-choice) and response motivation (honest vs. applicant) on scores for personality scales measuring Conscientiousness and Openness to Experience. Consistent with the hypotheses, cognitive ability was related to forced-choice personality scores in the applicant condition but not in the honest condition. Cognitive ability was unrelated to single-stimulus personality scores in both the applicant and honest conditions. The results suggest that controlling for cognitive ability can reduce the incremental predictive validity of forced-choice personality scales in applicant settings. Findings are discussed in terms of the importance of considering how item format influences the construct and criterion-related validity of personality tests used to make selection decisions.     

Roe's classification and the multidimensional nature of occupational perception

The purpose of this study was to compare college students' perceptions of the similarity of occupations with the groupings suggested by Roe (1956). Subjects were 90 male and 99 female university freshmen. Four illustrative titles were selected from five of Roe's eight groups. Subjects were asked to judge the similarity of each of the possible pairs of titles. The results were analyzed using multidimensional scaling procedures. The data provide partial support for Roe's postulated structure, although the subject's perceptions were generally more complex than suggested by Roe. In particular, the hypothesized circumplex arrangement of groups received only limited support.     

Additional Evidence of Attitudinal Influences in Performance Appraisal

The attitudes of twenty-nine managers toward performance appraisal and toward their organization were used to predict differences in mean ratings and measures of discrimination among ratees and performance dimensions. Substantial multiple correlations were obtained for all three dependent variables (between 32 and 46% of the variance in each measure was explained), and several individual correlations were statistically significant. Raters who showed low levels of confidence in the appraisal system, high levels of discomfort, or high levels of instrumental commitment were more likely to provide ratings that were unusually high or that did not discriminate well among ratees and/or dimensions. Raters who showed higher levels of attitudinal commitment or who perceived more risks associated with distorting ratings tended to provide lower ratings and to discriminate among ratees and/or dimensions.     

The influence of cue unreliability on judgment in a multiple cue probability learning task

The effect of cue reliability was tested in a two-cue multiple-cue probability learning (MCPL) task. Subjects were 42 undergraduates. The cue validities were.78 and.37. Unreliability in a cue was defined as variability in multiple observations of that cue on a given trial. That variability was provided by adding random errors to the true value of the cue. One group was given consistent cues (i.e., the more valid cue was the more reliable cue), one group was given inconsistent cues, and the control group was given true scores. Cue reliability did not affect subjects' consistency or achievement, but did interact with the presence of outcome feedback. The subjective weights showed that subjects thought they weighted the more reliable cue more heavily regardless of the cue validities or their actual cue weightings. There were wide individual differences in performance, illustrating the need for an idiographic-statistical approach in studying decision making.     

APPLICATION OF CONTENT VALIDITY METHODS TO THE DEVELOPMENT OF A JOB-RELATED PERFORMANCE RATING CRITERION

This study demonstrated the use of quantitative content validity procedures in the development of a job-related behavioral rating scale criterion for entry-level psychiatric aides. Work behavior items were developed by staff from 6 state psychiatric hospitals, placed in a content validity questionnaire using the Lawshe format, and given to a representative sample of 38 aides and supervisors. Seventy-eight of 83 items were found to be significantly job-relevant using the computation procedures of both Lawshe and Aiken. After the significant items were grouped into 4 categories with high interjudge agreement and placed in a rating scale format, ratings were obtained on 72 psychiatric aides from 4 hospitals. Items in the 4 categories were found to be internally consistent using coefficient alpha. Significant but low concurrent validities were established for 2 verbal ability selection tests using the rating criterion. The validities found were interpreted to be especially significant when the factors of low selection ratio, restriction in range, and limited rater training were considered.     

Putting Raters in Ratees’ Shoes: Perspective Taking and Assessment of Creative Products

This study reported 2 experiments that studied the effect of perspective taking on assessment of creative products by using human raters. Forty responses of 2 alternative uses tasks (AUTs) and 15 alien stories generated by 6th-grade students were used as assessment materials. Undergraduate students as the novice raters assessed the products under 3 experimental conditions: assessing without any information of the ratees, assessing only with age information of the ratees, and assessing with age information and taking the perspective of the ratees. Results of Experiment 1 showed significant differences in creativity ratings between group 1 and 2. But no significant difference was found between group 2 and 3. In Experiment 2, raters in group 1 used objective perception and raters in group 3 were asked to take the perspective for more time. Raters in group 3 assigned higher ratings than the other 2 groups but no difference was found between group 1 and 2. Overall, the results showed the effect of perspective taking on assessment of creative products. Possible mechanisms of the effect and implications for creativity assessment and creativity teaching practice were discussed.     

Self-evaluation,self-reinforcement,and locus of control

Two verbal recognition memory tests were administered to 24 internals and 24 externals. Subjects were required to self-evaluate (SE) their responses on the first test and both SE and self-reinforce (SR) on the second. There were no internal-external differences in performance accuracy on either task, but internals gave themselves consistently higher self-evaluations, administered more positive SR and less negative SR. There were no differences in the criteria used by the two groups for SR. SR differences appeared to be a function of the differential SE. It was suggested that external locus of control could be viewed as a self-regulation deficit. Externals are hypothesized to be unable to evaluate their own behavior adequately in the absence of external input and therefore do not make effective use of self-reinforcement. Discussion also made reference to the process of self-regulation.