Why Do Situational Interviews Predict Performance? Is it Saying How You Would Behave or Knowing How You Should Behave?

Purpose

The present study examined two theoretical explanations for why situational interviews predict work-related performance, namely (a) that they are measures of interviewees’ behavioral intentions or (b) that they are measures of interviewees’ ability to correctly decipher situational demands.

Design/Methodology/Approach

We tested these explanations with 101 students, who participated in a 2-day selection simulation.

Findings

In line with the first explanation, there was considerable similarity between what participants said they would do and their actual behavior in corresponding work-related situations. However, the underlying postulated mechanism was not supported by the data. In line with the second explanation, participants’ ability to correctly decipher situational demands was related to performance in both the interview and work-related situations. Furthermore, the relationship between the interview and performance in the work-related situations was partially explained by this ability to decipher situational demands.

Implications

Assessing interviewees’ ability to identify criteria might be of additional value for making selection decisions, particularly for jobs where it is essential to assess situational demands.

Originality/Value

The present study made an effort to open the ‘black box’ of situational interview validity by examining two explanations for their validity. The results provided only moderate support for the first explanation. However, the second explanation was fully supported by these results.

     

Memory for memory

Participants read short passages and 1 day later they answered questions via telephone about the passages (text facts) and about the experimental session (event facts). They were telephoned again 6 weeks later and answered the same questions about text and event facts. They also answered new questions about whether they remembered the answers they had given in the initial telephone interview (recall for prior memory performance). Although participants accurately remembered the majority of past memory successes, they were poor at remembering past memory failures. After being provided with the correct answer and tested again, the participants' performance improved somewhat, especially for memory failures. This suggests that some errors in recalling past forgetting might have been due to correctly remembering the answer previously given, but failing to realize that it had been wrong. These findings have implications for a variety of situations in which people are queried about past memory performance.     

Evidence of Predictive and Incremental Validity of Personality Factors,Vocal Attractiveness and the Situational Interview

This study evaluates the impact of the situational interview, personality measures, vocal attractiveness, and interactions between personality and vocal attractiveness on job performance. In a concurrent validity study with 154 customer service employees, a structured, situational interview was conducted and three of the Big Five personality factors (agreeableness, conscientiousness, and extraversion) are measured with two different frames‐of‐reference: at‐work and typical. Among the results are the validity of the situational interview and at‐work personality measures is established; vocal attractiveness correlates with both interview ratings and job performance; and finally, agreeableness and conscientiousness are found to vary in a pattern related to levels of vocal attractiveness such that both personality factors predict performance more strongly for people with more attractive voices.     

The use of impression management tactics in structured interviews: a function of question type?

This study investigated impression management tactic use during structured interviews containing both experience-based and situational questions. Specifically, the authors examined whether applicants' use of impression management tactics depended on question type. Results from 119 structured interviews indicated that almost all of the applicants used some form of impression management. Significantly more assertive than defensive impression management tactics were used, and among assertive tactics, applicants tended to use self-promotion rather than ingratiation. However, different question types prompted the use of different impression management tactics. Ingratiation tactics were used significantly more when applicants answered situational questions, whereas self-promotion tactics were used significantly more when applicants answered experience-based questions. Furthermore, the use of self-promotion and ingratiation tactics was positively related to interviewer evaluations.     

Social intelligence and interview accuracy: Individual differences in the ability to construct interviews and rate accurately

This research examined differences in interviewers’ ability to identify effective interview questions and to accurately rate interviewees’ responses. Given the theoretical association between these interview activities and the construct of social intelligence (SI), a performance‐based measure of SI was developed utilizing situational judgment test methodology. The initial step was to examine evidence of the psychometric properties and construct validity of the new SI measure. The SI measure, a test of general mental ability (GMA), and a personality inventory were then used to examine aspects of rater performance. Participants chose a set of interview questions and viewed interviewee responses during a videotaped interview. Results showed that people higher in SI and GMA chose interview questions judged by experts to be superior and were more accurate in rating interviewee responses. The SI measure explained variance beyond GMA and outperformed a trait‐based SI measure. Implications for the selection and training of interviewers are discussed.     

Effects of situational demand in the role-play assessment of assertive behavior

Assertive and nonassertive subjects role played responses to interpersonal situations of known response difficulty in which various types of assertions were appropriate to evaluate the interactive effects of demand and difficulty on assertive performance. Subjects responded as they normally would (low demand) and as if they had just finished an extensive assertive training program (high demand). Role-play data, rated for overall assertion, revealed that (a) subjects were more assertive under high than low-demand; (b) assertive performance varied with situational difficulty; and (c) self-reports of assertive ability predicted behavior only when role-play situations were easy or demand low, and high demand had preceded low demand. It was concluded that assertive performance is strongly influenced by situational and cognitive variables, that self-reports predict behavior only in specific assessment circumstances, and that role plays tend to measure maximal, rather than typical, performance. These findings are discussed in terms of their implications concerning the validity of role-play assessment procedures.An earlier version of this paper was presented at the annual convention of the Association for Advancement of Behavior Therapy, Toronto, Canada, November 1981.     

Validity of Stimulus and Response Components in a Video Test of Social Competence

Various stimulus components (video, orally-presented questions) and response components (multiple-choice, written, orally-given replies) of situational judgement tests of occupational social competency were investigated as to their impact upon the validity for a behavior-oriented role playing criterion while keeping test content constant. The stimulus component video alone had no impact upon validity. The response components contributed to validity. Validity increased with improved fidelity of response components. Concerning stimulus-response-combinations, the validity of two video tests (r = 0.17 and r = 0.36) was not higher than the validity of similar oral questioning (r = 0.13 and r = 0.37) but was significantly lower than a situational interview (r = 0.59). Response fidelity proved to be a bottleneck regarding validity of video tests. As a result, it is recommended that, in order to maximize validity of video and multimedia efforts, the developers of video- and multimedia tests focus special attention on response fidelity.     

Cognitive Ability Tests and the Situational Interview: A Test of Incremental Validity

This study investigated the validity and incremental validity of a situational interview beyond that of a composite measure of cognitive ability. Forty-seven factory service technicians underwent an interview and took four cognitive ability tests. Supervisors rated the performance of these subjects in a concurrent validation study. The interview was found to be a valid predictor of a supervisor rating of performance (r = 0.32, p < 0.05 uncorrected), however, was unable to show incremental validity over ability tests (Incremental R²= 0.05, n.s.). Limitations of the present study and directions for future research are discussed.     

“The interviewer is a machine!” Investigating the effects of conventional and technology-mediated interview methods on interviewee reactions and behavior

Despite the growing number of organizations interested in the use of asynchronous video interviews (AVIs), little is known about its impact on interviewee reactions and behavior. We randomly assigned participants (N = 299) from two different countries (Switzerland and India) to a face-to-face interview, an avatar-based video interview (with an avatar as a virtual recruiter), or a text-based video interview (with written questions) and collected data on a set of self-rated and observer-rated criteria. Overall, we found that whereas participants reported more negative reactions towards the two asynchronous interviews, observer ratings revealed similar performance across the three interviews and lower stress levels in the two AVIs. These findings suggest that despite technology-mediated interview methods still not being well-accepted, interviewees are not at a disadvantage when these methods are used in terms of how well interviewees perform and how stressed they appear to external observers. Implications are discussed.     

“You See It Everywhere. It’s Just Natural.”: Contextualizing the Role of Peers,Family, and Neighborhood in Initial Substance Use

The current study utilizes in-depth interviews with inner city African-American and Latino adolescents to understand how they negotiate initial substance use. We applied the social norms approach to explore the role of peers, family, and neighborhood on adolescents’ initial substance use. Utilizing data from 36 interview participants, our analysis revealed that extended family members were pivotal in providing adolescents with their initial alcohol; however, female adolescents were more likely than male adolescents to acquire marijuana from their male peers, for free. Understanding situational contexts underlying initial substance use is imperative for future interventions with this population.     

Candidates' Ability to Identify Criteria in Nontransparent Selection Procedures: Evidence from an assessment center and a structured interview

In selection procedures like assessment centers (ACs) and structured interviews, candidates are often not informed about the targeted criteria. Previous studies have shown that candidates' ability to identify these criteria (ATIC) is related to their performance in the respective selection procedure. However, past research has studied ATIC in only one selection procedure at a time, even though it has been assumed that ATIC is consistent across situations, which is a prerequisite for ATIC to contribute to selection procedures' criterion‐related validity. In this study, 95 candidates participated in an AC and a structured interview. ATIC scores showed cross‐situational consistency across the two procedures and accounted for part of the relationship between performance in the selection procedures. Furthermore, ATIC scores in one procedure predicted performance in the other procedure even after controlling for cognitive ability. Implications and directions for future research are discussed.     

Assessment of Situational Demands in a Selection Interview: Reflective style or sensitivity?

Correctly understanding situational demands is necessary to handle social situations appropriately. Past selection research has shown that candidates who are better at identifying the targeted dimensions in an interview or an assessment center, in fact, perform better in these procedures. However, at least two different processes might be responsible for the obtained findings. First, candidates might differ in their ability to correctly interpret given cues, meaning that some candidates generate better (i.e., more accurate) ideas than others. Second, some candidates might generally reflect more upon potential demands and therefore generate more ideas concerning potentially targeted dimensions. The present study used signal detection theory to investigate to what degree these two processes are related to interview performance. The interview was administered during a selection simulation for university graduates (N = 147). Interviewees' assumptions concerning the targeted dimensions were assessed in a postinterview questionnaire in which they had to write down any hypotheses as to what a certain question was trying to assess. We found that generating better ideas was essential for candidates' interview performance and not the degree to which they generally generated ideas about targeted interview dimension.     

Identification of the targeted performance dimensions and self-promotion in interviews: investigations of uncharted waters

ABSTRACT

Previous research on interviewees’ ability to identify criteria (ATIC) found that interview performance is positively related to the degree to which interviewees identify the targeted performance criteria in a selection interview. Similarly, the use of self-promotion as an impression management tactic is also positively related to interview performance. The present study extended this research and used multilevel analyses to investigate whether intraindividual variation concerning identification of the performance criteria and the use of self-promotion is related to within-person performance fluctuations across interview questions. Furthermore, we also tested whether correct identification of the performance dimensions moderates the effects of self-promotion behaviour on interview performance. In a simulated selection interview, we found that ATIC and self-promotion were indeed positively related to interview performance on the between-person level (i.e., for the entire interview). However, no moderator effect was found. Furthermore, on the within-person level interviewees’ performance was better concerning questions for which they were better at identifying the performance criteria but worse concerning questions for which they used more self-promotion. Thus, identification of the targeted dimensions and self-promotion have diverging effects on the level of specific interview questions. Finally, we discuss implications for research and practice.     

中层管理人员结构化面试测评效度的现场研究

通过对某上市公司随机抽取的43位中层管理人员素质测评的现场研究,探讨结构化面试的信度效度问题。研究设计基于岗位分析与关键事件分析,采用3人小组面试的方法,同时实施情景面试与行为描述面试,综合测评被试岗位胜任能力。分析结果表明,评委要素评价内部一致性和评委间内部一致性都比较高,并与面试半年后上级评定的任务绩效和总体绩效显著相关,结构化面试具有较高的信度与预测效度。进一步比较情景面试和行为描述面试发现,这两种结构化面试有类似的信度,但是行为描述面试具有更高的效度。     

Evaluating the Face Validity of the ICECAP-O Capabilities Measure: A “Think Aloud” Study with Hip and Knee Arthroplasty Patients

The ICECAP-O index of capabilities measure for older people is intended to be used in the evaluation of health and social care interventions. Focusing on quality of life, rather than health or other influences on quality of life. This study evaluated the face validity of the self-administered ICECAP-O capabilities measure for older people by investigating how participants interpret and respond to questions using the cognitive interviewing technique. Twenty patients with osteoarthritis of the knee or hip participated in a cognitive interview whilst completing the ICECAP-O capabilities measure. Cognitive interviews were conducted using the concurrent ‘think aloud’ design. All interviews were audio recorded and transcribed verbatim and analysed (i) using a standardised classification scheme to identify response problems and (ii) qualitatively thematically analysed to explore the nature of the problems that the participants experienced when completing the measure. Problems were identified in 7 % of participants’ responses. The majority of problems identified were comprehension problems. Thematic analysis highlighted the extent to which participants’ distinguished between functioning and capability. Cognitive interviewing was a valuable technique for pre-testing the face validity, acceptability and content validity of the ICECAP-O capabilities measure. Participants had minimal difficulties completing the ICECAP-O capabilities measure. Those difficulties identified have prompted suggestions for improving the measure.