Development of Standard Anchors for Scoring Generic Past-Behaviour Questions in Structured Interviews

A structured interviewing system was developed which included an item bank of pre-written past-behaviour questions. Scoring anchors for each of 168 questions in the item bank were created from analysis of protocols from individual assessments. A concurrent validation study was conducted, focusing on the job of highway patrol sergeant. Job knowledge experts used a job analysis tool to select a subset of twelve questions from the item bank. Panel interviews were conducted with 32 highway patrol sergeants as subjects. Average inter-rater reliability was 0.65 and interview validity was 0.42 for group consensus scores and 0.44 for panel average scores. Discussion focused on practical issues affecting the interviewing system.