Retention and Recall: Incidental Learning of Visual and Auditory Material

This study examined the effect on observer agreement of switching from a system of overt reliability assessment to two successive systems of covert reliability measurement. A primary purpose was to see whether agreement obtained with the use of covert data checks would improve over time if observers were provided with accurate feedback regarding their level of agreement. Seventeen undergraduate psychology students served as Ss. They observed instructional interactions between preschool children and their teachers. During the two week “overt” check period (Phase I), observers were aware of when their observations were being “checked” by a previously designated reliability assessor. In the subsequent covert phases (weeks 3–7) this information was not available to them during the observational sessions. When covert monitoring was implemented, agreement rates initially dropped significantly below the “overt” measurement phase. Gradually agreement rates improved until, in weeks 6 and 7, they were not significantly different from the initial overt measures. Es should be aware that an overt check on observer agreement may not reflect the true reliability of an observational system. However, when observers are given accurate feedback on their level of agreement, they are able significantly to improve their vigilance and consistency in the use of the observational system.