Problematic effects of aggregation in z ROC analysis and a hierarchical modeling solution

The confidence-rating recognition memory paradigm has been used extensively for testing theories of memory. Current methods for analyzing confidence-rating data require that data be aggregated over participants, items, or both. We show that this aggregation threatens claims of curvature in zROCs and distorts estimates of signal-distribution variance—the very findings that are used to make fine-grained distinctions between competing models. We develop a hierarchical signal detection model that accounts for variability from the sampling of participants and items in addition to variability from the underlying psychological processes. The model provides for accurate signal detection parameter estimates. With accurate estimates, the validity of benchmark findings in recognition memory may be assessed.