Units of analysis and item statistics for environmental assessment scales

Choice of units of analysis is a critical problem in evaluating environmental assessment items. The “Ecological Fallacy” involves interpreting results based on ecological entities, such as environmental settings, as applying to individuals. A less familiar error, the “Individual Differences Fallacy,” involves interpreting results based on individuals as applying to settings. Although this second error has been ubiquitous, little is known about the empirical consequences of using different units of analysis in item analysis. This study examined this issue when the units were (1) individuals, (2) observed settings, or (3) “artificial” or random settings. Neither item validities based on individuals nor item validities based on settings yielded unequivocal results. However, a measure developed specifically for environmental research, the Split-Sample Correlation, appeared useful for identifying environmental items. Researchers constructing environmental assessment scales should select items with both high Split-Sample Correlations and high setting-level item validities.