Estimating the Reliability of Aggregated and Within-Person Centered Scores in Ecological Momentary Assessment

Consider two independent groups with K measures for each subject. For the j^th group and k^th measure, let μ^tjk be the population trimmed mean, j = 1, 2; k = 1, ..., K. This article compares several methods for testing H 0 : u1k = t2k such that the probability of at least one Type I error is, and simultaneous probability coverage is 1 - α when computing confidence intervals for μ^t1k - μ^t2k . The emphasis is on K = 4 and α = .05. For zero trimming the problem reduces to comparing means, but it is well known that when comparing means, arbitrarily small departures from normality can result in extremely low power relative to using say 20% trimming. Moreover, when skewed distributions are being compared, conventional methods for comparing means can be biased for reasons reviewed in the article. A consequence is that in some realistic situations, the probability of rejecting can be higher when the null hypothesis is true versus a situation where the means differ by a half standard deviation. Switching to robust measures of location is known to reduce this problem, and combining robust measures of location with some type of bootstrap method reduces the problem even more. Published articles suggest that for the problem at hand, the percentile t bootstrap, combined with a 20% trimmed mean, will perform relatively well, but there are known situations where it does not eliminate all problems. In this article we consider an extension of the percentile bootstrap approach that is found to give better results.