Sequential sampling designs for the two-parameter item response theory model

In optimal design research, designs are optimized with respect to some statistical criterion under a certain model for the data. The ideas from optimal design research have spread into various fields of research, and recently have been adopted in test theory and applied to item response theory (IRT) models. In this paper a generalized variance criterion is used for sequential sampling in the two-parameter IRT model. Some general principles are offered to enable a researcher to select the best sampling design for the efficient estimation of item parameters.     

Random error in holistic evaluations and additive decompositions of multiattribute utility — an empirical comparison

This paper details the results of an empirical investigation of the random errors associated with decomposition estimates of multiattribute utility. In a riskless setting, two groups of subjects were asked to evaluate multiattribute alternatives both holistically and with the use of an additive decomposition. For one group, the alternatives were described in terms of three attributes, and for the other in terms of five. Estimates of random error associated with the various elicitations (holistic, single-attribute utility, scaling constants, or weights) were obtained using a test-retest format. It was found for both groups that the additive decomposition had significantly smaller levels of random error than the holistic evaluation. However, the number of attributes did not seem to make a significant difference to the amount of random error associated with the decomposition estimates. The levels of error found in the various elicitations were consistent with theoretical bounds that have recently been proposed in the literature. These results show that the structure imposed on the problem through decomposition results in measurable improvement in quality of the multiattribute utility judgements, and contribute to a greater understanding of the decomposition method in decision analysis.     

Choice As A Function Of Reinforcement Ratios In Delayed Matching-to-sample

Pigeons were studied in two experiments using a delayed matching-to-sample task. In Experiment 1, 4 subjects were exposed to a task in which the proportion of reinforcement associated with matching and nonmatching, and the overall proportion of reinforcement associated with selecting each choice, regardless of the sample stimulus, were varied. Choice was sensitive to both proportions. A least squares regression analysis showed that Wixted's (1989) proportions of reinforcement model closely fit the data from Experiment 1; however, the model failed to make accurate qualitative predictions for some test conditions. In Experiment 2, 4 subjects were exposed to a delayed matching-to-sample task in which the retention intervals and the reduction in delay to reinforcement signaled by the onset of the sample stimulus were independently varied. When the retention interval was short and when the delay-reduction value of the sample stimulus was high, the sample exerted greater control over choice; the control by the overall proportion of reinforcements for selecting each choice stimulus was correspondingly low. Conversely, when the retention interval was long and the delay-reduction value of the sample stimulus was low, the sample exerted relatively less control over choice; control by the overall proportion of reinforcements obtained for selecting each choice stimulus was correspondingly high. A signal detection analysis found that sensitivity to reinforcement varied directly with retention interval. Data were also consistent with misallocation models. No evidence was found to suggest that pigeons ignore the rate at which selecting individual choice stimuli is reinforced, as has been reported in studies with human subjects.     

Direct assessment of quality of care in a geriatric nursing home

Direct observation was used to examine multiple aspects of care provided in a proprietary nursing home. The samples were taken at random intervals, 7 days per week for 5 weeks, across several categories of environmental and resident conditions as well as staff and resident activity. Results showed a high degee of compliance with predefined standards (based on current federal regulations) for environmental and resident conditions. Overall distributions of resident and staff activity showed results similar to those found in previous studies, with residents spending most of their time engaged in nonsocial activity and staff spending the majority of their time engaged in nonresident work. When data were analyzed across areas of the facility, times of day, and weekdays versus weekends, some differences were noted. Weekend versus weekday comparisons showed higher resident:staff ratios on weekends and more resident inactivity, but no significant differences in environmental or resident conditions. In addition, more frequent resident care, resident interaction, and resident activity were observed in Medicare units than in non-Medicare units. Results are discussed in terms of federal requirements for monitoring the quality of care in nursing homes and the potential use of time sampling expressly for this purpose.     

Action and emotion in everyday life

This study of naturally occurring behaviour employed a "beeper technique" to investigate the actions carried out by 152 subjects in eight different groups, viz. adult students, employed students who were also parents, unemployed, art students, doctoral students, alcoholics, retired people and a control group. Actions were sampled for seven subsequent days, five times per day between 8 a.m. and 10 p.m. at randomly selected occasions. Each action was rated on a number of variables, shortly after it was sampled (median delay approximately 10 min). Some of the findings were: Time allocation corresponded well with national estimates. Being under situational control was highly aversive. Instrumental and consummatory orientations correlated positively. The background variables were, on the whole, only rather weakly related to action ratings, but it was found that women and retired persons tended to report a higher level of well-being.     

Some empirical justification for a modest proposal regarding data acquisition via intermittent direct observation

Computer simulations of behavior and event records of a child's visual fixation on an adult person were generated and sampled using a modified time sampling procedure. The modified approach required making two independent scoring judgments per observation interval: if the behavior was initiated within the interval, the interval was scored; if the behavior was ongoing precisely at the end of the interval, the interval was again scored. The results showed that the within-interval and end-of-interval scores yielded, respectively, estimates of behavioral frequency and duration. The accuracy of the estimates was a function of the number of observations. Several field studies demonstrated high agreement among observers who employed the measurement system.     

Extending generalized symmetric means to arbitrary matrix sampling designs

Generalized symmetric means are redefined in a way which allows them to be calculated for any matrix sampling design. It is proved that these sample gsm's are unbiased estimates of the analogous population gsm's. Illustrative examples are given.This paper is based on Applied Technology Center for Education Technical Memorandum 75-03-01. Special appreciation is expressed to Ken Sirotnik and Tom Knapp for their suggestions on clarifing this presentation.     

Analytic smoothing for equipercentile equating under the common item nonequivalent populations design

A cubic spline method for smoothing equipercentile equating relationships under the common item nonequivalent populations design is described. Statistical techniques based on bootstrap estimation are presented that are designed to aid in choosing an equating method/degree of smoothing. These include: (a) asymptotic significance tests that compare no equating and linear equating to equipercentile equating; (b) a scheme for estimating total equating error and for dividing total estimated error into systematic and random components. The smoothing technique and statistical procedures are explored and illustrated using data from forms of a professional certification test.     

Retrospective questions and correlations

In survey research it is not uncommon to ask questions of the following type: “How many times did you undertake actiona in reference periodT of length τ?.” The relationship is established between τ and the correlation of the number of reported actions with some background variable. To this end it is assumed that the process of actions satisfies a renewal model with individual heterogeneity. Also a model has to be formulated for possible recall effects. Applications are given in the field of medical consumption.     

Measures of observer agreement when binomial data are collected in free operant situations

This paper examines a model and defines reasonable assumptions underlying different measures of observer agreement for categorical data collected in free operant situations. It is assumed that two or more observers classify operant behaviors of subjects into occurrences and nonoccurrences by recognition by validated response classes (categories) such that the rates of false positives and observer biases are acceptably low. Thus errors are mostly omissions, i.e., failing to observe events that occur. Four alternative cases are derived, together with formulas for calculating significance tests, variances, and standard errors, three of which do not depend on knowledge of the proportion of time points at which the event does not occur.We wish to acknowledge NICHD Grant HD-10570, The Neuropharmacology of Developmental Disorders, George Breese, Ph.D., and C. T. Gualtieri, M.D., Principal Investigators; NIEHS Grant ES-01104; USPHS Grant HD-03110; and MCH Project 916 to the Division for Disorders of Development and Learning.