Taking a closer look: time sampling and measurement error

A person manufactured his in-seat behavior for 15, 30-min sessions so that there were three blocks of five sessions where the behavior occurred 20%, 50%, and 80% of the time. Whole interval, partial interval, and momentary time-sample measures of the behavior were taken and compared to the continuous measure of the behavior i.e., per cent of time the behavior occurred. For interval time sampling, the difference between the continuous and sample measures i.e., measurement error, was: (1) extensive, (2) unidirectional, (3) a function of the time per response, and (4) inconsistent across changes in the continuous measure. A procedural analysis demonstrated that the frequency and duration of behavior are confounded in interval time sampling. Momentary time sampling was found to be superior to interval time sampling in estimating the duration a behavior occurs.     

A COMPARISON OF INDIRECT MEASURES FOR LONG-DURATION BEHAVIORS1

A computer program was developed to simulate long-duration behavior. The program generated data by using two time-series models linked together by a covariance parameter. By varying the parameters of the program, the characteristics of the simulated behavior were altered, including the mean and variance of the durations of behavior, the mean and variance of the interresponse time, and the degree of relationship within and between durations of behavior and interresponse times. Four data-collection methods were applied to the simulated data: whole interval, partial interval, momentary time-sampling, and continuous measures. The accuracies of the first three recording measures were judged by comparing them to the continuous measure. The results indicated that only the momentary time-sampling approach yielded unbiased results. The degree of bias for the whole and partial interval measures was a function of the ratio of the interval recording length to the sum of the mean duration of behavior and the mean interresponse time. Unfortunately, it was concluded that researchers cannot estimate the magnitudes of these latter two parameters for most behaviors of interest and, consequently, are unable to choose an appropriate interval length. Therefore, it was recommended that the use of interval recording approaches be greatly restricted and that momentary time-sampling be substituted where possible.     

AN EMPIRICAL METHOD FOR DETERMINING AN APPROPRIATE INTERVAL LENGTH FOR RECORDING BEHAVIOR

The study sought to examine the effects of varying interval length on the representation of data obtained using modified frequency time sampling. A 7-category scale was used to observe reliably the behavior of eight psychiatric inpatients. Using electronic real time recording equipment, it was possible to computer analyze the obtained data at varying interval lengths, the shortest interval being 1.0 seconds. It was found that increasing the interval length had little effect on the percentage of total duration recorded within each behavioral category, suggesting that this is a relatively stable measure of behavior. Percentage total events for each category was less stable with increasing interval lengths. The number of recorded events within each category tended to decrease, while their average durations tended to increase, as a function of increasing the interval length. The data suggest that the current practice of determining interval length in an arbitrary fashion, or on the basis of convention, should be abandoned. Rather, such a decision should be empirically determined for each particular observation scale and subject group. One method by which this might be achieved is presented.     

ON THE INABILITY OF INTERVAL TIME SAMPLING TO REFLECT FREQUENCY OF OCCURRENCE DATA1

Interval time sampling yields the result, percentage of intervals scored. It is rudimentary to note that this measure per se does not constitute a response dimension. It is a useful behavioral measure, therefore, only to the extent that it accurately reflects the nature and degree of the fundamental dimensions from which it is drawn i.e., frequency and duration. The correspondence between scored intervals and response duration is fairly well understood (Journal of Applied Behavior Analysis, 1975, 8 , 463–469; 1977, 10 , 325–332). This study determined the correspondence between scored intervals and response frequency. Eleven, 30-minute experimental sessions that differed along the variables of frequency of occurrence and time per response (the average length of a response per session) were computer simulated. In the first group of four sessions, the frequencies were 45, 100, 150, and 300; in these sessions, all responses ranged from one to three seconds. In the second group of four sessions, the frequencies were 31, 61, 101, and 152; in these sessions, all responses ranged from three to nine seconds. In the last group of three sessions, the frequencies were 25, 34, and 50; here, all responses ranged from nine to 27 seconds. The response distribution within the above ranges was rectangular, with each whole second represented once. The responses were selected by a random number generator, and on each trial every number in the distribution had an equal probability of occurrence. These provisions produced a linear pattern of responding. The time per response in the three groups of sessions were 2, 6, and 18 seconds. For all sessions, event recordings were made and analyzed. The analysis consisted of using partial interval time sampling to determine the percentage of intervals scored; this total was subdivided into intervals containing (1) single responses, (2) multiple responses, (3) continuing responses, and (4) response initiations or terminations. The analysis was performed when the length of the observation interval was 5, 10, 20, 30, 60, and 120 seconds. An additional session drawn from a study that contained real-life data was subjected to this same analysis. The most significant results were derived by finding the ratio of scored intervals containing single responses to the total intervals scored. If every scored interval contained a single response, this ratio would equal 1.0; if no scored interval contained a single response, the ratio would be 0.0. It can be seen that this ratio is an objective expression of the validity of interval time sampling as a measure of response frequency. Of 66 data points (11 sessions × six observation lengths per session), only five were equal to or greater than 0.80. These five points were all found in just two sessions (f = 45, 100). A validity index of less than 0.50 was observed in 49 of the 66 points. Also, the validity index increased, peaked, and then decreased within sessions as the length of the observation interval was increased. The results from the real-life session were in close agreement with those obtained from the simulations. The importance of these and previous findings lies in the demonstration that changes in scored intervals need not represent true behavior change. The data indicate that there are many combinations of behavioral frequency and duration where interval time sampling cannot produce valid measurement results.     

Processes of Social Influence on Sampling Behavior: The Effect of Other's Positive Attitude and Receiver's Prior Experience

Studies about attitude formation have pointed out the importance of sampling behavior. When thinking about actual social interaction scenes, it may be better to consider them as attitude-updating processes between those who have already formed attitudes based on some experience. Sixty-five participants took part in the experiment, in which the other's positive attitude and the participant's own prior experience were manipulated. After participants had been presented five types of puzzles (experience: direct vs. indirect), which had been made to improve analytic ability, their attitudes towards the puzzles were measured (“pre attitude”). They were shown the other's positive attitude (inconsistent vs. consistent), and they were given 10 min of free time before “testing on their analytic ability.” The amount of time spent on each puzzle was recorded to index sampling behavior and the attitudes towards the puzzles were measured after the free time (“post attitude”). The results of analyses showed that the other's positive attitude increased sampling behavior when the participant's attitude was positive (first-ranked) and negative (fifth-ranked), and that prior experience and sampling contributed to attitude change for the first-ranked puzzle. Those who had had direct experience and those who had spent more time sampling this puzzle tended to maintain their previous positive attitude. The potential boundary conditions of social influence on sampling behavior are discussed.     

测验同质性系数及其区间估计

在决定将多维测验分数合并成测验总分时, 应当考虑测验同质性。如果同质性太低, 合成总分没有什么意义。同质性高低可以用同质性系数来衡量。用来计算同质性系数的模型是近年来受到关注的双因子模型(既有全局因子又有局部因子), 测验的同质性系数定义为测验分数方差中全局因子分数方差所占的比例。本文用Delta法推导出计算同质性系数的标准误公式, 进而计算其置信区间。提供了简单的计算同质性系数及其置信区间的程序。用一个例子说明如何估计同质性系数及其置信区间, 通过模拟比较了用Delta法和用Bootstrap法计算的置信区间, 发现两者差异很小。     

Estimation of the Coefficient of Variation with Minimum Risk: A Sequential Method for Minimizing Sampling Error and Study Cost

The coefficient of variation is an effect size measure with many potential uses in psychology and related disciplines. We propose a general theory for a sequential estimation of the population coefficient of variation that considers both the sampling error and the study cost, importantly without specific distributional assumptions. Fixed sample size planning methods, commonly used in psychology and related fields, cannot simultaneously minimize both the sampling error and the study cost. The sequential procedure we develop is the first sequential sampling procedure developed for estimating the coefficient of variation. We first present a method of planning a pilot sample size after the research goals are specified by the researcher. Then, after collecting a sample size as large as the estimated pilot sample size, a check is performed to assess whether the conditions necessary to stop the data collection have been satisfied. If not an additional observation is collected and the check is performed again. This process continues, sequentially, until a stopping rule involving a risk function is satisfied. Our method ensures that the sampling error and the study costs are considered simultaneously so that the cost is not higher than necessary for the tolerable sampling error. We also demonstrate a variety of properties of the distribution of the final sample size for five different distributions under a variety of conditions with a Monte Carlo simulation study. In addition, we provide freely available functions via the MBESS package in R to implement the methods discussed.     

Sample size planning for composite reliability coefficients: accuracy in parameter estimation via narrow confidence intervals

Composite measures play an important role in psychology and related disciplines. Composite measures almost always have error. Correspondingly, it is important to understand the reliability of the scores from any particular composite measure. However, the point estimates of the reliability of composite measures are fallible and thus all such point estimates should be accompanied by a confidence interval. When confidence intervals are wide, there is much uncertainty in the population value of the reliability coefficient. Given the importance of reporting confidence intervals for estimates of reliability, coupled with the undesirability of wide confidence intervals, we develop methods that allow researchers to plan sample size in order to obtain narrow confidence intervals for population reliability coefficients. We first discuss composite reliability coefficients and then provide a discussion on confidence interval formation for the corresponding population value. Using the accuracy in parameter estimation approach, we develop two methods to obtain accurate estimates of reliability by planning sample size. The first method provides a way to plan sample size so that the expected confidence interval width for the population reliability coefficient is sufficiently narrow. The second method ensures that the confidence interval width will be sufficiently narrow with some desired degree of assurance (e.g., 99% assurance that the 95% confidence interval for the population reliability coefficient will be less than W units wide). The effectiveness of our methods was verified with Monte Carlo simulation studies. We demonstrate how to easily implement the methods with easy-to-use and freely available software.     

Gray's Reinforcement Sensitivity Model and Child Psychopathology: Laboratory and Questionnaire Assessment of the BAS and BIS

The Behavioral Approach System (BAS) and Behavioral Inhibition System (BIS) are widely studied components of Gray's sensitivity to reinforcement model. There is growing interest in integrating the BAS and BIS into models of risk for psychopathology, however, few measures assess BAS and BIS functioning in children. We adapted a questionnaire measure and reaction time task from adult studies to assess the BAS and BIS in a sample of 9-12 year olds (N = 63; 42% female). A continuous performance task was also administered to assess physiological correlates of the BAS and BIS. Factor analysis supported 3 subdimensions of the BAS (Drive, Reward Responsivity, and Impulsivity/Fun seeking), and one dimension of the BIS (sensitivity to punishment). Results supported the utility of the questionnaire measures and reaction time task as assessments of BAS and BIS functioning. Moreover, these measures were associated with internalizing and externalizing problem behavior as expected. High levels of impulsivity/fun seeking, but not drive or reward responsivity, were associated with high levels of externalizing problems, whereas high levels of sensitivity to punishment were associated with high levels of internalizing behavior problems. The relation between physiological indices and caregiver reports of the BAS, BIS, and problem behavior were complex and not consistent with expectation.     

The role of social support as a moderator of the acute stress response: In Situ versus empirically-derived associations

Abstract

This investigation was designed to (1) evaluate the role of social support during an acute naturalistic stressor i.e., emergency dental treatment, (2) assess whether an in situ and an empirical measure of support exhibited consistent relationships with anxiety and (3) examine the relationship between the in situ and empirical measures of support. All participants completed a measure of social support prior to treatment and an anxiety measure prior to and following treatment. The effect of support on pre-treatment anxiety, post-treatment anxiety and change in anxiety during the treatment process was assessed. According to the in situ support measure, individuals high in support were accompanied to the clinic, and individuals low in support were unaccompanied. According to the empirical support measure, individuals high in support had actual support scores above the sample median. Individuals low in support had actual support scores below the sample median. Individuals high in in situ support were found to have significantly higher levels of pre-treatment anxiety. No other effects of support were observed for either support measure. Furthermore, while there was some evidence that the in situ and empirical measures of support were associated with consistent outcomes, in the main, they exhibited independence.     

Maximizing Scales Do Not Reliably Predict Maximizing Behavior in Decisions from Experience

In this paper, we explore the relationships between psychometric and behavioral measures of maximization in decisions from experience (DfE). In two experiments, we measured choice behavior in two experimental paradigms of DfE and self‐reported maximizing tendencies using three prominent scales of maximization. In the repeated consequentialist choice paradigm, participants made repeated choices between two unlabeled options and received consequential feedback on each trial. In the sampling paradigm, participants freely sampled from two options and received feedback on their sampling before making a single consequential choice. Individuals exhibited different degrees of maximizing behavior in both paradigms and across different payoff distributions, but none of the maximizing scales predicted this behavior. These results indicate that maximization scales address constructs that are different from the maximization behavior observed in DfE, and that these measures will need to be improved to reflect behavioral aspects of choice and search from experience. Copyright © 2017 John Wiley & Sons, Ltd.     

The development and psychometric evaluation of a new measure of dissociative activities

The psychometric properties of a new measure of dissociative behaviors, the Scale of Dissociative Activities (SODAS), were evaluated. Undergraduates (n = 533) completed the SODAS, 2 other established self-report measures of dissociative behaviors, and a measure of socially desirable responding. A randomly selected subset of participants (n = 100) also participated in an additional reliability and validity follow-up study. The SODAS was found to be internally consistent (a =.95) and temporally stable (r =.77) over an average 38-day interval. The validity of the SODAS was established by a moderately low negative correlation with social desirability, high correlations with other self-report measures of dissociation, and moderate to high correlations with samples of dissociative activities assessed in naturalistic environments with experience sampling methodology. Overall, the SODAS compared favorably to other existing self-report measures of dissociative behaviors. Directions for future research on the SODAS are discussed.     

The use of momentary time sampling to assess both frequency and duration of behavior

When the time-sampling interval size can be set so it is smaller than both the shortest behavior instance and the shortest interresponse time, momentary time sampling will yield an accurate count of the frequency of behavior occurrence. It will also yield an unbiased estimate of behavior duration.     

Interval recording for duration events: a re‐evaluation

In two experiments, events that were recorded using continuous duration recording (CDR) were rescored using 10‐s partial interval (PIR), 10‐s momentary time sampling (MTS) and 20‐s MTS. Results of Experiment 1 showed that data paths generated by each interval method produced conclusions about functional control that were similar to those based on CDR when using reversal designs; however, for multielement designs, 10‐s PIR was prone to showing differentiation between data paths that was not evident with CDR. Results of Experiment 2 showed that both 10‐s and 20‐s MTS yielded data paths on behavior–behavior relations (e.g., covarying responses) that were consistent with CDR whereas 10‐s PIR produced some behavior–behavior patterns that were not. In both experiments, 10‐s MTS generated data paths that were nearly identical to the respective CDR data paths. The implications of these findings for researchers and clinicians are briefly discussed. Copyright © 2007 John Wiley & Sons, Ltd.     

Hoping for more: The influence of outcome desirability on information seeking and predictions about relative quantities

People must often engage in sequential sampling in order to make predictions about the relative quantities of two options. We investigated how directional motives influence sampling selections and resulting predictions in such cases. We used a paradigm in which participants had limited time to sample items and make predictions about which side of the screen contained more of a critical item. Sampling selections were biased by monetary desirability manipulations, and participants exhibited a desirability bias for both dichotomous and continuous predictions.     

Choice with a fixed requirement for food, and the generality of the matching relation

Pigeons were trained on choice procedures in which responses on each of two keys were reinforced probabilistically, but only after a schedule requirement had been met. Under one arrangement, a fixed-interval choice procedure was used in which responses were not reinforced until the interval was over; then a response on one key would be reinforced, with the effective key changing irregularly from interval to interval. Under a second, fixed-ratio choice procedure, responses on either key counted towards completion of the ratio and then, once the ratio had been completed, a response on the probabilistically selected key would produce food. In one experiment, the schedule requirements were varied for both fixed-interval and fixed-ratio schedules. In the second experiment, relative reinforcement rate was varied. And in a third experiment, the duration of an intertrial interval separating choices was varied. The results for 11 pigeons across all three experiments indicate that there were often large deviations between relative response rates and relative reinforcement rates. Overall performance measures were characterized by a great deal of variability across conditions. More detailed measures of choice across the schedule requirement were also quite variable across conditions. In spite of this variability, performance was consistent across conditions in its efficiency of producing food. The absence of matching of behavior allocation to reinforcement rate indicates an important difference between the present procedures and other choice procedures; that difference raises questions about the specific conditions that lead to matching as an outcome.     

The assessment of adolescent self-esteem: A comparison of methods1

The present series of studies explores the relationship between two conceptions of self-esteem: the self as experienced by the individual and the self as presented to others. Traditional measures of self-esteem are employed, and two new methods of assessing self-esteem in adolescents are introduced: an observational measure of self-esteem behavior (presented self) and a repeated self-report measure (experienced self). The presented self (observed behavior and peer ratings) and the experienced self (Rosenberg, Lerner, and a new method, paging devices) measures were consistent within themselves in assessing self-esteem. The presented self and the experienced self were not, however, significantly related to each other. We propose a moderator variable (“defensiveness”) to explain these discrepancies, and suggest that the observation of adolescent behavior provides a more accurate assessment of self-esteem than self-report measures.     

True Overconfidence in Interval Estimates: Evidence Based on a New Measure of Miscalibration

Overconfidence is often regarded as one of the most prevalent judgment biases. Several studies show that overconfidence can lead to suboptimal decisions of investors, managers, or politicians. Recent research, however, questions whether overconfidence should be regarded as a bias and shows that standard “overconfidence” findings can easily be explained by different degrees of knowledge of agents plus a random error in predictions. We contribute to the current literature and ongoing research by extensively analyzing interval estimates for knowledge questions, for real financial time series, and for artificially generated charts. We thereby suggest a new method to measure overconfidence in interval estimates, which is based on the implied probability mass behind a stated prediction interval. We document overconfidence patterns, which are difficult to reconcile with rationality of agents and which cannot be explained by differences in knowledge as differences in knowledge do not exist in our task. Furthermore, we show that overconfidence measures are reliable in the sense that there exist stable individual differences in the degree of overconfidence in interval estimates, thereby testing an important assumption of behavioral economics and behavioral finance models: stable individual differences in the degree of overconfidence across people. We do this in a “field experiment,” for different levels of expertise of subjects (students on the one hand and professional traders and investment bankers on the other hand), over time, by using different miscalibration metrics, and for tasks that avoid common weaknesses such as a non‐representative selection of trick questions. Copyright © 2012 John Wiley & Sons, Ltd.     

Quantifying false positives in simulated events using partial interval recording and momentary time sampling with dual‐criteria methods

The dual‐criteria (DC) and conservative dual‐criteria (CDC) methods allow clinicians and researchers to quantify the occurrence of false‐positive outcomes within single‐case experimental designs. The purpose of the current study was to use these DC and CDC methods to measure the incidence of false positives with simulated data collected via discontinuous interval methods (i.e., momentary time sampling, partial‐interval recording) as a function of data series length. We generated event data to create 316,800 unique simulated data series for analysis. In Experiment 1, we evaluated how changes in relevant parameters (i.e., interval sizes, event durations, IRT‐to‐event‐run ratios) produced false positives with momentary time sampling procedures. We also assessed the degree that the CDC method produced fewer false positives than the DC method with simulated interval data. In Experiment 2, we used similar procedures to quantify the occurrence of false positives with partial‐interval recording data. We successfully replicated outcomes from previous research in the current study, though such results only highlight the generality of the procedures relating to false positive (and not false negative) outcomes. That is, these results indicate MTS and PIR may adequately control for false positives, but our conclusions are limited by a lack of data on power.     

A Working Memory Model of a Common Procedural Error

Systematic errors In performance are an important aspect of human behavior that have not received adequate explanation. One such systematic error is termed postcompletion error; a typical example is leaving one's card In the automatic teller after withdrawing cash. This type of error seems to occur when people have an extra step to perform in a procedure after the main goal has been satisfied. The fact that people frequently make this type of error, but do not make this error every time, may best be explained by considering the working memory load at the time the step is to be performed: The error is made when the load on working memory is high, but will not be made when the load is low. A model of performance In the task was constructed using Just and Carpenter's (1992) CAPS that predicted that high working memory load should be associated with postcompletion errors. Two experiments confirmed that such errors can be produced in a laboratory as well as a naturalistic setting, and that the conditions under which the CAPS model makes the error are consistent with the conditions under which the errors occur in the laboratory.