Sample size and the detection of means: A signal detection account

Using statistical theory as a basis, Kareev (e.g., 1995) claimed that people's ability to correctly infer the existence of a population correlation should be greater for small than for large samples. Simulations by R. B. Anderson, Doherty, Berg, and Friedrich (2005) identified conditions favoring small samples but could not determine whether such an advantage was due to sampling skew, variance, or central tendency displacement. In the present study, we investigated theoretical effects of sample size (n) on the detection of population means under circumstances in which sampling variance is unconfounded with skew or central tendency displacement. The results demonstrate an extremely limited, criterion-specific, small-sample advantage that was attributable to n-related sampling variance and that occurred only with highly conservative, suboptimal criterion placement.     

Sample size and correlational inference

In 4 studies, the authors examined the hypothesis that the structure of the informational environment makes small samples more informative than large ones for drawing inferences about population correlations. The specific purpose of the studies was to test predictions arising from the signal detection simulations of R. B. Anderson, M. E. Doherty, N. D. Berg, and J. C. Friedrich (2005). The results of a simulation study in the present article confirmed and extended previous theoretical claims (R. B. Anderson et al., 2005) that in a yes/no correlation detection task, small-sample advantages should occur but should be restricted to particular decision conditions. In 3 behavioral studies, participants viewed larger or smaller samples of data pairs and judged whether each sample had been drawn from a population characterized by a zero correlation or from one characterized by a greater-than-zero correlation. Consistent with traditional statistical theory, accuracy tended to be greater for larger than for smaller samples, though there was a small-sample advantage in 1 experimental condition. The results are discussed in relation to alternative theoretical and behavioral paradigms such as those of Y. Kareev (e.g., 2005) and K. Fiedler and Y. Kareev (2006).     

Small samples do not cause greater accuracy--but clear data may cause small samples: comment on Fiedler and Kareev (2006)

Fiedler and Kareev (2006) have claimed that taking a small sample of information (as opposed to a large one) can, in certain specific situations, lead to greater accuracy--beyond that gained by avoiding fatigue or overload. Specifically, they have argued that the propensity of small samples to provide more extreme evidence is sufficient to create an accuracy advantage in situations of high caution and uncertainty. However, a close examination of Fiedler and Kareev's experimental results does not reveal any strong reason to conclude that small samples can cause greater accuracy. We argue that the negative correlation between sample size and accuracy that they reported (found only for the second half of Experiment 1) is also consistent with mental fatigue and that their data in general are consistent with the causal structure opposite to the one they suggest: Rather than small samples causing clear data, early clear data may cause participants to stop sampling. More importantly, Experiment 2 provides unequivocal evidence that large samples result in greater accuracy; Fiedler and Kareev only found a small sample advantage here when they artificially reduced the data set. Finally, we examine the model that Fiedler and Kareev used; they surmised that decision makers operate with a fixed threshold independent of sample size. We discuss evidence for an alternative (better performing) model that incorporates a dynamic threshold that lowers with sample size. We conclude that there is no evidence currently to suggest that humans benefit from taking a small sample, other than as a tactic for avoiding fatigue, overload, and/or opportunity cost-that is, there is no accuracy advantage inherent to small samples.     

Does decision quality (always) increase with the size of information samples? Some vicissitudes in applying the law of large numbers

Adaptive decision making requires that contingencies between decision options and their relative assets be assessed accurately and quickly. The present research addresses the challenging notion that contingencies may be more visible from small than from large samples of observations. An algorithmic account for such a seemingly paradoxical effect is offered within a satisficing-choice framework. Accordingly, a choice is only made when the sample contingency describing the relative evaluation of the 2 options exceeds a critical threshold. Small samples, because of the high dispersion of their sampling distribution, facilitate above-threshold contingencies. Across a broad range of parameters, the resulting small-sample advantage in terms of hits is stronger than their disadvantage in false alarms. Computer simulations and experiments support the model predictions. The relative advantage of small samples is most apparent when information loss is low, when the threshold is high relative to the ecological contingency, and when the sampling process is self-truncated.     

Angoff's delta method revisited: improving DIF detection under small samples

Most methods for detecting differential item functioning (DIF) are suitable when the sample sizes are sufficiently large to validate the null statistical distributions. There is no guarantee, however, that they will still perform adequately when there are few respondents in the focal group or in both the reference and the focal group. Angoff's delta plot is a potentially useful alternative for small-sample DIF investigation, but it suffers from an improper DIF flagging criterion. The purpose of this paper is to improve this classification rule under mild statistical assumptions. This improvement yields a modified delta plot with an adjusted DIF flagging criterion for small samples. A simulation study was conducted to compare the modified delta plot with both the classical delta plot approach and the Mantel-Haenszel method. It is concluded that the modified delta plot is consistently less conservative and more powerful than the usual delta plot, and is also less conservative and more powerful than the Mantel-Haenszel method as long as at least one group of respondents is small.     

On the generality of optimal versus objective classifier feedback effects on decision criterion learning in perceptual categorization

Biased category payoff matrices engender separate reward- and accuracy-maximizing decision criteria Although instructed to maximize reward, observers use suboptimal decision criteria that place greater emphasis on accuracy than is optimal. In this study, objective classifier feedback (the objectively correct response) was compared with optimal classifier feedback (the optimal classifier's response) at two levels of category discriminability when zero or negative costs accompanied incorrect responses for two payoff matrix multiplication factors. Performance was superior for optimal classifier feedback relative to objective classifier feedback for both zero- and negative-cost conditions, especially when category discriminability was low, but the magnitude of the optimal classifier advantage was approximately equal for zero- and negative-cost conditions. The optimal classifier feedback performance advantage did not interact with the payoff matrix multiplication factor. Model-based analyses suggested that the weight placed on accuracy was reduced for optimal classifier feedback relative to objective classifier feedback and for high category discriminability relative to low category discriminability. In addition, the weight placed on accuracy declined with training when feedback was based on the optimal classifier and remained relatively stable when feedback was based on the objective classifier. These results suggest that feedback based on the optimal classifier leads to superior decision criterion learning across a wide range of experimental conditions.     

Matching-to-sample by an echolocating dolphin (Tursiops truncatus)

An adult male dolphin was trained to perform a three-alternative delayed matching-to-sample task while wearing eyecups to occlude its vision. Sample and comparison stimuli consisted of a small and a large PVC plastic tube, a water-filled stainless steel sphere, and a solid aluminum cone. Stimuli were presented under water and the dolphin was allowed to identify the stimuli through echolocation. The echolocation clicks emitted by the dolphin to each sample and each comparison stimulus were recorded and analyzed. Over 48 sessions of testing, choice accuracy averaged 94.5% correct. This high level of accuracy was apparently achieved by varying the number of echolocation clicks emitted to various stimuli. Performance appeared to reflect a preexperimental stereotyped search pattern that dictated the order in which comparison items were examined and a complex sequential-sampling decision process. A model for the dolphin's decision-making processes is described.     

Limits to participative leadership: Task,structure and skill as contingencies-a German-British comparison

The research investigated the relationship between managerial decision-making and a set of specified contingent situational factors: decision type, perceived skill requirements and objective skill inputs. The study was based on 663 German and British managers in two interlocking senior management levels of 37 large enterprises. The data were collected by means of Group Feedback Analysis as part of a large study of managerial decision-making in eight countries. The results show a significant (p = <.05) relationship between the choice of decision styles and the postulated contingency variables. Only 1 % of 615 senior managers consistently use a single decision style, more than two-thirds use four or five different styles. Very large variations occur as a function of different decision tasks, perceived skill requirements and objective skill availability. There are differences between the relatively well matched samples of German and British managers, but they are less significant than the broadly similar way in which both samples respond to the particular contingencies under investigation. The results are interpreted in the context of an open systems contingency framework. Their action implications are seen to suggest a link with sociotechnical theory on job design and wider issues of organisational and social policy.     

Perceptual independence of pitch and loudness in a signal detection experiment: A processing model for 2ATFC (21FC) experiments

The two-alternative temporal forced choice (2ATFC) experiment is used to measure the relative detectabilities of a frequency change, an amplitude increase, or both together. Subjects’ performance is best when both (redundant) cues are available. This improvement is fit better by a decision threshold model than by an information integration model. Since decision processes that can lead to the decision threshold prediction in (2ATFC) experiments are not obvious, an information processing model which does is proposed. The model makes additional predictions which fit the results of an experiment which sometimes deletes information from the first or the second observation interval. The model is not consistent with the signal detection theory interpretation of the 2ATFC experiment, and these results call into question that interpretation. It is concluded that pitch and loudness are perceptually independent for the 2ATFC experiment.     

Cerebral Asymmetry and Decision Strategies in Mental Rotation: A Psychophysical Analysis

The accuracy with which observers judged the parity of pairs of rotated images in the two visual fields was determined from receiver operating characteristic (ROC) analysis of confidence ratings. In one experiment, observers judged whether pairs of letters were of the same parity (that is, both normal or both backwards) or of different parities (one normal and one backwards). A small right visual field advantage in the observers' accuracy was found in this mental rotation task. In a second experiment, observers judged whether pairs of pictures were of the same parity or of different parities. Unlike the first experiment, no evidence of a consistent visual field advantage was found in this mental rotation task. The decision strategy adopted by the observers in making same-different judgements about rotated stimuli was examined. The symmetrical shape of the ROCs obtained was consistent with the adoption of the likelihood-ratio decision strategy, a result which supplemented previous evidence that this decision strategy is adopted when same-different judgements are made about multidimensional stimuli.     

Grammatical number elicits SNARC and MARC effects as a function of task demands

Despite the robustness of the spatial–numerical association of response codes (SNARC) and linguistic markedness of response codes (MARC) effect, the mechanisms that underlie these effects are still under debate. In this paper, we investigate the extraction of quantity information from German number words and nouns inflected for singular and plural using two alternative forced choice paradigms. These paradigms are applied to different tasks to investigate how access to quantity representation is modulated by task demands. In Experiment 1, we replicated previous SNARC findings for number words—that is, a relative left-hand advantage for words denoting small numbers and a right-hand advantage for words denoting large numbers in semantic tasks (parity decision and quantity comparison). No SNARC effect was obtained for surface or lexical processing tasks (font categorization and lexical decision). In Experiment 2, we found that German words inflected for singular had a relative left-hand advantage, and German words inflected for plural a relative right-hand advantage, showing a SNARC-like effect for grammatical number. The effect interfered, however, with a MARC-like effect based on the markedness asymmetry of singulars and plurals. These two effects appear to be dissociated by response latency rather than task demands, with MARC being more pronounced in early responses and SNARC being more pronounced in late responses. The present findings shed light on the relationship of conceptual number and grammatical number and constrain current accounts of the SNARC and MARC effects.     

Task bias and the accuracy of judgment: Setting a baseline for expected group performance

When a judgment task evokes unbiased estimates (i.e. the errors in individual judgments are distributed randomly around the true value), mathematical aggregation of individual estimates, even by a simple arithmetic mean, often will outperform all group members. However, when a task evokes biased estimates, mathematical aggregation does not perform so well. In this study, simulated data were accumulated to specify the expected' accuracy of mathematical aggregation relative to the accuracy of observed judgment of individual group members under varying conditions of task bias. Three types of judgment tasks were employed: (1) single-estimate, holistic tasks, (2) multiple-estimate, ranking tasks, and (3) multi-cue, decomposed tasks. Findings indicated across all task types that a large percentage of judgment-making group estimates formed strictly by computing the arithmetic mean of individual estimates performed better than their most capable members when a judgment task evoked little or no bias, a result particularly pronounced for ranking tasks. When the task was more greatly bias-evoking, a large percentage of parallel groups performed more poorly than average (or median) members, again a pattern more starkly evident for ranking tasks. These results suggest that the extent to which a judgment task evokes bias in a population of prospective group members is an important explanatory variable deserving much greater attention in the study of group performance. For example, an assertion about the efficacy of a particular group intervention based on a reliable demonstration of group performance as accurate as the most capable members may be unfounded when a task evokes no bias, since the baseline standard under such conditions should be much higher. By selecting tasks and populations that jointly produced highly biased estimates, researchers can lower the performance floor enough to detect (with reasonably small samples of groups) experimental effects should they occur.     

An ARC-REM model for accuracy and response time in recognition and recall

This article presents a model for accuracy and response time (RT) in recognition and cued recall, fitted to free-response and signal-to-respond data from Experiment 1 of P. A. Nobel and R. M. Shiffrin (2001). The model posits that recognition operates through parallel activation in a single retrieval step and cued recall operates as a sequential search. Because the data for recognition showed that variations in list length and study time per list had a large effect on accuracy but a small or negligible effect on (a) free-response RT distributions and (b) retrieval dynamics in signal-to-respond, the timing of the recognition decision is based on an assessment of retrieval completion (ARC), rather than on a sufficiency of evidence in favor of 1 of the response options. By assuming within-trial forgetting, the model predicts both the dissociation of accuracy and RT and the finding that errors are slower than correct responses. For cued recall, this model was incorporated as the 1st step in a search consisting of cycles of sampling and recovery.     

团体决策支持系统和团体讨论对团体决策的影响

通过模拟实验的方法,探讨了团体决策支持系统（GDSS）和团体讨论对团体决策的影响。以大学生为被试,运用小组决策任务,对小组决策过程进行了研究。实验记录了在有无GDSS的支持和有无团体讨论两个变量所决定的4种实验条件下,小组决策取得结果的正确性和组内的一致性。用ANOVA方法分析了4种条件下的实验结果,发现GDSS能提高团体决策的正确性,而团体讨论将有助于成员做出一个正确的、一致的决策     

Selecting polychoric instrumental variables in confirmatory factor analysis: An alternative specification test and effects of instrumental variables

The polychoric instrumental variable (PIV) approach is a recently proposed method to fit a confirmatory factor analysis model with ordinal data. In this paper, we first examine the small-sample properties of the specification tests for testing the validity of instrumental variables (IVs). Second, we investigate the effects of using different numbers of IVs. Our results show that specification tests derived for continuous data are extremely oversized at all sample sizes when applied to ordinal variables. Possible modifications for ordinal data are proposed in the present study. Simulation results show that the modified specification tests with all available IVs are able to detect model misspecification. In terms of estimation accuracy, the PIV approach where the IVs outnumber the endogenous variables by one produces a lower bias but a higher variation than the PIV approach with more IVs for correctly specified factor loadings at small samples.     

Thinking styles and risky decision‐making: Further exploration of the affect–probability weighting link

People usually overweight small probabilities and underweight large probabilities leading to the familiar inverse S‐shaped weighting function. This research explores the link between affect and the structure of probability weighting from the perspective of thinking dispositions, a concept central to dual system theories of reasoning. The effects of affective priming and cognitive load on both probability weighting and the value function are also examined. The evidence suggests that thinking styles do have predictive implications for risky decision‐making. Participants with a more affective thinking style tend to be more risk‐seeking in small probability gambles. However, increasing access to the affective system by affective priming or cognitive load manipulations tend to reduce risk‐seeking behavior in small probability gambles as well as reduce risk averse behavior in large probability gambles. Previous research, manipulating the affective nature of lottery outcomes, found evidence for an increase in curvature (more overweighting of small probabilities and more underweighting of large probabilities) of the weighting function for affect‐rich outcomes, lending support to a hope‐and‐fear deconstruction of probability weighting. The present research suggests that increased anticipatory emotions characterized by the elevation of the weighting function (more overweighting at all probabilities) is also important and could sometimes be more significant than hope‐and‐fear in decision‐making under risk. An integrated approach incorporating the impact of affect on all three, the elevation and curvature of probability weighting as well as the curvature of the value function explains the empirical findings. Copyright © 2010 John Wiley & Sons, Ltd.     

The Effects of Decision Outcome Dispersion Upon Organizational Decision Making

The effects of varying decision outcome dispersion on organizational decision making were investigated under individual and group decision making conditions. Thirty-six female and pg]36 male subjects made decisions for organizational decision scenarios in which outcomes affected primarily the decision maker, people other than the decision maker, or a group of which the decision maker was a member. Subjects rated their levels of perceived risk and confidence in their decisions and made decisions within a simulated context of either a small or a large organization. Results indicated that subjects perceived significantly less risk and more confidence in their decisions when outcomes affected primarily themselves rather than others regardless of whether the decisions were made individually or by a group. Males perceived their decisions as significantly more risky than females. Induced organizational size did not significantly influence decision making.

     

Capacity limitations and the detection of correlations: comment on Kareev (2000)

Y. Kareev (2000) argued that the limited capacity of working memory may be an adaptive advantage for the early detection of useful correlations. His analysis indeed suggests that the optimal sample size is close to G. A. Miller's (1956) "magical number 7 +/- 2." The authors point out logical and statistical limitations of Y. Kareev's (2000) analysis, including that it neglects that the adaptive value is not determined by the hit rate but by the posterior probability of hit and that only signal trials are considered. The authors' analysis demonstrates that when these limitations are corrected for, the alleged benefit for small samples does not occur, and larger samples imply considerable improvement in the detection of correlations.