Comparative evaluation of two superior stopping rules for hierarchical cluster analysis

A split-sample replication stopping rule for hierarchical cluster analysis is compared with the internal criterion previously found superior by Milligan and Cooper (1985) in their comparison of 30 different procedures. The number and extent of overlap of the latent population distributions was systematically varied in the present evaluation of stopping-rule validity. Equal and unequal population base rates were also considered. Both stopping rules correctly identified the actual number of populations when there was essentially no overlap and clusters occupied visually distinct regions of the measurement space. The replication criterion, which is evaluated by clustering of cluster means from preliminary analyses that are accomplished on random partitions of an original data set, was superior as the degree of overlap in population distributions increased. Neither method performed adequately when overlap obliterated visually discernible density nodes.This research was supported in part by NIMH grant 5R01 MH 32457 14.     

Stability criteria

Three approaches to the determination of behavioral stability were examined. In the first, a learning curve was fit to acquisition data (from Cumming and Schoenfeld, 1960), and the “experiment” stopped when the data approached sufficiently close to the theoretical asymptote. In the second, the data were analyzed for variability and linear and quadratic trend. In the third, the experiment was stopped when the magnitude of the daily changes in the data fell below a criterion. Accuracy was measured as deviation between the average value of the dependent variable when the experiment was stopped, and the average value over the last 100 sessions. The first approach was most accurate, but at the cost of requiring the most sessions and being the most difficult to apply. Both the second and third approaches provided acceptable criteria with a reasonable cost-accuracy tradeoff. The second approach permits a continuous adjustment of the criteria to accommodate the variability intrinsic in the experimental paradigm. The third, nomothetic, approach also takes into account the decreasing marginal utility of extended training sessions.     

Stopping rules for multi-category computerized classification testing

Computerized classification testing (CCT) aims to classify persons into one of two or more possible categories to make decisions such as mastery/non-mastery or meet most/meet all/exceed. A defining feature of CCT is its stopping criterion: the test terminates when there is enough confidence to make a decision. There is abundant research on CCT with a single cut-off, and two common stopping criteria are the sequential probability ratio test (SPRT) statistic and the generalized likelihood ratio statistic (GLR). However, there is a relative scarcity of research extending the SPRT to the multi-hypothesis case for when there is more than one cut-off. In this paper, we propose a new multi-category GLR (mGLR) statistic as well as a stochastically curtailed version of the CCT with three or more categories. A simulation study was conducted to show that the mGLR statistic outperformed the existing stopping rules by generating shorter average test length without sacrificing classification accuracy. Results also revealed that the stochastically curtailed mGLR successfully increased test efficiency in certain testing conditions.     

A cautionary note on using internal cross validation to select the number of clusters

A highly popular method for examining the stability of a data clustering is to split the data into two parts, cluster the observations in Part A, assign the objects in Part B to their nearest centroid in Part A, and then independently cluster the Part B objects. One then examines how close the two partitions are (say, by the Rand measure). Another proposal is to split the data into k parts, and see how their centroids cluster. By means of synthetic data analyses, we demonstrate that these approaches fail to identify the appropriate number of clusters, particularly as sample size becomes large and the variables exhibit higher correlations.The authors express their thanks to the Sol C. Snider Entrepreneurial Center, Wharton School, for support of this project.     

Estimation of the Coefficient of Variation with Minimum Risk: A Sequential Method for Minimizing Sampling Error and Study Cost

The coefficient of variation is an effect size measure with many potential uses in psychology and related disciplines. We propose a general theory for a sequential estimation of the population coefficient of variation that considers both the sampling error and the study cost, importantly without specific distributional assumptions. Fixed sample size planning methods, commonly used in psychology and related fields, cannot simultaneously minimize both the sampling error and the study cost. The sequential procedure we develop is the first sequential sampling procedure developed for estimating the coefficient of variation. We first present a method of planning a pilot sample size after the research goals are specified by the researcher. Then, after collecting a sample size as large as the estimated pilot sample size, a check is performed to assess whether the conditions necessary to stop the data collection have been satisfied. If not an additional observation is collected and the check is performed again. This process continues, sequentially, until a stopping rule involving a risk function is satisfied. Our method ensures that the sampling error and the study costs are considered simultaneously so that the cost is not higher than necessary for the tolerable sampling error. We also demonstrate a variety of properties of the distribution of the final sample size for five different distributions under a variety of conditions with a Monte Carlo simulation study. In addition, we provide freely available functions via the MBESS package in R to implement the methods discussed.     

A new secretary problem with rank-based selection and cardinal payoffs

We present an extension of the secretary problem in which the decision maker (DM) sequentially observes up to n applicants whose values are random variables X₁,X₂,…,X_n drawn i.i.d. from a uniform distribution on [0,1]. The DM must select exactly one applicant, cannot recall released applicants, and receives a payoff of x_t, the realization of X_t, for selecting the tth applicant. For each encountered applicant, the DM only learns whether the applicant is the best so far. We prove that the optimal policy dictates skipping the first sqrt(n)-1 applicants, and then selecting the next encountered applicant whose value is a maximum.     

Diagnosticity,confidence, and the need for information

Recent research has suggested that people prefer to use the most diagnostic available information as the basis for their choices and decisions, and are most confident in those decisions when information is highly diagnostic. However, the effect of information diagnosticity on the need for additional information has yet to be investigated; that is, in an optional stopping task, will the amount of information requested depend upon information diagnosticity? Three models of the role of diagnosticity in information use were examined; expected value, a confidence criterion, and information cost. Subjects attempted to categorize stimuli with the aid of information of varying costs and diagnosticity levels. They requested more information when it was obtained at a low cost. More importantly, across cost conditions, subjects consistently requested greater amounts of information when that information was of a low diagnosticity. These data seem most consistent with use of a confidence criterion that is adjusted for information costs.     

哮喘控制稳定后如何减停糖皮质激素

糖皮质激素是最有效的抗气道炎症药物,吸入糖皮质激素(ICS)是首选哮喘控制药物。由于治疗后达到哮喘临床控制(症状、肺功能)需要一定时间,ICS达到一定剂量后量效曲线较平坦,而全身不良反应却逐渐增加。因此,达到临床控制后在长期监测下至少维持哮喘控制3个月以上,开始减少50%ICS剂量,直至ICS减至最低维持剂量。如为ICS加另一种控制药(长效β2受体激动剂、白三烯调节剂、缓释茶碱)联合治疗达到控制者,在ICS减至最低剂量时再停用另一种控制药。应用最低剂量ICS维持控制达1年方可考虑停药。     

Should I test more babies? Solutions for transparent data peeking

Research with infants is often slow and time-consuming, so infant researchers face great pressure to use the available participants in an efficient way. One strategy that researchers sometimes use to optimize efficiency is data peeking (or “optional stopping”), that is, doing a preliminary analysis (whether a formal significance test or informal eyeballing) of collected data. Data peeking helps researchers decide whether to abandon or tweak a study, decide that a sample is complete, or decide to continue adding data points. Unfortunately, data peeking can have negative consequences such as increased rates of false positives (wrongly concluding that an effect is present when it is not). We argue that, with simple corrections, the benefits of data peeking can be harnessed to use participants more efficiently. We review two corrections that can be transparently reported: one can be applied at the beginning of a study to lay out a plan for data peeking, and a second can be applied after data collection has already started. These corrections are easy to implement in the current framework of infancy research. The use of these corrections, together with transparent reporting, can increase the replicability of infant research.     

Future‐biased search: the quest for the ideal

Decision‐makers with ideal candidates already in mind often extend search beyond optimal endpoints when searching for the best option among a sequential list of alternatives. Extended search is investigated here using three laboratory experiments; individuals in these tasks exhibit future‐bias, delaying choice beyond normative benchmarks. Searchers' behavior is consistent with setting high thresholds based on a focal ideal outcome without full attention to its probability or the value of second‐best alternatives; the behavior is partially debiased by manipulating which outcomes are in the searchers' focal set. Documenting future‐bias in sequential search tasks offers new insights for understanding self‐control and intertemporal choice by providing a situation in which thresholds may be set too high and myopic behavior does not prevail. Copyright © 2008 John Wiley & Sons, Ltd.