Peer review: Selecting the best science

The major challenge facing today’s biomedical researchers is the increasing competition for available funds. The competitive review process, through which the National Institutes of Health (NIH) awards grants, is built upon review by a committee of expert scientists. The NIH is firmly committed to ensuring that its peer review system is fair and objective. Wendy Baldwin, Ph.D., is Deputy Director for Extramural Research, National Institutes of Health. This paper is based on a presentation at a workshop, “Advances in Peer Review Research”, American Association for the Advancement of Science Meeting, Baltimore, MD, February 9, 1996.     

Selecting the best unbalanced repeated measures model

This study examined the performance of selection criteria available in the major statistical packages for both mean model and covariance structure. Unbalanced designs due to missing data involving both a moderate and large number of repeated measurements and varying total sample sizes were investigated. The study also investigated the impact of using different estimation strategies for information criteria, the impact of different adjustments for calculating the criteria, and the impact of different distribution shapes. Overall, we found that the ability of consistent criteria in any of the their examined forms to select the correct model was superior under simple covariance patterns than under complex covariance patterns, and vice versa for the efficient criteria. The simulation studies covered in this paper also revealed that, regardless of method of estimation used, the consistent criteria based on number of subjects were more effective than the consistent criteria based on total number of observations, and vice versa for the efficient criteria. Furthermore, results indicated that, given a dataset with missing values, the efficient criteria were more affected than the consistent criteria by the lack of normality.     

Why best cannot last: Cultural differences in predicting regression toward the mean

Four studies were conducted to investigate cultural differences in predicting and understanding regression toward the mean. We demonstrated, with tasks in such domains as athletic competition, health and weather, that Chinese are more likely than Canadians to make predictions that are consistent with regression toward the mean. In addition, Chinese are more likely than Canadians to choose a regression‐consistent explanation to account for regression toward the mean. The findings are consistent with cultural differences in lay theories about how people, objects and events develop over time.     

Selecting the number of classes under latent class regression: a factor analytic analogue

Recently, the regression extension of latent class analysis (RLCA) model has received much attention in the field of medical research. The basic RLCA model summarizes shared features of measured multiple indicators as an underlying categorical variable and incorporates the covariate information in modeling both latent class membership and multiple indicators themselves. To reduce complexity and enhance interpretability, one usually fixes the number of classes in a given RLCA. Often, goodness of fit methods comparing various estimated models are used as a criterion to select the number of classes. In this paper, we propose a new method that is based on an analogous method used in factor analysis and does not require repeated fitting. Two ideas with application to many settings other than ours are synthesized in deriving the method: a connection between latent class models and factor analysis, and techniques of covariate marginalization and elimination. A Monte Carlo simulation study is presented to evaluate the behavior of the selection procedure and compare to alternative approaches. Data from a study of how measured visual impairments affect older persons’ functioning are used for illustration.This work was supported by National Institute on Aging (NIA) Program Project P01-AG-10184-03. The author wishes to thank Dr. Karen Bandeen-Roche for her stimulating comments and helpful discussions, and Drs. Gary Rubin and Sheila West for kindly making the Salisbury Eye Evaluation data available.     

Components of correlation and extensions of the lens model equation

By applying a general procedure for analyzing a correlation coefficient into components, the lens model equation is extended to a) analyze the effects of different types of variation and b) analyze the relations between judgmental systems that are not based on the same set of cues.This research was supported by National Institute of Mental Health Grant MH 16437. The author wishes to thank John Castellan and Derick Steinmann for helpful comments on an earlier version of this paper.     

Some comments on selecting the best of several binomial populations or the bivariate normal population having the largest correlation coefficient

Using variance stabilizing transformations, this note describes approximate solutions to the ranking and selection problem of selecting the best binomial population or the bivariate normal population with the largest correlation coefficient.     

Death by non-feeding: Not in the baby's best interests

It has recently been suggested that doctors have a duty to act in their patient's best interest and that this duty demands that life-sustaining treatment—including food and fluids—should sometimes be withheld or withdrawn and the patient allowed to die. In this article, the author explores the scope of the ‘best interests principle’ in the context of treatment decisions for seriously handicapped newborn infants. She argues that those who hold that it is permissible to starve or dehydrate an infant to death are mistaken to think that this course of action is in the infant's best interests. While it may be true that there are times whendeath is, everything considered, in an infant's best interests, a slow and distressingmethod of bringing death about is not. Since death by dehydration and starvation is not benign, the withholding of food and fluids is generally not in an infant's best interests. The author concludes by suggesting thatwhenever the withdrawal or non-employment of life-sustaining means imposes a heavy burden on the infant, the ‘best interests principle’ would demand that the infant be killed rather than allowed to die.     

Behavioral control by the response–reinforcer correlation

Using a discrete‐trials procedure, two experiments examined the effects of response–reinforcer correlations on responding while controlling molecular variables that operated at the moment of reinforcer delivery (e.g., response–reinforcer temporal contiguity, interresponse times preceding reinforcement). Each trial consisted of three successive components: Response, Timeout, and Reinforcement, with the duration of each component held constant. The correlation between the number of responses in the Response component and reinforcer deliveries in the Reinforcement component was varied. In the Positive‐correlation condition, a larger number of responses in the Response component programmed a higher reinforcement rate (Experiment 1) or a shorter time to reinforcement (Experiment 2) in the Reinforcement component. Although programmed in this way, the actual reinforcer delivery was dependent on, and occurred immediately after, a response in the Reinforcement component. In the Zero‐correlation condition, the programmed rates of reinforcement (Experiment 1) or the times to reinforcement (Experiment 2) in the Reinforcement component of each trial were yoked to those in the preceding Positive‐correlation condition. Responding in the Response component was higher in the Positive‐ than in the Zero‐correlation condition, without systematic changes in molecular variables. The results suggest that the response–reinforcer correlation can be a controlling variable of behavior.     

Eliminating the IQ-RT correlation by eliminating an experimental confound

Three experiments investigated the relation between visual scanning demands, reaction time (RT), and psychometrically defined intelligence (IQ). Prior studies have shown reliable correlations between RT and IQ in the range of −.20 to −.80. However, these studies have confounded the number of possible stimuli (stimulus uncertainty) with the size of the area in which the stimuli may appear (visual angle). Experiment 1 replicated these studies retaining this confound. As the number of stimuli increased from one to eight, the visual angle was permitted to increase as well (from 0° to 30°). The results showed that RT varied in accord with Hick's (1952) law, and a median correlation between IQ and six RT parameters (subjects' mean RTs and standard deviations at three levels of stimulus uncertainty) of −.47 was observed. Experiment 2 removed the confound, varying only stimulus uncertainty, and the median IQ-RT correlation declined to −.02. Experiment 3 held stimulus uncertainty constant at 1 bit (two stimuli) and varied visual angle; a median correlation of −.19 was observed. It was concluded that many of the previously reported correlations may not have hinged on speed of information processing alone, but at least in part on subjects' abilities to scan the display across which the stimuli appeared.     

Increasing the salience of one's best selves can undermine inspiration by outstanding role models

The accessibility of people's highest hopes and achievements can affect their reactions to upward comparisons. Three studies showed that, under normal circumstances, individuals were inspired by an outstanding role model; their motivation and self-evaluations were enhanced. However, when their most positive self-views were temporarily or chronically activated, inspiration was undermined, and individuals' motivation and self-evaluations tended to decrease. Another study found that role models inspired participants to generate more spectacular hopes and achievements than they would have generated otherwise. It appears that increasing the accessibility of one's best selves undercuts inspiration because it constrains the positivity of the future selves one may imagine and prevents one from generating the more spectacular future selves that the role model normally inspires.     

Validation of the cognitive structure of the test of signs by structural equation modeling

Validation of the cognitive structure of the test of signs by structural equation modeling. The present work is aimed to carry out a validation study of the cognitive operations required for the correct solution of items of a math test which includes basic arithmetic operations between integer numbers. The validation of the hypothesized cognitive structure is made by means of structural equation modeling and triangulation methods. Results show strong and positive cognitive subordination relationships between some items but the structural equation model fit only provides a partial support for the proposed structure. However, the triangulation procedure provides further evidence of validity.     

Replies to comments on "Which method delivers greater signal-to-noise ratio: Structural equation modelling or regression analysis with weighted composites?" by Yuan and Fang (2023)

Observational data typically contain measurement errors. Covariance-based structural equation modelling (CB-SEM) is capable of modelling measurement errors and yields consistent parameter estimates. In contrast, methods of regression analysis using weighted composites as well as a partial least squares approach to SEM facilitate the prediction and diagnosis of individuals/participants. But regression analysis with weighted composites has been known to yield attenuated regression coefficients when predictors contain errors. Contrary to the common belief that CB-SEM is the preferred method for the analysis of observational data, this article shows that regression analysis via weighted composites yields parameter estimates with much smaller standard errors, and thus corresponds to greater values of the signal-to-noise ratio (SNR). In particular, the SNR for the regression coefficient via the least squares (LS) method with equally weighted composites is mathematically greater than that by CB-SEM if the items for each factor are parallel, even when the SEM model is correctly specified and estimated by an efficient method. Analytical, numerical and empirical results also show that LS regression using weighted composites performs as well as or better than the normal maximum likelihood method for CB-SEM under many conditions even when the population distribution is multivariate normal. Results also show that the LS regression coefficients become more efficient when considering the sampling errors in the weights of composites than those that are conditional on weights.     

Illusory correlation in the perception of performance by self or a salient other

Past research has shown that judges overestimate the degree of relative association between a salient group target and an infrequent behavior. However, judges also overestimate the degree of relative association between a salient individual target and a frequent behavior. The present study investigated judgments when either the self or a salient individual was one of the targets. Subjects either performed a knowledge task along with three other participants or they observed the performance of four participants, one of whom was made salient. The knowledge task was presented as either important or unimportant. In addition, in one condition, all the participants succeeded on 70% of the trials. In the other condition, all the participants failed 70% of the time. Results in the salient individual target condition replicated previous results, in that the salient target was perceived as more strongly associated with the frequent behavior than were the nonsalient targets. This pattern of illusory correlation was also obtained for the self, with one exception: When the self and the other participants predominantly failed and the outcome was important, no such illusory correlation was shown. The results are interpreted in terms of motivational processes that are engaged by threats to the self. These motivational processes counteract or supplant the cognitive biases that typically lead to the illusory correlation based on the perception of relatively greater association between a salient individual target and the frequent behavior. Results are discussed in terms of their relevance to self-perception processes, social comparison, and research on attributional biases.