Modeling Clustered Data with Very Few Clusters

Small-sample inference with clustered data has received increased attention recently in the methodological literature, with several simulation studies being presented on the small-sample behavior of many methods. However, nearly all previous studies focus on a single class of methods (e.g., only multilevel models, only corrections to sandwich estimators), and the differential performance of various methods that can be implemented to accommodate clustered data with very few clusters is largely unknown, potentially due to the rigid disciplinary preferences. Furthermore, a majority of these studies focus on scenarios with 15 or more clusters and feature unrealistically simple data-generation models with very few predictors. This article, motivated by an applied educational psychology cluster randomized trial, presents a simulation study that simultaneously addresses the extreme small sample and differential performance (estimation bias, Type I error rates, and relative power) of 12 methods to account for clustered data with a model that features a more realistic number of predictors. The motivating data are then modeled with each method, and results are compared. Results show that generalized estimating equations perform poorly; the choice of Bayesian prior distributions affects performance; and fixed effect models perform quite well. Limitations and implications for applications are also discussed.     

Accommodating Small Sample Sizes in Three-Level Models When the Third Level is Incidental

Small samples sizes are a pervasive problem when modeling clustered data. In two-level models, this problem has been well studied, and several resources provide guidance for modeling such data. However, a recent review of small-sample clustered data methods has noted that no studies have investigated methods for modeling three-level data with small sample sizes. Furthermore, strategies for two-level models do not necessarily translate to the three-level context. Moreover, three-level models are prone to small samples because the “small sample” designation is primarily based on the sample size of the highest level, and large samples are increasingly difficult to amass as one progresses up a hierarchy. In this study, we focus on the case when the third level is incidental, meaning that the third level is important to consider but there are no explicit research questions at the third level. This study performs a simulation study to examine the performance of seven methods for modeling three-level data with a small sample at the third level. A motivating educational psychology example is also provided to demonstrate how the choice of method can greatly affect results.     

Development of an abbreviated Social Phobia and Anxiety Inventory (SPAI) using item response theory: the SPAI-23

An abbreviated version of the Social Phobia and Anxiety Inventory (SPAI) was developed using methods based in nonparametric item response theory. Participants included a nonclinical sample of 1,482 undergraduates (52% female, mean age = 19.4 years) as well as a clinical sample of 105 individuals (56% female, mean age = 36.4 years) diagnosed with either generalized (73%) or specific social phobia (27%). Twenty-three of the 45 SPAI items demonstrated good discrimination along the social anxiety continuum. In addition, option characteristic curves (OCCs) indicated that the SPAI's 7-point scale may generate errors in ranking individuals. Thus, options were collapsed to improve item performance. No gender differences emerged between any of the items' OCCs, suggesting that items function similarly among men and women. The abbreviated version also correlated highly with the original 45-item SPAI and exhibited similar patterns of correlations with measures of social anxiety. The SPAI-23 has considerable practical benefits, including a screening of both social and agoraphobic anxiety as well as decreased assessment and scoring time.     

Effects of self-correction on the spelling performance of junior high students with learning disabilities

The purpose of this study was to compare the effects of Self-Correction and Traditional spelling on the acquisition, maintenance, and generalization of spelling words with five junior high school students with learning disabilities. During Traditional spelling students received a weekly list of 20 unknown words. Daily 20-minute assignments with these words varied among writing them, arranging them in alphabetical order, dividing the words into syllables, and using a dictionary to locate word meaning. Students were tested on the 20 words at the end of the week. During Self-Correction, students received 20 words on a 5-column sheet of paper. Columns were arranged so that stimulus words could be hidden by folding the paper back, and later exposed after the teacher dictated and the student wrote the words. Students used proofreading marks to self-correct. Sessions lasted 20 minutes, and weekly, delayed, and generalized assessments were conducted. Results indicated that for all five students the Self-Correction procedure was more effective for word acquisition than Traditional spelling. Also, for four of the five students, maintenance of words was higher under Self-Correction. Generalization occurred for three students. Finally, measures of social validity indicated that the students preferred Self-Correction over Traditional spelling, although two teachers in regular classrooms did not notice significant changes in the overall spelling performance for the students. Implications for the classroom practitioner are discussed.     

Using Concept Mapping to Operationalize Mental Well-Being for Men and Boys

A primary purpose of many prevention-oriented interventions is to improve the general well-being and quality of life for individuals and their communities. Unfortunately, well-being is often poorly defined, with definitions embracing related issues of quality of life, happiness, and physical health. Mental well-being as a concept is also poorly defined, particularly for different population groups. As part of a larger study to assess community-level prevention efforts aimed at men and boys, a participatory approach was used to operationalize mental well-being from a male-centered, community-based perspective using concept mapping. A set of 96 statements perceived as important aspects of mental well-being for men and boys were developed and sorted by 90 participants from the study communities. The 8-cluster solution was selected as the most parsimonious and the best conceptual fit in relation to the mental well-being concept, namely, positive self-worth, supportive community, community connections, positive masculinity, responsive institutions, strong social connection, dignity and respect, and safety. These eight clusters of mental well-being, which were classified into two overarching domains of socio-environmental and emotional well-being, can provide a basis by which to assess community-based programs aimed at this population.     

Nonlinear Growth Models as Measurement Models: A Second-Order Growth Curve Model for Measuring Potential

Recent methodological work has highlighted the promise of nonlinear growth models for addressing substantive questions in the behavioral sciences. In this article, we outline a second-order nonlinear growth model in order to measure a critical notion in development and education: potential. Here, potential is conceptualized as having three components—ability, capacity, and availability—where ability is the amount of skill a student is estimated to have at a given timepoint, capacity is the maximum amount of ability a student is predicted to be able to develop asymptotically, and availability is the difference between capacity and ability at any particular timepoint. We argue that single timepoint measures are typically insufficient for discerning information about potential, and we therefore describe a general framework that incorporates a growth model into the measurement model to capture these three components. Then, we provide an illustrative example using the public-use Early Childhood Longitudinal Study–Kindergarten data set using a Michaelis-Menten growth function (reparameterized from its common application in biochemistry) to demonstrate our proposed model as applied to measuring potential within an educational context. The advantage of this approach compared to currently utilized methods is discussed as are future directions and limitations.     

Poisson Multilevel Models with Small Samples

Recent methodological studies have investigated the properties of multilevel models with small samples. Previous work has primarily focused on continuous outcomes and little attention has been paid to count outcomes. The estimation of count outcome models can be difficult because the likelihood has no closed-form solution, meaning that approximation methods are required. Although adaptive Gaussian quadrature (AGQ) is generally seen as the gold standard, its comparative performance has been investigated with larger samples. AGQ approximates the full likelihood, a function that is known to produce biased estimates with small samples with continuous outcomes. Conversely, penalized quasi-likelihood (PQL) is considered to be a less desirable approximation; however, it can approximate the restricted likelihood function, a function that is known to perform well with smaller samples with continuous outcomes. The goal of this paper is to compare the small sample bias of full likelihood methods to the linearization bias of PQL with restricted likelihood. Simulation results indicate that the linearization bias of PQL is preferable to the finite sample bias of AGQ with smaller samples.     

Parenting behaviors in parents with anxiety disorders

Anxiety disorders are familial, and although considerable evidence supports the role of genetic/biological parameters in their development, these alone do not entirely explain their etiology. In this study, the role of parenting behavior as a possible factor in the transmission of anxiety from parent to child was examined. Using interview, self-report, and direct behavioral observation, behaviors of parents with an anxiety disorder were compared to those of parents without an anxiety disorder on a number of dimensions, but particularly with respect to whether anxious parents actively inhibited their children from engaging in normal age appropriate activities. These behaviors were assessed during routine activities and in a structured non-conflictual play task. Although anxious parents did not overtly restrict their child's behavior in either type of activity, they reported higher levels of distress when their children were engaged in these activities. Similarly, the "emotional climate" in families with an anxious parent differed significantly from families without an anxious parent. The results are discussed in terms of how parenting behaviors might influence the development of maladaptive anxiety via social learning and information transfer, and their heuristic implications.