The Performance of Multivariate Methods for Two-Group Comparisons with Small Samples and Incomplete Data

Abstract

In intervention studies having multiple outcomes, researchers often use a series of univariate tests (e.g., ANOVAs) to assess group mean differences. Previous research found that this approach properly controls Type I error and generally provides greater power compared to MANOVA, especially under realistic effect size and correlation combinations. However, when group differences are assessed for a specific outcome, these procedures are strictly univariate and do not consider the outcome correlations, which may be problematic with missing outcome data. Linear mixed or multivariate multilevel models (MVMMs), implemented with maximum likelihood estimation, present an alternative analysis option where outcome correlations are taken into account when specific group mean differences are estimated. In this study, we use simulation methods to compare the performance of separate independent samples t tests estimated with ordinary least squares and analogous t tests from MVMMs to assess two-group mean differences with multiple outcomes under small sample and missingness conditions. Study results indicated that a MVMM implemented with restricted maximum likelihood estimation combined with the Kenward–Roger correction had the best performance. Therefore, for intervention studies with small N and normally distributed multivariate outcomes, the Kenward–Roger procedure is recommended over traditional methods and conventional MVMM analyses, particularly with incomplete data.     

A Comparison of ML,WLSMV, and Bayesian Methods for Multilevel Structural Equation Models in Small Samples: A Simulation Study

Multilevel structural equation models are increasingly applied in psychological research. With increasing model complexity, estimation becomes computationally demanding, and small sample sizes pose further challenges on estimation methods relying on asymptotic theory. Recent developments of Bayesian estimation techniques may help to overcome the shortcomings of classical estimation techniques. The use of potentially inaccurate prior information may, however, have detrimental effects, especially in small samples. The present Monte Carlo simulation study compares the statistical performance of classical estimation techniques with Bayesian estimation using different prior specifications for a two-level SEM with either continuous or ordinal indicators. Using two software programs (Mplus and Stan), differential effects of between- and within-level sample sizes on estimation accuracy were investigated. Moreover, it was tested to which extent inaccurate priors may have detrimental effects on parameter estimates in categorical indicator models. For continuous indicators, Bayesian estimation did not show performance advantages over ML. For categorical indicators, Bayesian estimation outperformed WLSMV solely in case of strongly informative accurate priors. Weakly informative inaccurate priors did not deteriorate performance of the Bayesian approach, while strong informative inaccurate priors led to severely biased estimates even with large sample sizes. With diffuse priors, Stan yielded better results than Mplus in terms of parameter estimates.     

Testing Serial Crime Events for Randomness in Day‐of‐Week Patterns with Small Samples

This paper discusses exact tests for evaluating whether a series of offences are randomly distributed across days of the week for small sample sizes. The context is if an analyst has identified a series of related events, can the analyst determine if those events are randomly distributed with respect to the day‐of‐week given only a few offences? This paper develops exact reference distributions because the number of potential permutations is small, and this research finds that the likelihood ratio G‐test under realistic circumstances is quite powerful. Only three crimes need to occur on the same day of the week to reject the null. Several examples of using the test under realistic circumstances are illustrated; a series of thefts of catalytic converters where the exact dates are unknown, gang shootings, and arsons over a year. Copyright © 2015 John Wiley & Sons, Ltd.     

Detecting Treatment Effects with Small Samples: The Power of Some Tests Under the Randomization Model

Randomization tests are often recommended when parametric assumptions may be violated because they require no distributional or random sampling assumptions in order to be valid. In addition to being exact, a randomization test may also be more powerful than its parametric counterpart. This was demonstrated in a simulation study which examined the conditional power of three nondirectional tests: the randomization t test, the Wilcoxon–Mann–Whitney (WMW) test, and the parametric t test. When the treatment effect was skewed, with degree of skewness correlated with the size of the effect, the randomization t test was systematically more powerful than the parametric t test. The relative power of the WMW test under the skewed treatment effect condition depended on the sample size ratio.     

Sexual Abuse and Subsequent Risky Sexual Behaviors: A Competency Model for Treatment of Adolescent Females

Sexual abuse is an antecedent to many psychological and behavioral issues that accompany client families when they present for therapy. This article reviews relevant literature regarding sexual abuse in childhood/adolescence as it relates to resultant risky sexual behaviors and their combined impact on individual and family functioning. Treatment guidelines using Trauma Focused Cognitive Behavior Therapy are presented in the context of three key elements of clinical competence: knowledge, awareness, and skills.     

Cognitive Diagnosis for Small Educational Programs: The General Nonparametric Classification Method

The focus of cognitive diagnosis (CD) is on evaluating an examinee’s strengths and weaknesses in terms of cognitive skills learned and skills that need study. Current methods for fitting CD models (CDMs) work well for large-scale assessments, where the data of hundreds or thousands of examinees are available. However, the development of CD-based assessment tools that can be used in small-scale test settings, say, for monitoring the instruction and learning process at the classroom level has not kept up with the rapid pace at which research and development proceeded for large-scale assessments. The main reason is that the sample sizes of the small-scale test settings are simply too small to guarantee the reliable estimation of item parameters and examinees’ proficiency class membership. In this article, a general nonparametric classification (GNPC) method that allows for assigning examinees to the correct proficiency classes with a high rate when sample sizes are at the classroom level is proposed as an extension of the nonparametric classification (NPC) method (Chiu and Douglas in J Classif 30:225–250, 2013). The proposed method remedies the shortcomings of the NPC method and can accommodate any CDM. The theoretical justification and the empirical studies are presented based on the saturated general CDMs, supporting the legitimacy of using the GNPC method with any CDM. The results from the simulation studies and real data analysis show that the GNPC method outperforms the general CDMs when samples are small.     

A Big Regulatory Tool-Box for a Small Technology

There is little doubt that the development and commercialisation of nanotechnologies is challenging traditional state-based regulatory regimes. Yet governments currently appear to be taking a non-interventionist approach to directly regulating this emerging technology. This paper argues that a large regulatory toolbox is available for governing this small technology and that as nanotechnologies evolve, many regulatory advances are likely to occur outside of government. It notes the scientific uncertainties facing us as we contemplate nanotechnology regulatory matters and then examines the notion of regulation itself, suggesting new ways to frame our understanding of both regulation and the regulatory tools relevant to nanotechnologies. By drawing upon three different conceptual lenses of regulation, the paper articulates a wide range of potential regulatory tools at hand. It also focuses particularly on the ways various tools are currently being used or perhaps may be employed in the future. The strengths and weaknesses characterising these tools is examined as well as the different actors involved. The paper concludes that we will increasingly face debate over what is likely to work most effectively in regulating nano technologies, the legitimacy of these different potential approaches, and the speed at which these different regimes may be employed.

Determination of Weights for Health Related Quality of Life Indicators among Kidney Patients: A Fuzzy Decision Making Method

Health-Related Quality of Life (HRQoL) is one of the significant current discussions in the health fraternity. It encompasses multidimensional indicators and serves the purpose of evaluating health quality among patients. Patients’ perceptions of the impact of disease and treatment and the indicators such as physical, psychological, social function and well being are normally investigated. However there is no clear suggestion of which indicators contributed more than others. The arbitrary nature of HRQoL paves the way for fuzzy theory in evaluation of indicators. This paper describes the application of a fuzzy decision making method in ranking indicators of HRQoL among kidney patients. Four experts in health fraternity were selected as decision makers to elicit information regarding health related status of chronic kidney disease patients over eight HRQoL indicators. The decision makers were required to rate the regularity of experiencing health-related problems in linguistic judgment among the patients. The five linguistics variables were used as input data to a modified version of Fuzzy Simple Additive Weight decision making model. The modified six-step method was possible to tap the extent of decision makers’ opinions on the severity of HRQoL experienced by the patients. It is shown that the indicator of role-physical recorded the lowest problematic level while the indicator of mental health recorded the highest problematic level experienced by the patients. The ranking signifies the impact of the indicators to health quality specifically the chronic kidney disease patients.     

A Method of Co-Therapy for Schizophrenic Families

This paper describes a model of treatment for families in which one child has been given the diagnosis of schizophrenia. Male and female therapists adopt structured roles that are used as a paradigm for exploring family patterns. The paradigm employs a three-generational hypothesis that defines schizophrenia as a lifelong, restricting mode of relating, involving the primary family triad, that makes the child vulnerable to repeated psychotic episodes. Therapy is directed at the difficulties of autonomous development within the family by focusing on differentiating the relationship between nuclear and extended family members, expecially defining and strengthening generational and sexual boundaries. The second focus of therapy is on the resolution of issues of separation and loss, with attention to unresolved mourning in the family of origin and the concomitant stress in giving up the infantile relationship with the index child in the nuclear family. Stages of therapy will be described including: (a) initiation of therapy, (b) breaking of fusion, (c) repair of alienation, and (d) solidifying the marital alliance and generational boundaries. Case illustrations will be included.     

Preferences for Online Mental Health Services Among Australian and Indian Samples: A Cross-Cultural Comparison

Online mental health services provide a point-of-access to mental healthcare that may otherwise be unavailable or limited, particularly in developing countries. Nevertheless, there is a lack of research into individual differences between those who prefer online mental health services and those who prefer traditional in-person services, and whether these differences vary as a function of culture. This study investigated differences in preferences for online or in-person mental health services on e-health literacy, age, education level, and comfort using the internet in a general community sample recruited from Australia and India. A total of 487 participants (31.6% male; mean age?=?33.55, SD?=?12.20, range 18–78), 297 Australians and 190 Indians, completed an online or paper-and-pencil survey. A significant negative relationship between age and e-health literacy was found with younger ages associated with higher e-health literacy. Furthermore, e-health literacy scores were significantly higher for the Australian sample. Age, e-health literacy, country-of-residence, education level, and comfort in using the internet did not predict mental health service preference. The results suggest that preference for in-person or online mental healthcare is independent of demographic and cultural factors and indicate that online mental health services may be acceptable to Indian health service consumers.     

Original Adult MMPI Norms in Normal Samples: A Review With Implications for Future Developments

Among the most consistent criticisms of the Minnesota Multiphasic Personality inventory (MMPI) has been the adequacy of the standardly employed adult norms. Recent work at the Mayo Foundation has provided revised adult norms, and the Restandardization Project at the University of Minnesota Press will soon provide an additional norm set. This article reviews findings from 21 studies of normal adults, comparing obtained MMPI scale values to the standard norms. These comparisons indicate that the traditional adult norms may have contained detectable degrees of bias since their publication, and that consistent patterns of differences from standard MMPI norms can be found in independent studies published as early as 1949. Two separate considerations impact on the development of new adult norms: the research need for adult norms which accurately reflect the response patterns of the general population and the clinical need to evaluate individual psychiatric patients on norms that produce codetypes related to an empirical literature spanning more than 40 years. This study notes the inherent tension between these two purposes and discusses the implications of new adult MMPI norms for research and clinical practice.     

Compositionwork: A Method for Self-Investigation

This article presents compositionwork as a method based on dialogical self theory, according to which the self is understood as a multiplicity of I-positions (different sides of the self) in the landscape of the mind. The person making the composition uses language and stones to represent relevant I-positions of her- or himself self, thus externalizing these and creating a personal landscape of mind. As a result a meta-position is taken to this landscape, which facilitates its overview and understanding and may stimulate its further development. The method relates to the two basic dimensions of the self: verbal/reflexive and nonverbal/sensory, the integration of which forms an optimal condition for the development of the self. The idea of representing I-positions by using stones is inspired by Japanese rock gardens, which often represent the organization or development of the self. By their abstract and sensory quality, stones help to evoke and symbolize nonverbal aspects of the I-positions. First, we present the theoretical concepts relevant for compositionwork, and then we elaborate on the meaning of Japanese gardens as a representation of the mind and the use of stones as symbols. Finally, we present the method itself and examples of its application.     

A Simple Statistical Method for Psychology

Schafer, Roy. Psychoanalytic Interpretation in Rorschach Testing: Theory and Application. New York: Grune &; Stratton, 1954. Pp. 446. Reviewed by Richard H. Dana.     

A Multi-Study Examination of Well-Being Theory in College and Community Samples

Well-being theory (WBT) proposes five indicators of well-being [i.e., positive emotion, engagement, relationships, meaning, achievement (PERMA)] that are, independently, empirically supported predictors of flourishing (i.e., an optimal level of well-being; Seligman in Flourish: a visionary new understanding of happiness and well-being. Free Press, NY, 2011). However, there is limited empirical support for the multidimensional model suggested by WBT. Two studies sought to test and validate the higher-order factor structure of the five components of PERMA and PERMA’s ability to predict concurrent and prospective flourishing outcomes (e.g., physical health, college success). In Study 1, a longitudinal examination of college students, participants completed measures of well-being (including four of the five PERMA indicators), physical health, and college success at the end of their sophomore, junior, and senior years. In Study 2, a larger, cross-sectional study was conducted online to further validate the PERMA model with a broader sample and all five PERMA indicators. Participants completed measures similar to those administered at Study 1 and other measures used to validate Study 1 measures. Results from Study 2 further validated the PERMA model by comparing Study 1 measures to established measures and by adding meaning to the model. Study 1 and Study 2 PERMA models predicted markers of well-being (e.g., vitality, life satisfaction) and flourishing (e.g., physical health). The two studies reported here provide cross-sectional and longitudinal support that WBT is useful for predicting flourishing.