Methodological issues in interviewing and using self-report questionnaires with people with mental retardation

In this article the authors review methodological issues that arise when interviews and self-report questionnaires are used with people with mental retardation and offer suggestions for overcoming some of the difficulties described. Examples are drawn from studies that use qualitative methodology, quantitative studies assessing different question types, and studies reporting on the development of instruments measuring psychiatric symptoms, self-concept, and quality of life. Specific problems that arise with respect to item content (e.g., quantitative judgments, generalizations), question phrasing (e.g., modifiers), response format (e.g., acquiescence, multiple-choice questions), and psychometric properties (factor structure and validity) are discussed. It is argued that because many self-report questionnaires include questions that have been found to be problematic in this population, more attention needs to be paid to establishing the validity of such measures and to clearly defining the population for which the instrument is designed.     

The Use of Exploratory Factor Analysis and Principal Components Analysis in Communication Research

Exploratory factor analysis is a popular statistical technique used in communication research. Although exploratory factor analysis (EFA) and principal components analysis (PCA) are different techniques, PCA is often employed incorrectly to reveal latent constructs (i.e., factors) of observed variables, which is the purpose of EFA. PCA is more appropriate for reducing measured variables into a smaller set of variables (i.e., components) by keeping as much variance as possible out of the total variance in the measured variables. Furthermore, the popular use of varimax rotation raises some concerns about the relationships among the factors that researchers claim to discover. This paper discusses the distinct purposes of PCA and EFA, using two data sets as examples to highlight the differences in results between these procedures, and also reviews the use of each technique in three major communication journals: Communication Monographs, Human Communication Research, and Communication Research.     

探索性因素分析——最近10年的评述

目的：(1)介绍国外心理统计学界对探索性因素分析中几个重大问题的基本观点;(2)系统地评述过去10年里(1991～2000年)我国心理学研究对这一技术的使用情况;(3)强调运用这一技术时值得注意的一些事项,以期这一技术对我国心理学研究发挥更大的作用。方法：作检索了《心理学报》和《心理科学》在1991～2000年间与探索性因素分析有关的章,对其中以探索性因素分析为主要研究方法的重点章(feature article)进行了编码,统计了频数及百分比分布。结果：我国心理学研究在积极使用探索性因素分析这一先进统计技术时还存在一些问题,主要表现在：(1)在确定因素个数时,倾向于机械地依靠桌个单一方法来作决定,(2)大量使用正交旋转,(3)过于依赖SPSS,(4)对因素分析过程中的重要信息／结果报告不够。结论：探索性因素分析在过去十年得到了广泛的应用,如能吸收国外同行的一些观点,探索性因素分析这一技术在我国心理学研究中必将获得更广泛有效的应用。     

Bifactor models and rotations: exploring the extent to which multidimensional data yield univocal scale scores

The application of psychological measures often results in item response data that arguably are consistent with both unidimensional (a single common factor) and multidimensional latent structures (typically caused by parcels of items that tap similar content domains). As such, structural ambiguity leads to seemingly endless "confirmatory" factor analytic studies in which the research question is whether scale scores can be interpreted as reflecting variation on a single trait. An alternative to the more commonly observed unidimensional, correlated traits, or second-order representations of a measure's latent structure is a bifactor model. Bifactor structures, however, are not well understood in the personality assessment community and thus rarely are applied. To address this, herein we (a) describe issues that arise in conceptualizing and modeling multidimensionality, (b) describe exploratory (including Schmid-Leiman [Schmid & Leiman, 1957] and target bifactor rotations) and confirmatory bifactor modeling, (c) differentiate between bifactor and second-order models, and (d) suggest contexts where bifactor analysis is particularly valuable (e.g., for evaluating the plausibility of subscales, determining the extent to which scores reflect a single variable even when the data are multidimensional, and evaluating the feasibility of applying a unidimensional item response theory (IRT) measurement model). We emphasize that the determination of dimensionality is a related but distinct question from either determining the extent to which scores reflect a single individual difference variable or determining the effect of multidimensionality on IRT item parameter estimates. Indeed, we suggest that in many contexts, multidimensional data can yield interpretable scale scores and be appropriately fitted to unidimensional IRT models.     

Factor structure of the BASC-2 Behavioral and Emotional Screening System Student Form

The BASC-2 Behavioral and Emotional Screening System (BESS) Student Form (Kamphaus & Reynolds, 2007) is a recently developed youth self-report rating scale designed to identify students at risk for behavioral and emotional problems. The BESS Student Form was derived from the Behavior Assessment System for Children-Second Edition Self-Report of Personality (BASC-2 SRP; Reynolds & Kamphaus, 2004) using principal component analytic procedures and theoretical considerations. Using 3 samples, the authors conducted exploratory factor analyses (EFA) and confirmatory factor analyses (CFA) to understand the underlying factor structure of the BESS Student Form. The results of the EFA suggested that the SRP contained a 4-factor (i.e., Personal Adjustment, Inattention/Hyperactivity, Internalizing, School Problems) emergent structure, which was supported by CFA in 2 additional samples. Practical and research implications are discussed.     

A Comparative Investigation of Rotation Criteria Within Exploratory Factor Analysis

Exploratory factor analysis (EFA) is a commonly used statistical technique for examining the relationships between variables (e.g., items) and the factors (e.g., latent traits) they depict. There are several decisions that must be made when using EFA, with one of the more important being choice of the rotation criterion. This selection can be arduous given the numerous rotation criteria available and the lack of research/literature that compares their function and utility. Historically, researchers have chosen rotation criteria based on whether or not factors are correlated and have failed to consider other important aspects of their data. This study reviews several rotation criteria, demonstrates how they may perform with different factor pattern structures, and highlights for researchers subtle but important differences between each rotation criterion. The choice of rotation criterion is critical to ensure researchers make informed decisions as to when different rotation criteria may or may not be appropriate. The results suggest that depending on the rotation criterion selected and the complexity of the factor pattern matrix, the interpretation of the interfactor correlations and factor pattern loadings can vary substantially. Implications and future directions are discussed.     

A Validation of the Inventory of Callous-Unemotional Traits in a Community Sample of Young Adult Males

Callous-unemotional (CU) traits have been shown to delineate a subgroup of individuals at high risk for exhibiting severe and persistent criminal behavior. The Inventory of Callous-Unemotional Traits (ICU; Frick 2004) was recently developed as a comprehensive rating scale designed to measure multiple facets of CU traits. However, validation of this measure has been limited to youth in adolescence and emerging adulthood (age range?=12–20), leaving questions about the utility of this measure in early adulthood unanswered. The current study evaluated the factor structure of the ICU within a racially diverse and well characterized community sample of adult males (n?=?425) using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). While results found the best fitting model to be the three-bifactor structure that has been previously reported in adolescent samples, the fit indices were only marginally acceptable and suggest the need for scale refinement. Total and subscales scores demonstrated significant and distinct associations with relevant external criteria (e.g., delinquency, psychopathy, psychopathology, psychosocial functioning). Implications and directions for future research are discussed.     

Ordinary Least Squares Estimation of Parameters in Exploratory Factor Analysis With Ordinal Data

Exploratory factor analysis (EFA) is often conducted with ordinal data (e.g., items with 5-point responses) in the social and behavioral sciences. These ordinal variables are often treated as if they were continuous in practice. An alternative strategy is to assume that a normally distributed continuous variable underlies each ordinal variable. The EFA model is specified for these underlying continuous variables rather than the observed ordinal variables. Although these underlying continuous variables are not observed directly, their correlations can be estimated from the ordinal variables. These correlations are referred to as polychoric correlations. This article is concerned with ordinary least squares (OLS) estimation of parameters in EFA with polychoric correlations. Standard errors and confidence intervals for rotated factor loadings and factor correlations are presented. OLS estimates and the associated standard error estimates and confidence intervals are illustrated using personality trait ratings from 228 college students. Statistical properties of the proposed procedure are explored using a Monte Carlo study. The empirical illustration and the Monte Carlo study showed that (a) OLS estimation of EFA is feasible with large models, (b) point estimates of rotated factor loadings are unbiased, (c) point estimates of factor correlations are slightly negatively biased with small samples, and (d) standard error estimates and confidence intervals perform satisfactorily at moderately large samples.     

What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research

It is a common practice to export instruments developed in one culture to another. Little is known about the consequences of making inappropriate comparisons in cross-cultural research. Several studies were conducted to fill in this gap. Study 1 examined the impact of lacking factor loading invariance on regression slope comparisons. When factor loadings of a predictor are higher in the reference group (e.g., United States), for which the scale was developed, than in the focal group (e.g., China), into which the scale was imported, the predictive relationship (e.g., self-esteem predicting life satisfaction) is artificially stronger in the reference group but weaker in the focal group, creating a bogus interaction effect of predictor by group (e.g., self-esteem by culture); the opposite pattern is found when the reference group has higher loadings in an outcome variable. Studies 2 and 3 examined the impact of lacking loading and intercept (i.e., point of origin) invariance on factor means, respectively. When the reference group has higher loadings or intercepts, the mean is overestimated in that group but underestimated in the focal group, resulting in a pseudo group difference.     

Multilevel Factor Analysis: Reporting Guidelines and a Review of Reporting Practices

We provide reporting guidelines for multilevel factor analysis (MFA) and use these guidelines to systematically review 72 MFA applications in journals across a range of disciplines (e.g., education, health/nursing, management, and psychology) published between 1994 and 2014. Results are organized in terms of the (a) characteristics of the MFA application (e.g., construct measured), (b) purpose (e.g., measurement validation), (c) data source (e.g., number of cases at Level 1 and Level 2), (d) statistical approach (e.g., maximum likelihood), and (e) results reported (e.g., intraclass correlations for indicators and latent variables, standardized factor loadings, fit indices). Results from this review have implications for applied researchers interested in expanding their approaches to psychometric analyses and construct validation within a multilevel framework and for methodologists using Monte Carlo methods to explore technical and methodological issues grounded in realistic research design conditions.     

The Santa Clara Strength of Religious Faith Questionnaire (SCSORF): A Validation Study on Iranian Muslim Patients Undergoing Dialysis

The Santa Clara Strength of Religious Faith Questionnaire (SCSORF) is an often used and validated scale that is uncommonly utilized in culturally diverse populations. The purpose of this research investigation was to adapt the SCSORF for use among Iranian Muslim patients undergoing dialysis and to examine the reliability and validity of the scale among this population. A total of 428 patients (228 females, 200 males, M age = 52.2 years, SD = 10) were selected from five dialysis center in Tehran and Qazvin, Iran. A comprehensive forward–backward translation system was used for cross-cultural translation. Patients completed a baseline questionnaire obtaining demographic and clinical information as well as the SCSORF, the Age Universal Religious Orientation Scale (AUROS), the religious life inventory (RLI), and the Duke University religion index (DUREL). 2 weeks later, patients were asked to complete the SCSORF once again. Reliability of the SCSORF was examined using internal consistency and test-rest reliability. Convergent validity and factor structure using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were also examined. Cronbach’s α for the single construct of the SCSORF was 0.89 with adequate test–retest reliability measured over a 2 week period. SCSORF scores were significantly correlated with AUROS, RLI and the DUREL. The EFA generated a single factor solution for the SCSORF while these results were confirmed by the CFA in an independent sample. Findings demonstrated that the SCSORF has favorable reliability, convergent validity, and divergent validity among Iranian Muslim patients undergoing dialysis and is recommended for use by clinicians (e.g., nephrologists) to measure strength of religious faith among patients.     

Item factor analysis: current approaches and future directions

The rationale underlying factor analysis applies to continuous and categorical variables alike; however, the models and estimation methods for continuous (i.e., interval or ratio scale) data are not appropriate for item-level data that are categorical in nature. The authors provide a targeted review and synthesis of the item factor analysis (IFA) estimation literature for ordered-categorical data (e.g., Likert-type response scales) with specific attention paid to the problems of estimating models with many items and many factors. Popular IFA models and estimation methods found in the structural equation modeling and item response theory literatures are presented. Following this presentation, recent developments in the estimation of IFA parameters (e.g., Markov chain Monte Carlo) are discussed. The authors conclude with considerations for future research on IFA, simulated examples, and advice for applied researchers.     

A factor analytic investigation of the BASC-2 Behavioral and Emotional Screening System Parent Form: psychometric properties, practical implications, and future directions

The Behavior Assessment System for Children, Second Edition (BASC-2) Behavioral and Emotional Screening System Parent Form (BESS Parent; Kamphaus & Reynolds, 2007) is a recently developed instrument designed to identify behavioral and emotional risk in students. To describe the underlying factor structure for this instrument, exploratory (EFA) and confirmatory factor analyses (CFA) were conducted utilizing two subsets of a large, nationally-representative sample. The results of the EFA suggested that the BESS Parent contained a four-factor latent structure (i.e., Externalizing, Internalizing, Adaptive Skills, and Inattention), which was supported by CFA. Results support further investigation into utilizing four subscales in addition to an overall risk score; distributional and reliability information for the BESS Parent subscales is provided. Practical implications for school psychologists interested in early identification and directions for future research are discussed.     

A tutorial on some issues and methods in serial pattern research

In a review of major trends in serial pattern research, the issue of structural ambiguity and its relationship to representations of hierarchical and nonhierarchical patterns is considered. Also discussed is the role of various kinds of serial rules that can relate events within a sequence (e.g., nominal, ordinal, interval rules), as well as new developments in representing time relationships. Some practical problems that arise in the study of perception of serial structure are shown to involve its formal specification and the enlistment of appropriate controls. Major pitfalls are illustrated using recent research with music-like patterns (e.g., Deutsch, 1980).     

Risk factors for child psychological abuse

We review the risk and protective factors for child emotional abuse. Two main directions can be derived from this review. First and foremost, definitional issues must be resolved. Second, some clues as to important future research directions emerged from the current risk factor literature. Distal historical variables (e.g., relationships with fathers perceived as less caring, and being yelled at daily as a child), current enduring personality factors (e.g., aggression and hostility, neuroticism), environmental stressors (e.g., very low income) and proximal variables (verbal and physical aggression between parents) all appear to be related to child psychological abuse. Once definitional issues are resolved, models beginning with these risk factors should be developed and tested.     

Beyond Abu Ghraib: The 2010 APA Ethics Code Standard 1.02 and Competency for Execution Evaluations

To learn whether criticism and regulation of research practices have been followed by a reduction of deception or use of more acceptable approaches to deception, the contents of all 1969, 1978, 1986, and 1992 issues of the Journal of Personality and Social Psychology were examined. Deception research was coded according to type of (non)informing (e.g., false informing, consent to deception, no informing), possible harmfulness of deception employed (e.g., powerfulness of induction, morality of the behavior induced, privacy of behavior), method of deception (e.g., bogus device or role, false purpose of study, false feedback), and debriefing employed. Use of confederates has been partly replaced by uses of computers. "Consent" with false informing declined after 1969, then rose in 1992. Changes in the topics studied (e.g., attribution, socialization, personality) largely accounted for the decline in deception in 1978 and 1986. More attention needs to be given to ways of respecting subjects' autonomy, to appropriate debriefing and desensitizing, and to selecting the most valid and least objectionable deception methods.     

3岁幼儿抑制和工作记忆的可分离性研究

尽管成年期个体的执行功能结构被认为是可分离的,然而,学龄前儿童的抑制和工作记忆的可否分离还存在争议。本文使用验证性因子分析研究181名三岁儿童执行功能的主要成分抑制和工作记忆的可分离性。抑制测查采用包礼物任务、糖果延迟任务和昼夜Stroop任务,工作记忆测查采用找贴画任务、Beads任务和数点数任务。通过探索性和验证性因素分析显示:3岁幼儿的测验任务数据更为拟合抑制和工作记忆两因素结构模型, 这说明三岁幼儿的抑制和工作记忆是独立发挥作用的,抑制和工作记忆功能早在学龄前期就已分离, 并且通过测量恒等性检验发现无性别差异。     

Parent Behavior Importance Questionnaire-Revised: Scale Development and Psychometric Characteristics

This paper reports the scale development and psychometric characteristics of the Parent Behavior Importance Questionnaire-Revised (PBIQ-R). To develop this measure, 502 subject matter experts (SMEs) evaluated 91 parenting behaviors in terms of parenting behavior specificity (e.g., bonding, discipline), importance level, and appropriateness for children of differing developmental stages (e.g., infant/toddler, adolescent). SME responses were used to develop the PBIQ-R and related subscales. The resulting 73 item measure provides a psychometrically strong avenue for determining respondents’ parenting behavior values. The measure and corresponding subscales are closely aligned with current parenting theory and developmental literature with respondents indicating the relative importance of a range of positive as well as negative parenting behaviors. Results suggest moderate to strong factor consistency, construct validity, and internal consistency. Findings are discussed relative to parenting theory and research as well as clinical, counseling, and school psychology practice.     

Statistical Issues in the Study of Temporal Data: Daily Experiences

This article reviews statistical issues that arise in temporal data, particularly with respect to daily experience data. Issues related to nonindependence of observations, the nature of data structures, and claims of causality are considered. Through the analysis of data from a single subject, we illustrate concomitant time-series analysis, a general method of examining relationships between two or more series having 50 or more observations. We also discuss detection of and remedies for the problems of trend, cycles, and serial dependency that frequently plague temporal data, and present methods of combining the results of concomitant time series across subjects. Issues that arise in pooling cross-sectional and time-series data and statistical models for addressing these issues are considered for the case in which there are appreciably fewer than 50 observations and a moderate number of subjects. We discuss the possibility of using structural equation modeling to analyze data structures in which there are a large number (e.g., 200) of subjects, but relatively few time points, emphasizing the different causal status of synchronous and lagged effects and the types of models that can be specified for longitudinal data structures. Our conclusion highlights some of the issues raised by temporal data for statistical models, notably the important roles of substantive theory, the question being addressed, the properties of the data, and the assumptions underlying each technique in determining the optimal approach to statistical analysis.     

Validating the Ohio Scales in a Juvenile Justice Sample of Youth with Behavioral Health Issues

Between 65 and 75 % of juvenile-justice involved (JJI) youth present with at least one behavioral health disorder. Many communities have developed diversion programs that provide behavioral health services to JJI youth, often in lieu of detention. A key component of successful diversion programming is accurate screening and assessment. The Ohio Scales, a validated instrument designed to track service effectiveness in clinical samples of youth, are now being used with juvenile justice populations. The purpose of this study is to validate the Ohio Scales in a JJI youth population (N = 2246). The population (ages 12–18) is derived from Ohio’s Behavioral Health Juvenile Justice Initiative, a diversion program for JJI youth with behavioral health issues. We conducted Confirmatory Factor Analyses (CFA) on all forms of the Ohio Scales (parent, youth and worker) to measure fit for one factor, four factor and four factor second order solutions. We also conducted an Exploratory Factor Analysis (EFA) on the Problem Severity factor in the youth form to determine the number of appropriate sub-factors. The EFA indicated that the Problem Severity factor should be broken into three hypothesized sub-factors: Externalizing, Internalizing and Delinquency. The CFA confirmed this solution. CFA results indicated the four factor second order solution fits superior to the other two solutions. Using the Ohio Scales Problem Severity measure as a three sub-factor measure may increase clinical applicability by allowing a clinician to specifically measure and target externalizing or internalizing issues during treatment.