Levels of analysis and sources of data in sales research: a multilevel-multisource review

Sales researchers are increasingly adopting a multilevel-multisource (MLMS) approach to answer many key questions involving sales managers, salespeople and customers. MLMS research involves the acquisition and analysis of data collected from two or more sources pertaining to multiple hierarchical levels and presents a number of opportunities and challenges for sales researchers to consider. The authors highlight the benefits and the drawbacks of MLMS research and describe various approaches for implementing an MLMS collection and analysis. Additionally, a review of the MLMS studies conducted in the sales domain is provided which summarizes and delineates relationships tested in the extant literature. Based on this review, the authors advance a number of underdeveloped areas of research where MLMS approaches can be applied to further the understanding of the dynamic conditions that typify sales research.     

Secondary Data Analysis: An Important Tool for Addressing Developmental Questions

Existing data sets can be an efficient, powerful, and readily available resource for addressing questions about developmental science. Many of the available databases contain hundreds of variables of interest to developmental psychologists, track participants longitudinally, and have representative samples. In this article, the authors discuss the benefits and limitations of secondary data analysis so that researchers can make informed decisions about when it makes sense to use existing data and when it does not. They then provide an overview of best practices for handling the most common conceptual, methodological, and practical issues that arise when adopting this approach.     

Intraclass correlation associated with therapists: estimates and applications in planning psychotherapy research

It is essential that outcome research permit clear conclusions to be drawn about the efficacy of interventions. The common practice of nesting therapists within conditions can pose important methodological challenges that affect interpretation, particularly if the study is not powered to account for the nested design. An obstacle to the optimal design of these studies is the lack of data about the intraclass correlation coefficient (ICC), which measures the statistical dependencies introduced by nesting. To begin the development of a public database of ICC estimates, the authors investigated ICCs for a variety outcomes reported in 20 psychotherapy outcome studies. The magnitude of the 495 ICC estimates varied widely across measures and studies. The authors provide recommendations regarding how to select and aggregate ICC estimates for power calculations and show how researchers can use ICC estimates to choose the number of patients and therapists that will optimize power. Attention to these recommendations will strengthen the validity of inferences drawn from psychotherapy studies that nest therapists within conditions.     

Best practices for missing data management in counseling psychology

This article urges counseling psychology researchers to recognize and report how missing data are handled, because consumers of research cannot accurately interpret findings without knowing the amount and pattern of missing data or the strategies that were used to handle those data. Patterns of missing data are reviewed, and some of the common strategies for dealing with them are described. The authors provide an illustration in which data were simulated and evaluate 3 methods of handling missing data: mean substitution, multiple imputation, and full information maximum likelihood. Results suggest that mean substitution is a poor method for handling missing data, whereas both multiple imputation and full information maximum likelihood are recommended alternatives to this approach. The authors suggest that researchers fully consider and report the amount and pattern of missing data and the strategy for handling those data in counseling psychology research and that editors advise researchers of this expectation.     

Intraclass Correlation Associated with Therapists: Estimates and Applications in Planning Psychotherapy Research

It is essential that outcome research permit clear conclusions to be drawn about the efficacy of interventions. The common practice of nesting therapists within conditions can pose important methodological challenges that affect interpretation, particularly if the study is not powered to account for the nested design. An obstacle to the optimal design of these studies is the lack of data about the intraclass correlation coefficient (ICC), which measures the statistical dependencies introduced by nesting. To begin the development of a public database of ICC estimates, the authors investigated ICCs for a variety outcomes reported in 20 psychotherapy outcome studies. The magnitude of the 495 ICC estimates varied widely across measures and studies. The authors provide recommendations regarding how to select and aggregate ICC estimates for power calculations and show how researchers can use ICC estimates to choose the number of patients and therapists that will optimize power. Attention to these recommendations will strengthen the validity of inferences drawn from psychotherapy studies that nest therapists within conditions.     

Reconsidering formative measurement

The relationship between observable responses and the latent constructs they are purported to measure has received considerable attention recently, with particular focus on what has become known as formative measurement. This alternative to reflective measurement in the area of theory-testing research is examined in the context of the potential for interpretational confounding and a construct's ability to function as a point variable within a larger model. Although these issues have been addressed in the traditional reflective measurement context, the authors suggest that they are particularly relevant in evaluating formative measurement models. On the basis of this analysis, the authors conclude that formative measurement is not an equally attractive alternative to reflective measurement and that whenever possible, in developing new measures or choosing among alternative existing measures, researchers should opt for reflective measurement. In addition, the authors provide guidelines for researchers dealing with existing formative measures.     

Planned missing data designs in psychological research

The authors describe 2 efficiency (planned missing data) designs for measurement: the 3-form design and the 2-method measurement design. The 3-form design, a kind of matrix sampling, allows researchers to leverage limited resources to collect data for 33% more survey questions than can be answered by any 1 respondent. Power tables for estimating correlation effects illustrate the benefit of this design. The 2-method measurement design involves a relatively cheap, less valid measure of a construct and an expensive, more valid measure of the same construct. The cost effectiveness of this design stems from the fact that few cases have both measures, and many cases have just the cheap measure. With 3 brief simulations involving structural equation models, the authors show that compared with the same-cost complete cases design, a 2-method measurement design yields lower standard errors and a higher effective sample size for testing important study parameters. With a large cost differential between cheap and expensive measures and small effect sizes, the benefits of the design can be enormous. Strategies for using these 2 designs are suggested.     

Effects of kindergarten retention on children's social-emotional development: an application of propensity score method to multivariate, multilevel data

This study examines the effects of kindergarten retention on children's social-emotional development in the early, middle, and late elementary years. Previous studies have generated mixed results partly due to some major methodological challenges, including selection bias, measurement error, and divergent perceptions of multiple respondents in different domains of child development. The authors address these challenges by using propensity score stratification to contend with selection bias and by embedding measurement models in hierarchical models to account for measurement error and to model dependence among observations. The authors' analyses of a series of multivariate models enable them to compare the retention effects across different respondents over different time points. In general, the results show no evidence suggesting that kindergarten retention does harm to children's social-emotional development. Rather, the findings suggest that, had the retained kindergartners been promoted to the first grade instead, they would possibly have developed a lower level of self-confidence and interest in reading and all school subjects 2 years later and would have displayed a higher level of internalizing problem behaviors at the end of the treatment year and 2 years later.     

密集追踪数据分析：模型及其应用

在心理学、教育学和临床医学等领域, 越来越多的研究者开始关注个体内部的行为、心理、临床效果等随时间而产生的动态变化, 重视针对个体的差异化建模。密集追踪是一种在短时间内对个体进行多个时间节点密集追踪测量的方法, 更适合用于研究个体内部心理过程等的动态变化及其作用机制。近年来, 密集追踪成为心理学研究的一大热点, 但许多密集追踪的研究分析仍停留在较为传统的方法。方法学领域已涌现出较多用于密集追踪数据分析的模型方法, 较为主流的模型包括以动态结构方程模型(Dynamic Structural Equation Model, DSEM)为代表的自上而下的建模方法, 以及以组迭代多模型估计(Group Iterative Multiple Model Estimation, GIMME)为代表的自下而上的建模方法。二者均可以方便地对密集追踪数据中的自回归及交叉滞后效应进行建模。     

Advancing Developmental Science via Unmoderated Remote Research with Children

ABSTRACT

This article introduces an accessible approach to implementing unmoderated remote research in developmental science – research in which children and families participate in studies remotely and independently, without directly interacting with researchers. Unmoderated remote research has the potential to strengthen developmental science by: (1) facilitating the implementation of studies that are easily replicable, (2) allowing for new approaches to longitudinal studies and studies of parent-child interaction, and (3) including families from more diverse backgrounds and children growing up in more diverse environments in research. We describe an approach we have used to design and implement unmoderated remote research that is accessible to researchers with limited programming expertise, and we describe the resources we have made available on a new website (discoveriesonline.org) to help researchers get started with implementing this approach. We discuss the potential of this method for developmental science and highlight some challenges still to be overcome to harness the power of unmoderated remote research for advancing the field.     

发展瀑布效应:儿童认知功能与社会适应领域的研究

发展瀑布效应指的是在复杂的发展过程中各种交互作用或相互作用的累积结果,这种发展的累积效应导致跨越不同水平、领域、系统或者不同世代的传播效应。本文阐述了研究者利用发展瀑布效应解释儿童认知功能与社会适应领域的发展时采用的不同模型,介绍了儿童认知功能与社会适应领域的发展瀑布效应,最后对发展瀑布效应在这两个领域的研究方法与应用价值进行了总结与展望。     

Structural equation modeling of multitrait-multimethod data: different models for different types of methods

The question as to which structural equation model should be selected when multitrait-multimethod (MTMM) data are analyzed is of interest to many researchers. In the past, attempts to find a well-fitting model have often been data-driven and highly arbitrary. In the present article, the authors argue that the measurement design (type of methods used) should guide the choice of the statistical model to analyze the data. In this respect, the authors distinguish between (a) interchangeable methods, (b) structurally different methods, and (c) the combination of both kinds of methods. The authors present an appropriate model for each type of method. All models allow separating measurement error from trait influences and trait-specific method effects. With respect to interchangeable methods, a multilevel confirmatory factor model is presented. For structurally different methods, the correlated trait-correlated (method-1) model is recommended. Finally, the authors demonstrate how to appropriately analyze data from MTMM designs that simultaneously use interchangeable and structurally different methods. All models are applied to empirical data to illustrate their proper use. Some implications and guidelines for modeling MTMM data are discussed.     

Language Measurement Equivalence of the Ethnic Identity Scale With Mexican American Early Adolescents

The current study considers methodological challenges in developmental research with linguistically diverse samples of young adolescents. By empirically examining the cross-language measurement equivalence of a measure assessing three components of ethnic identity development (i.e., exploration, resolution, and affirmation) among Mexican American adolescents, the study both assesses the cross-language measurement equivalence of a common measure of ethnic identity and provides an appropriate conceptual and analytical model for researchers needing to evaluate measurement scales translated into multiple languages. Participants are 678 Mexican-origin early adolescents and their mothers. Measures of exploration and resolution achieve the highest levels of equivalence across language versions. The measure of affirmation achieves high levels of equivalence. Results highlight potential ways to correct for any problems of nonequivalence across language versions of the affirmation measure. Suggestions are made for how researchers working with linguistically diverse samples can use the highlighted techniques to evaluate their own translated measures.     

Estimating Age-Based Developmental Trajectories Using Latent Change Score Models Based on Measurement Occasion

Abstract

Accelerated longitudinal designs (ALDs) are designs in which participants from different cohorts provide repeated measures covering a fraction of the time range of the study. ALDs allow researchers to study developmental processes spanning long periods within a relatively shorter time framework. The common trajectory is studied by aggregating the information provided by the different cohorts. Latent change score (LCS) models provide a powerful analytical framework to analyze data from ALDs. With developmental data, LCS models can be specified using measurement occasion as the time metric. This provides a number of benefits, but has an important limitation: It makes it not possible to characterize the longitudinal changes as a function of a developmental process such as age or biological maturation. To overcome this limitation, we propose an extension of an occasion-based LCS model that includes age differences at the first measurement occasion. We conducted a Monte Carlo study and compared the results of including different transformations of the age variable. Our results indicate that some of the proposed transformations resulted in accurate expectations for the studied process across all the ages in the study, and excellent model fit. We discuss these results and provide the R code for our analysis.     

The pedophilic criminal

After a review of the literature dealing with pedophilia, the results of an analysis of 100 forensic psychiatric reports dealing with pedophile criminals are described. They show that, except for a few homosexual pedophiles, pedophilia is a pseudoperversion originating from different developmental conditions and, in individual cases, verifiable personality traits. The authors discuss problems involved in the forensic-psychiatric assessment of these delinquents. Attention is drawn to the necessity of purposeful, coordinated further education in this respect to enable the existing considerable discrepancies between forensic-psychiatric evaluation of these and other sexual deviants to be overcome.     

Twelve Frequently Asked Questions About Growth Curve Modeling

Longitudinal data analysis has long played a significant role in empirical research within the developmental sciences. The past decade has given rise to a host of new and exciting analytic methods for studying between-person differences in within-person change. These methods are broadly organized under the term growth curve models. The historical lines of development leading to current growth models span multiple disciplines within both the social and statistical sciences, and this in turn makes it challenging for developmental researchers to gain a broader understanding of the current state of this literature. To help address this challenge, the authors pose 12 questions that frequently arise in growth curve modeling, particularly in applications within developmental psychology. They provide concise and nontechnical responses to each question and make specific recommendations for further readings.     

Modeling fMRI Data: Challenges and Opportunities

We offer an introduction to the five papers that make up this special section. These papers deal with a range of the methodological challenges that face researchers analyzing fMRI data—the spatial, multilevel, and longitudinal nature of the data, the sources of noise, and so on. The papers all provide analyses of data collected by a multi-site consortium, the Function Biomedical Informatics Research Network. Due to the sheer volume of data, univariate procedures are often applied, which leads to a multiple comparisons problem (since the data are necessarily multivariate). The papers in this section include interesting applications, such as a state-space model applied to these data, and conclude with a reflection on basic measurement problems in fMRI. All in all, they provide a good overview of the challenges that fMRI data present to the standard psychometric toolbox, but also to the opportunities they offer for new psychometric modeling.     

Moving targets: A developmental framework for understanding children's changes following disasters

This paper proposes a developmental framework for disaster studies with children that allows researchers to explore the interaction between developmental change (defined as change that is extended, self-regulated, qualitative, and progressive) and cataclysmic change. It outlines three levels of analysis related to disasters: 1) observing the harm of disasters in the context of age-related changes in children, 2) demonstrating how children are restored to age-typical developmental trajectories following disasters, and 3) investigating how specific disaster experiences are integrated over time with children's ongoing developmental processes, such as their understanding of self and their views of the world. The paper identifies existing studies at each level of the framework, outlines how these levels of analysis can be used in constructing developmental models of disaster response, and offers suggestions for expanding disaster research into additional developmental areas.     

In Search Of True Things Worth Knowing: Considerations For A New Article Prototype

Within social and personality psychology, the existing “old prototype” of a publishable article is at odds with new expectations for transparent reporting. If researchers anticipate having to report everything while continuing to aim for a research product that includes multiple studies, examining a novel effect, with only statistically significant results, this will have negative implications for initial decisions about what research to conduct. First, researchers will be discouraged from collecting additional data because this could potentially mar existing findings. Second, they will be discouraged from pursuing questions for which the answers are unknown, as this would be a waste if the results do not fit old‐prototype expectations. These practices undermine what seem to be two universal values within personality and social psychology: truth and interestingness. Suggestions for a “new prototype” that de‐emphasizes p‐value cutoffs, multiple studies, and novelty will be discussed with an eye toward encouraging research decisions that foster true and interesting findings.     

Using Big Data and Machine Learning in Personality Measurement: Opportunities and Challenges

This conceptual paper examines the promises and critical challenges posed by contemporary personality measurement using big data. More specifically, the paper provides (i) an introduction to the type of technologies that give rise to big data, (ii) an overview of how big data is used in personality research and how it might be used in the future, (iii) a framework for approaching big data in personality science, (iv) an exploration of ideas that connect psychometric reliability and validity, as well as principles of fairness and privacy, to measures of personality that use big data, (v) a discussion emphasizing the importance of collaboration with other disciplines for personality psychologists seeking to adopt big data methods, and finally, (vi) a list of practical considerations for researchers seeking to move forward with big data personality measurement and research. It is expected that this paper will provide insights, guidance, and inspiration that helps personality researchers navigate the challenges and opportunities posed by using big data methods in personality measurement. © 2020 European Association of Personality Psychology