A practical guide to multilevel modeling

Collecting data from students within classrooms or schools, and collecting data from students on multiple occasions over time, are two common sampling methods used in educational research that often require multilevel modeling (MLM) data analysis techniques to avoid Type-1 errors. The purpose of this article is to clarify the seven major steps involved in a multilevel analysis: (1) clarifying the research question, (2) choosing the appropriate parameter estimator, (3) assessing the need for MLM, (4) building the level-1 model, (5) building the level-2 model, (6) multilevel effect size reporting, and (7) likelihood ratio model testing. The seven steps are illustrated with both a cross-sectional and a longitudinal MLM example from the National Educational Longitudinal Study (NELS) dataset. The goal of this article is to assist applied researchers in conducting and interpreting multilevel analyses and to offer recommendations to guide the reporting of MLM analysis results.     

Discounting: A practical guide to multilevel analysis of choice data

Multilevel modeling provides the ability to simultaneously evaluate the discounting of individuals and groups by examining choices between smaller sooner and larger later rewards. A multilevel logistic regression approach is advocated in which sensitivity to relative reward magnitude and relative delay are considered as separate contributors to choice. Examples of how to fit choice data using multilevel logistic models are provided to help researchers in the adoption of these methods.     

A practical guide to understanding reliability in studies of within-person variability

This article concerns how to estimate reliability (defined as the internal consistency of responses to a scale) in designs that are commonly used in studies of within-person variability. I present relevant issues, describe common errors, make recommendations for best practice, and discuss unresolved issues and future directions. I describe how to estimate the reliability of scales administered in studies in which observations are nested within persons, such as daily diary and “beeper” studies and studies of social interaction. Multilevel modeling analyses that include a measurement level can estimate the occasionlevel (e.g., days or beeps or interactions) reliability of scales. In such models, items on a scale are nested within occasions of measurement and occasions of measurement are nested within persons.     

Discounting: A practical guide to multilevel analysis of indifference data

Multilevel modeling provides the ability to simultaneously evaluate the discounting of individuals and groups using indifference point data. After considering the conditions when weaknesses emerge in estimating individual discounting as a prelude to estimating group discounting, examples are provided that indicate that multilevel modeling improves estimation in the presence of variability and missing data, and when trying to fit two‐parameter discounting functions. Concrete examples of how to fit nonlinear multilevel models are provided to help researchers in the adoption of these methods.     

Generalizability theory: A practical guide to study design,implementation, and interpretation

Generalizability Theory (GT) offers increased utility for assessment research given the ability to concurrently examine multiple sources of variance, inform both relative and absolute decision making, and determine both the consistency and generalizability of results. Despite these strengths, assessment researchers within the fields of education and psychology have been slow to adopt and utilize a GT approach. This underutilization may be due to an incomplete understanding of the conceptual underpinnings of GT, the actual steps involved in designing and implementing generalizability studies, or some combination of both issues. The goal of the current article is therefore two-fold: (a) to provide readers with the conceptual background and terminology related to the use of GT and (b) to facilitate understanding of the range of issues that need to be considered in the design, implementation, and interpretation of generalizability and dependability studies. Given the relevance of this analytic approach to applied assessment contexts, there exists a need to ensure that GT is both accessible to, and understood by, researchers in education and psychology. Important methodological and analytical considerations are presented and implications for applied use are described.     

Analyzing sequential relations in observational data: A practical guide

Many observational systems in basic and applied research produce a record of sequences of events over time. Within such observational systems, important information may be found in the frequency of the transitions between events that does not emerge in the typical researcher's focus on absolute event frequency. Indeed, if many behaviors are controlled by closely adjacent preceding events, then substantial prediction and control can be obtained through knowledge and manipulation of causal event transitions. A method for analyzing and testing sequential dependencies between events is proposed as part of an integrated package of computer-based data entry, storage, and analysis procedures. The mathematical portion of these techniques is based on the statistic, kappa,which is applicable to determining whether particular transitions among events differ from chance and whether particular transitions differ significantly across groups of subjects. Low-cost hardware and software to implement the proposed procedures are described.This work has been supported by Grant 1 RO1 HD19245-01A1 from the National Institute of Child Health and Human Development to the two senior authors and by grants of equipment from Commodore Computers, Inc., Koala Corporation, and NEC Telephones, Inc.     

A practitioner's guide to meta-analysis

Describes the application to community issues of the meta-analytic research strategies increasingly used in many field of psychology. First, we highlight the potential value of meta-analysis to community research. Second, we describe six major steps involved in conducting an effective meta-analysis. These steps include formulating the initial research question(s), locating relevant studies, abstracting critical information from each study, and presenting, analyzing, and interpreting the resultant data. In this guide, the major aspects of meta-analysis are discussed with particular emphasis on the procedures that are most critical to the validity of its conclusions. Greater familiarity with the techniques, issues, and potential of meta-analysis may stimulate investigators to make more effective use of this powerful approach to integrating research in community psychology.     

A guide to dynamical analysis

The number and variety of methods used in dynamical analysis has increased dramatically during the last fifteen years, and the limitations of these methods, especially when applied to noisy biological data, are now becoming apparent. Their misapplication can easily produce fallacious results. The purpose of this introduction is to identify promising new methods and to describe safeguards that can be used to protect against false conclusions.     

A survival guide to fission

The fission of a person involves what common sense describes as a single person surviving as two distinct people. Thus, say most metaphysicians, this paradox shows us that common sense is inconsistent with the transitivity of identity. Lewis’s theory of overlapping persons, buttressed with tensed identity, gives us one way to reconcile the common sense claims. Lewis’s account, however, implausibly says that reference to a person about to undergo fission is ambiguous. A better way to reconcile the claims of common sense, one that avoids this ambiguity, is to recognize branching persons, persons who have multiple pasts or futures.

N-of-1 methods: A practical guide to exploring trajectories of behaviour change and designing precision behaviour change interventions

Objectives(1) To introduce N-of-1 methods and how they can help the researchers identify predictors of behavioural outcomes, (2) to provide examples of studies that test individual theory-based predictions of physical activity and/or exercise; (3) to provide a practical example dataset to illustrate how to design and undertake a basic analysis for an N-of-1 study; and (4) to suggest a future agenda for N-of-1 physical activity and exercise research.DesignFactors for consideration when designing an N-of-1 study include variability of predictors and outcomes, assessment frequency and appropriate analysis methods. Existing literature and piloting can help inform these aspects.MethodsWe use a dataset of 24 individuals who collected data over 28 days to illustrate example analysis procedures. Data, guidance and associated SPSS and R syntax are made available to provide researchers with tools to learn about and practice N-of-1 analysis.ResultsGuidance on dealing with missing data, looking at graphical representations of N-of-1 data, managing autocorrelation using the prewhitening method and analysing N-of-1 datasets is provided. Using the example dataset, we demonstrate how to identify antecedents of physical activity (steps) to assess directionality of associations. We also include an overview of aggregating N-of-1 datasets using multilevel modelling.ConclusionsN-of-1 methodology provides a means of tracking individual patterns of behaviour and identifying potential antecedents of physical activity and exercise to help determine causality. Assisted by mobile technologies, there is great potential to enrich our understanding of movement behaviour using this approach to inform interventions.     

A practical guide to time-frequency analysis in the study of human motor behavior: the contribution of wavelet transform

The authors present a practical guide for studying nonstationary data on human motor behavior in a time-frequency representation. They explain the limits of classical methods founded exclusively on the time or frequency basis and then answer those limits with the windowed Fourier transform and the wavelet transform (WT) methods, both of which are founded on time-frequency bases. The authors stress an interest in the WT method because it permits access to the whole complexity of a signal (in terms of time, frequency, amplitude, and phase). They then show that the WT method is well suited for the analysis of the interaction between two signals, particularly in human movement studies. Finally, to demonstrate its practical applications, the authors apply the method to real data.     

Different randomized multiple-baseline models for different situations: A practical guide for single-case intervention researchers

Single-case researchers often implement multiple-baseline designs as their preferred methodology for intervention evaluations. Recent writings and empirical investigations have argued in favor of incorporating various forms of randomization into such designs for the purpose of elevating the intervention study's internal validity and scientific credibility. In this article, we consider a variety of randomized multiple-baseline designs and associated randomization statistical tests, along with their potential strengths and limitations. In what amounts to a practical guide, we refer school psychology researchers to these versatile randomization procedures for planning and executing their intervention studies.     

An A to Z guide to DSM-IV conundrums

The work on the 4th edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) poses many puzzling conundrums that touch on complicated and important theoretical and practical issues. None of these can be resolved in the DSM-IV, but we hope that the Task Force's decisions will be informed by thorough reviews of the currently available evidence and extensive input from all sectors of the mental health field. In this article we provide an alphabetical guide to DSM-IV conundrums that we hope will stimulate comments, suggestions, and criticisms about the work of the Task Force.     

A practical approach to school psychology

Exclusive use of the clinical model in school psychology is impractical. This is discussed in terms of incidence of adjustment problems, psychologist-student ratios, and undesirable side effects that have resulted from sole use of the clinical model. Reasons for implementing developmental and preventive mental health programs in the schools are discussed, and examples of such programs are presented. Some alternatives to the usual ways of handling the remediation of currently existing adjustment problems are suggested.