Fractional factorial designs for legal psychology

Researchers considering novel or exploratory psycholegal research are often able to easily generate a sizable list of independent variables (IVs) that might influence a measure of interest. Where the research question is novel and the literature is not developed, however, choosing from among a long list of potential variables those worthy of empirical investigation often presents a formidable task. Many researchers may feel compelled by legal psychology's heavy reliance on full-factorial designs to narrow the IVs under investigation to two or three in order to avoid an expensive and unwieldy design involving numerous high-order interactions. This article suggests that fractional factorial designs provide a reasonable alternative to full-factorial designs in such circumstances because they allow the psycholegal researcher to examine the main effects of a large number of factors while disregarding high-order interactions. An introduction to the logic of fractional factorial designs is provided and several examples from the social sciences are presented.     

A Bayesian hierarchical model for the measurement of working memory capacity

Working memory is the memory system that allows for conscious storage and manipulation of information. The capacity of working memory is extremely limited. Measurements of this limit, and what affects it, are critical to understanding working memory. Cowan (2001) and Pashler (1988) suggested applying multinomial tree models to data from change detection paradigms in order to estimate working memory capacity. Both Pashler and Cowan suggested simple formulas for estimating capacity with these models. However, in many cases, these simple formulas are inadequate, and may lead to inefficient or biased estimation of working memory capacity. I propose a Bayesian hierarchical alternative to the Pashler and Cowan formulas, and show that the hierarchical model outperforms the traditional formulas. The models are easy to use and appropriate for a wide range of experimental designs. An easy-to-use graphical user interface for fitting the hierarchical model to data is available.     

Modeling on the simulation writer interactive program

The Simulation Writer Interactive Program (SWIP) is an extension of the Michigan Experimental Simulation Supervisor program series, and is designed to permit individuals who are not skilled programmers to create numerical simulation models, primarily for instructional purposes. SWIP elicts from the user information about the manipulable (independent), central (intermediate), and observable (dependent) variables in the user’s model, as well as any necessary tables and control information. A powerful editor is available within SWIP for making modifications. SWIP deduces the structure of the user’s model from the information about the variables, and produces an interpreted version of the model which is used to generate simulated data. A variety of model structures are possible in SWIP, including multivariate models, models for repeated measures experiments, and timeseries models. A context-dependent question-answering facility is also available.     

Nominal analysis of “variance”

Nominal responses are the natural way for people to report actions or opinions. Because nominal responses do not generate numerical data, they have been underutilized in behavioral research. On those occasions in which nominal responses are elicited, the responses are customarily aggregated over people or trials so that large-sample statistics can be employed. A new analysis is proposed that directly associates differences among responses with particular sources in factorial designs. A pair of nominal responses either matches or does not; when responses do not match, they vary. That analogue to variance is incorporated in the nominal analysis of “variance” (Nanova ) procedure, wherein the proportions of matches associated with sources play the same role as do sums of squares in an anova . The Nanova table is structured like an ANOVA table. The significance levels of the N ratios formed by comparing proportions are determined by resampling. Fictitious behavioral examples featuring independent groups and repeated measures designs are presented. A Windows program for the analysis is available.     

A computerized slot machine simulation to investigate the variables involved in gambling behavior

The present paper describes a computerized slot machine simulation designed to examine many of the potential variables involved in gambling behavior. This program was created in Visual Basic Version 6.0 and is designed to run on any Windows 95 or higher equipped computer. The program allows for experimenter manipulation of probabilities of payoffs, visual display of potential winning symbols, exact sequences of wins and losses, as well as pose run time questions to the subject on prespecified trials. The program records data on a trial-bytrial basis to a text file for easy importation into many data analysis and statistical programs. Conceptual and experimental questions that may be addressed with the described program are also discussed.     

Testing Mean Differences among Groups: Multivariate and Repeated Measures Analysis with Minimal Assumptions

To date, there is a lack of satisfactory inferential techniques for the analysis of multivariate data in factorial designs, when only minimal assumptions on the data can be made. Presently available methods are limited to very particular study designs or assume either multivariate normality or equal covariance matrices across groups, or they do not allow for an assessment of the interaction effects across within-subjects and between-subjects variables. We propose and methodologically validate a parametric bootstrap approach that does not suffer from any of the above limitations, and thus provides a rather general and comprehensive methodological route to inference for multivariate and repeated measures data. As an example application, we consider data from two different Alzheimer’s disease (AD) examination modalities that may be used for precise and early diagnosis, namely, single-photon emission computed tomography (SPECT) and electroencephalogram (EEG). These data violate the assumptions of classical multivariate methods, and indeed classical methods would not have yielded the same conclusions with regards to some of the factors involved.     

Multilevel models for the experimental psychologist: Foundations and illustrative examples

Although common in the educational and developmental areas, multilevel models are not often utilized in the analysis of data from experimental designs. This article illustrates how multilevel models can be useful with two examples from experimental designs with repeated measurements not involving time. One example demonstrates how to properly examine independent variables for experimental stimuli or individuals that are categorical, continuous, or semicontinuous in the presence of missing data. The second example demonstrates how response times and error rates can be modeled simultaneously within a multivariate model in order to examine speed—accuracy trade-offs at the experimental-condition and individual levels, as well as to examine differences in the magnitude of effects across outcomes. SPSS and SAS syntax for the examples are available electronically.     

Evaluation of computer-based laboratory simulation models to teach scientific research strategies

The LABSIM simulation program is described, followed by a general evaluation of simulation and an indication of possible ways to improve the effectiveness of simulation models.     

Rank‐based permutation approaches for non‐parametric factorial designs

Inference methods for null hypotheses formulated in terms of distribution functions in general non‐parametric factorial designs are studied. The methods can be applied to continuous, ordinal or even ordered categorical data in a unified way, and are based only on ranks. In this set‐up Wald‐type statistics and ANOVA‐type statistics are the current state of the art. The first method is asymptotically exact but a rather liberal statistical testing procedure for small to moderate sample size, while the latter is only an approximation which does not possess the correct asymptotic α level under the null. To bridge these gaps, a novel permutation approach is proposed which can be seen as a flexible generalization of the Kruskal–Wallis test to all kinds of factorial designs with independent observations. It is proven that the permutation principle is asymptotically correct while keeping its finite exactness property when data are exchangeable. The results of extensive simulation studies foster these theoretical findings. A real data set exemplifies its applicability.     

Fractional Design Wizard: A computer program for cost-effective experimental research design

The computer program Fractional Design Wizard creates fractional factorial designs that are cost-effective and especially useful for discarding irrelevant factors from a large number of possible candidates. The program is intended for researchers who are relativelynew to the field of fractional design and who want to acquaint themselves with the use of fractions for the reduction of large experimental designs. Fractional designs allow estimation of main effects, and sometimes two-way interactions, without one’s having to examine all treatment conditions. The program needs Microsoft Windows 95 or better and 32 MB of memory. In a step-by-step fashion, the user can specify the required properties of the fractional design. When there are more valid designs, the user can generate these successively If necessary the user can go back to diminish the requirements. The output can be copied, printed, and saved. The program generates all the information that is needed for the use and interpretation of fractional designs. A help file explains the use of the program and also the purpose, the analysis, and the interpretation of fractional designs. The program, which is written in Object Pascal, is available as freeware on www.fss.uu.nl/ms/hl/fracdes.htm.     

Match: A program to assist in matching the conditions of factorial experiments

In most experiments that involve between-subjects or between-items factorial designs, the items and/or the participants in the various experimental groups differ on one or more variables, but need to be matched on all other factors that can affect the outcome measure. Matching large groups of items or participants on multiple dimensions is a difficult and time-consuming task, yet failure to match conditions will lead to suboptimal experiments. We describe a computer program, "Match", that automates this process by selecting the best-matching items from larger sets of candidate items. In most cases, the program produces near-optimal solutions in amatter of minutes and selects matches that are typically superior to those obtained using hand matching or other semiautomated processes. We report the results of a case study in which Match was used to generate matched sets of experimental items (words varying in length and frequency) for a published study on language processing. The program was able to come up with better-matching item sets than those hand-selected by the authors of the original study, and in a fraction of the time originally taken up with stimulus matching.     

Multilevel factorial experiments for developing behavioral interventions: power, sample size, and resource considerations

Factorial experimental designs have many potential advantages for behavioral scientists. For example, such designs may be useful in building more potent interventions by helping investigators to screen several candidate intervention components simultaneously and to decide which are likely to offer greater benefit before evaluating the intervention as a whole. However, sample size and power considerations may challenge investigators attempting to apply such designs, especially when the population of interest is multilevel (e.g., when students are nested within schools, or when employees are nested within organizations). In this article, we examine the feasibility of factorial experimental designs with multiple factors in a multilevel, clustered setting (i.e., of multilevel, multifactor experiments). We conduct Monte Carlo simulations to demonstrate how design elements-such as the number of clusters, the number of lower-level units, and the intraclass correlation-affect power. Our results suggest that multilevel, multifactor experiments are feasible for factor-screening purposes because of the economical properties of complete and fractional factorial experimental designs. We also discuss resources for sample size planning and power estimation for multilevel factorial experiments. These results are discussed from a resource management perspective, in which the goal is to choose a design that maximizes the scientific benefit using the resources available for an investigation.     

The Observer: A software system for collection and analysis of observational data

The Observer is a general-purpose software package for event recording and data analysis in behavioral research. It allows any IBM-type personal computer to serve as an event recorder. In addition, The Observer can generate dedicated event-recording programs for several types of non-IBM-compatible portable and hand-held computers and transfer files between the PC and such computers. The user specifies options through menus. The configuration can be either used directly for event recording on the PC or passed on to a program generator that creates a program to collect data on a hand-held computer. Observational data from either type of computer can be analyzed by the program. Event-recording configurations can be tailored to many different experimental designs. Keys can be designated as events, and modifiers can be used to indicate the limits of an event. The program allows grouping of events in classes and distinction between mutually exclusive versus nonexclusive events and duration events versus frequency events. Timing of events is accurate to 0.1 sec. An on-line electronic notepad permits notes to be made during an observation session. The program also includes on-line error correction. User comments as well as independent variables can be stored together with the observational data. During data analysis, the user can select the level of analysis and the type of output file. The Observer calculates frequency of occurrence and duration for classes of events, individual events, or combinations of events. For analysis of concurrence, one can select the number of nesting levels and the order of nesting. Output can be generated in the form of sorted event sequence files, text report files, and tabular ASCII files. The results can be exported to spreadsheet and statistical programs.     

A software tool for auditory and speech perception experimentation

A computer program capable of supporting auditory and speech perception experimentation is described. Given a continuum of acoustic stimuli, the program (Paradigm) allows the user to present those stimuli under different, well-known psychophysical paradigms (simple identification, identification with a rating scale, 2IAX, ABX, AXB, and oddity task). For discrimination tests, both high uncertainty (roving designs) and minimal psychophysical uncertainty (fixed designs) procedures are available. All the relevant time intervals can be precisely specified, and feedback is also available. Response times can be measured as well. Furthermore, the program stores subjects’ responses and provides summaries of experimental results for both individual subjects and groups. The program runs on Microsoft Windows (3.1 or 95) on personal computers equipped with any soundboard.     

Uncovering mental processes with factorial experiments

The analysis of reaction time (RT) additivity and interactions via factorial manipulation is a widely used and potentially powerful tool for elucidating mental processing; however, current implementation is limited by the scope of systems for which predictions are available, as well as statistical weakness. Predictions for an expanded set of system dimensions (serial/parallel, independent/dependent, and selective/nonselective influence) are given. The theorems are not limited to particular families of distributions. Statistical considerations are examined in the context of an exemplary nonadditive parallel system which adequately fits “additive” data but will only be rejected as additive when power is increased beyond the traditional criteria.     

The interaction of word frequency and concreteness in immediate serial recall

Word frequency and word concreteness are language attributes that have been shown to independently influence the recall of items in verbal short-term memory (STM). It has been argued that such effects are evidence for the action of long-term memory knowledge on STM traces. However, research to date has not investigated whether these variables interact in serial recall. In two experiments, we examined the behavior of these variables under factorial manipulation and demonstrated that the effect of word frequency is dependent on the level of concreteness of items. Serial recall performance is examined with reference to two explanatory approaches: Walker and Hulme’s (1999) dual-redintegration account and language-based models of STM. The data indicate that language-based models are more compatible with the observed effects and challenge the view that frequency and concreteness effects in STM are the products of distinct mechanisms.     

CPCA: A program for principal component analysis with external information on subjects and variables

A program is described for principal component analysis with external information on subjects and variables. This method is calledconstrained principal component analysis (CPCA), in which regression analysis and principal component analysis are combined into a unified framework that allows a full exploration of data structures both within and outside known information on subjects and variables. Many existing methods are special cases of CPCA, and the program can be used for multivariate multiple regression, redundancy analysis, double redundancy analysis, dual scaling with external criteria, vector preference models, and GMANOVA (growth curve models).     

D-optimal design for the Rasch counts model with multiple binary predictors

In this paper we derive optimal designs for the Rasch Poisson counts model and its extended version of the (generalized) negative binomial counts model incorporating several binary predictors for the difficulty parameter. To efficiently estimate the regression coefficients of the predictors, locally D-optimal designs are developed. After an introduction to the Rasch Poisson counts model and its extension, we will specify these models as particular generalized linear models. Based on this embedding, optimal designs for both models including several binary explanatory variables will be presented. Therefore, we will derive conditions on the effect sizes for certain designs to be locally D-optimal. Finally, it is pointed out that the results derived for the Rasch Poisson models can be applied for more general Poisson regression models which should receive more attention in future psychological research.