Characteristics of useful evaluation data: An example from community mental health centers

The results of an empirical study of the use of evaluation data in community mental health centers are reported. A mailed survey on evaluation use was conducted among the directors of 164 community mental health centers in 19 states; 140 completed questionnaires were returned. Results indicate that certain types of data have important impacts in a majority of centers. Systems resources management data were most highly used, followed by need assessment data, and client utilization data. Least used were data on outcomes of intervention and community impact. Data use appears closely tied to the utility of the data in carrying out priority management tasks in a center. Findings have important implications for community psychologists who plan, administer, or evaluate mental health services. The broader role of evaluation in community psychology is also discussed.     

Monitoring running-wheel movement using a serial mouse and an IBM-compatible system

A running-wheel-movement-detection system is described for use with an IBM-compatible system and a serial mouse. It is an adaptation of a system developed for use on a Commodore 64. The IBM-compatible system provides obvious advantages in speed, memory, data storage, and programming ease over the earlier Commodore 64 system.     

Using personal digital assistants (PDAs) for the collection of safety belt use data in the field

From 1975 through 2003, because of the use of safety belts, an estimated 180,000 deaths from traffic crashes have been prevented. In order to assess the gains that have been made in safety belt use across the U.S., in individual states, and in communities, belt use surveys are conducted at regular intervals to determine use rates. The most valid method for surveying safety belt use is through direct observation. Direct observation surveys are conducted along roadways by trained researchers looking into passing vehicles and recording safety belt use. This method of data collection has been effective in the past through the use of paper-and-pencil data recording, yet it could be improved through the use of electronic and communication technology. Reported here is a study designed to compare electronic data collection, using personal digital assistants (PDAs), with collection using the traditional paperand-pencil method during the annual statewide survey of safety belt use in Michigan that we have been conducting since 1984. The goals of the study were to develop a PDA database program for data entry in the field, to directly compare the PDA data collection process with the paper-and-pencil method on both accuracy and speed, and to assess mechanical and environmental factors, such as battery life, screen visibility, and reaction to adverse weather, that may act as limitations to the PDA method, in comparison with the paper-and-pencil method. In a direct comparison of methods, two observers collected data at the same roadway intersections, one using paper and a pencil and one using a PDA equipped with our custom software. The study showed that the PDA method was as fast and as accurate as the paper-and-pencil method. There were no adverse effects on the PDA caused by environmental conditions. The PDA was superior to the paper-and-pencil method in rainy weather and for data collector supervision. In addition, the use of the PDA obviated the need for entry of paper-recorded data into an electronic format. We conclude that the use of PDAs for safety belt field data collection is superior to the paper-and-pencil method. A Web address is given where the custom PDA safety belt data collection software can be obtained free of charge.     

A computer program for determining optimal data transformations minimizing skew

A program is presented that solves for an optimal transformation of data that minimizes skew. Various possible transformations are represented by a function that depends upon a single parameter, λ. A process of iterative approximations is employed to find a value of λ that will transform the data so as to produce a third moment equal to zero. The transformation found by this process may be used as computed or used to indicate which of the commonly employed transformations will most adequately counteract skew. The use of this program in processing data before analysis of variance is discussed.     

TACIT: An open-source text analysis,crawling, and interpretation tool

As human activity and interaction increasingly take place online, the digital residues of these activities provide a valuable window into a range of psychological and social processes. A great deal of progress has been made toward utilizing these opportunities; however, the complexity of managing and analyzing the quantities of data currently available has limited both the types of analysis used and the number of researchers able to make use of these data. Although fields such as computer science have developed a range of techniques and methods for handling these difficulties, making use of those tools has often required specialized knowledge and programming experience. The Text Analysis, Crawling, and Interpretation Tool (TACIT) is designed to bridge this gap by providing an intuitive tool and interface for making use of state-of-the-art methods in text analysis and large-scale data management. Furthermore, TACIT is implemented as an open, extensible, plugin-driven architecture, which will allow other researchers to extend and expand these capabilities as new methods become available.     

An MS-DOS-compatible data logger for recording subjects’ self-ratings

Some research designs require subjects to make ratings of their own internal cognitive or affective states or the behavior of other subjects. An ideal device for this purpose should allow for electronic data storage and be easy to use for research subjects. The communication box (Markman & Poltrock, 1982) has been popular for this purpose but consists of dated technology. Other data-logging devices are too complicated for subjects’ use. This paper describes an electronic data logger which uses readily available hardware. A dual-tone multiple-frequency (DTMF) telephone keypad encoder is used to generate unique tones that are stored on the audio track of a videotape or on an audio tape. A DTMF receiver decoder IC is used to decode the audio signals. The decoded signals can be read on a light-emitting diode display or entered directly into a personal computer file via an RS-232 serial interface.     

Production and control of auditory signals in attention research using the Apple-Psych system

A technical issue in auditory spatial attention research is how best to provide low-cost, accurate control of the frequency, duration, and location of acoustic signals. Three solutions to the problem are compared: use of the internal oscillator of the Apple lIe, use of a Mountain A/D+D/A card, and use of an external oscillator chip. The unique advantages and drawbacks of each system are discussed. All share some common problems of external speaker interface, timing control, and interface with the software. Hardware and software solutions implemented on the Apple-Psych system are discussed, along with data that demonstrate the precision control achieved with the system.     

A computer-aided digital audio recording and encoding system for improving the encoding of verbal reports

The increasing use of verbal reports in psychological research requires tools for improving the ease and reliability of collecting and coding verbal report data. An approach is described that maintains the verbal report data in digitally recorded audio form throughout the collecting and encoding processes. A new computer-aided encoding tool, CAPAS, is described, which randomly selects and plays individual protocol segments and stores computer keyboard-entered codes in an SPSS-formatted data file.     

On belief bias in syllogistic reasoning

A multinomial model is used to disentangle the respective contributions of reasoning processes and response bias in conclusion-acceptance data that exhibit belief bias. A model-based meta-analysis of 22 studies reveals that such data are structurally too sparse to allow discrimination of different accounts of belief bias. Four experiments are conducted to obtain richer data, allowing deeper tests through the use of the multinomial model. None of the current accounts of belief bias is consistent with the complex pattern of results. A new theory of belief bias is proposed that assumes that most reasoners construct only one mental model representing the premises as well as the conclusion or, in the case of an unbelievable conclusion, its logical negation. New predictions derived from the theory are confirmed in 4 additional studies.     

Trends in married couples' time use: Evidence from 1977–78 and 1987–88

Conflicting evidence regarding married couples' time use trends has recently emerged in the literature. A critical review of this literature suggests that variations in data quality and methodological approaches may be responsible for many of the observed differences. Capitalizing on the insights gained from this review, a trend analysis of couples' time use is presented in this paper. The analysis is based on two cross-sections of time diary data collected from two-parent, two-child households. The results suggest that the relationships between socioeconomic variables and couples' time use changed during the period from 1977–78 to 1987–88. On average, these changes lead to an increase of over seven hours per week in married women's productive work time (i.e., market work plus household work). Correspondingly, the productive work time of husbands increased by an average of three and one-half hours per week. Both increases came at the expense of time that was formerly spent in more leisure-oriented activities.Financial support for the collection of the data used in this study was provided by the Utah State Agricultural Experiment Station and the University of Utah Research Committee. A grant of computer time from the Utah Supercomputing Institute, which is funded by the State of Utah and the IBM Corporation, is greatfully acknowledged. Additional support has been provided by the Department of Home Economics and Consumer Education at Utah State University where Dr. Zick was on leave during the 1989–90 academic year. W. Keith Bryant, Ken R. Smith, and an anonymous reviewer each gave helpful comments on an earlier draft of this article.     

Application of local linear embedding to nonlinear exploratory latent structure analysis

In this paper we discuss the use of a recent dimension reduction technique called Locally Linear Embedding, introduced by Roweis and Saul, for performing an exploratory latent structure analysis. The coordinate variables from the locally linear embedding describing the manifold on which the data reside serve as the latent variable scores. We propose the use of semiparametric penalized spline methods for reconstruction of the manifold equations that approximate the data space. We also discuss a crossvalidation strategy that can guide in selecting an appropriate number of latent variables. Synthetic as well as real data sets are used to illustrate the proposed approach. A nonlinear latent structure representation of a data set also serves as a data visualization tool.     

A multivariate reduced-rank growth curve model with unbalanced data

A multivariate reduced-rank growth curve model is proposed that extends the univariate reducedrank growth curve model to the multivariate case, in which several response variables are measured over multiple time points. The proposed model allows us to investigate the relationships among a number of response variables in a more parsimonious way than the traditional growth curve model. In addition, the method is more flexible than the traditional growth curve model. For example, response variables do not have to be measured at the same time points, nor the same number of time points. It is also possible to apply various kinds of basis function matrices with different ranks across response variables. It is not necessary to specify an entire set of basis functions in advance. Examples are given for illustration.The work reported in this paper was supported by Grant A6394 from the Natural Sciences and Engineering Research Council of Canada to the second author. We thank Jennifer Stephan for her helpful comments on an earlier version of this paper. We also thank Patrick Curran and Terry Duncan for kindly letting us use the NLSY and substance use data, respectively. The substance use data were provided by Grant DA09548 from the National Institute on Drug Abuse.     

Multi-response permutation procedure as an alternative to the analysis of variance: An SPSS implementation

A permutation test typically requires fewer assumptions than does a comparable parametric counterpart. The multi-response permutation procedure (MRPP) is a class of multivariate permutation tests of group difference useful for the analysis of experimental data. However, psychologists seldom make use of the MRPP in data analysis, in part because the MRPP is not implemented in popular statistical packages that psychologists use. A set of SPSS macros implementing the MRPP test is provided in this article. The use of the macros is illustrated by analyzing example data sets.     

Time-sharing SKED and OS/8: A new version

A new time-sharing SKED program has been developed to simultaneously conduct up to 12 behavioral experiments while the OS/8 operating system is used in the “background.” The new system provides improved control over the SKED program while allowing the experimenter to develop new programs or analyze data during the experimental session. A minimum of 16K of memory and an OS/8 device is required to use this system.     

SNP: A system for nonadaptive procedures

A software system which is currently being used to collect and analyze data for a wide range of auditory psychophysical procedures is described. Its principal advantages are its scope, its ease of use during data collection, the uniformity it lends to data collection procedures in widely different experiments, and the fact that no additional programming is required to run most experiments. Although the system reflects a specific interface configuration and the availability of an 8K PDP-15, the programming has been done in FORTRAN and contains a number of features which may be of interest to even Es building hard-wired programming devices.     

An adaptive approach for modelling multiattribute choice

An adaptive approach for modelling individual-level choice among multiattribute alternatives using the binary logit model is presented. The algorithm involves the collection of paired comparison data. In an effort to maximize the amount of information obtainable from each response, it is based on the experimental design criterion of D-optimality. A simulation study indicates that the proposed algorithm outperforms other sequential selection approaches in terms of estimation accuracy and predictive efficiency under certain circumstances. The results appear to encourage the use of such an adaptive algorithm for individual-level modelling in light of the potential reduction in data requirements without significant loss in predictive accuracy.     

Improving the encoding of verbal reports by using MPAS: A computer-aided encoding system

The dramatically increased use of verbal report methodologies in psychological research has created a need for new tools to improve the efficiency and reliability of encoding these data. A computer-aided protocol encoding system called MPAS (Multiple Protocol Analysis System) presents individual protocol segments in a randomised order to one or more coders and then stores computer keyboard-entered codes for later output to an SPSS formatted data file. In the present paper, MPAS is described, and a brief example is provided of how MPAS can be used in a con-trolled laboratory study to rigorously analyze verbal protocol data.     

A general index of self-reported sleep: the sleep quotient

Utilizing multiple sleep measures, a practice common to virtually all sleep assessments, poses problems both for statistical analyses in the form of inflated Type I error rate and for interpretation by the inability to specify general sleep change. A sleep quotient (SQ) is proposed that appears to overcome these problems. The paper presents archival norms for self-report sleep data, demonstrates the use of these norms in transforming novel data to standardized units, and casts the standardized score in the form of an intelligence quotient, now called a SQ, with a mean of 100 and a standard deviation of 15. Previously published data are reanalyzed by use of SQ scores, methods of interpreting SQ data are demonstrated, and advantages of SQ analyses are described. The SQ appears to be a valid, useful composite sleep index permitting the measurement of general sleep.