Toward an Experimental Timing Standards Lab: Benchmarking precision in the real world

Much discussion has taken place over the relative merits of various platforms and operating systems for real-time data collection. Most would agree that, provided great care is taken, many are capable of millisecond timing precision. However, to date, much of this work has focused on the theoretical aspects of raw performance. It is our belief that researchers would be better informed if they could place confidence limits on their own specific paradigms in situ and without modification. To this end, we have developed a millisecond precision test rig that can control and time experiments on a second presentation machine. In this paper, we report on the specialisthardware and software used. We elucidate the importance of the approach in relation to real-world experimentation.     

Millisecond timing on PCs and Macs

A real-time, object-oriented solution for displaying stimuli on Windows 95/98, MacOS and Linux platforms is presented. The program, written in C++, utilizes a special-purpose window class (GLWindow), OpenGL, and 32-bit graphics acceleration; it avoids display timing uncertainty by substituting the new window class for the default window code for each system. We report the outcome of tests for real-time capability across PC and Mac platforms running a variety of operating systems. The test program, which can be used as a shell for programming real-time experiments and testing specific processors, is available at http://www.cs.dal.ca/~macinnwj. We propose to provide researchers with a sense of the usefulness of our program, highlight the ability of many multitasking environments to achieve real time, as well as caution users about systems that may not achieve real time, even under optimal conditions.     

Micro Experimental Laboratory: An integrated system for IBM PC compatibles

Micro Experimental Laboratory (MEL) is a third-generation integrated software system for experimental research. The researcher fills in forms, and MEL writes the experimental program, runs the experiments, and analyzes the data. MEL includes a form-based user interface, automatic programming, computer tutorials, a compiler, a real-time data acquisition system, database management, statistical analysis, and subject scheduling. It can perform most reaction time, questionnaire, and text comprehension experiments with little or no programming. It includes a Pascal-like programming language and can call routines written in standard languages. MEL operates on IBM PC compatible computers and supports most display controllers. MEL maintains millisecond timing with high-speed text and graphics presentation. MEL provides a systematic approach to dealing with nine concerns in running an experimental laboratory.     

FTAP: A Linux-based program for tapping and music experiments

This paper describes FTAP, a flexible data collection system for tapping and music experiments. FTAP runs on standard PC hardware with the Linux operating system and can process input keystrokes and auditory output with reliable millisecond resolution. It uses standard MIDI devices for input and output and is particularly flexible in the area of auditory feedback manipulation. FTAP can run a wide variety of experiments, including synchronization/continuation tasks (Wing & Kristofferson, 1973), synchronization tasks combined with delayed auditory feedback (Aschersleben & Prinz, 1997), continuation tasks with isolated feedback perturbations (Wing, 1977), and complex alterations of feedback in music performance (Finney, 1997). Such experiments have often been implemented with custom hardware and software systems, but with FTAP they can be specified by a simple ASCII text parameter file. FTAP is available at no cost in source-code form.     

Amiga 1000 hardware timing and reaction-time key interfacing

A method of hardware reaction timing with millisecond accuracy, using one of the Amiga’s CIA 8520 chips, is described. The registers of this chip can be set to enable cascaded timing that functions independently of the CPU and, thereby, avoids the problems of software timing in a multitasking environment. In addition, the interfacing of a pair of reaction-time keys to one of the Amiga’s game controller connectors and a program for polling this port for keypresses are described.     

A satellite system for controlling real-time experiments

In this paper, we describe a computer system for controlling real-time psychology experiments. We begin by considering the kinds of experiments that are performed in our laboratory and examining the hardware and software requirements of these experiments. We then review various systems along a continuum ranging from general-purpose timesharing systems to stand-alone dedicated processors. The capabilities of these systems are described in comparison with our requirements. The features of the PARASITE-FS system are then described. This host-satellite system includes a local file system and monitor program, real-time device drivers with flexible interrupt processing capabilities, user-level routines for controlling the real-time devices, and a package for controlling the timing of experimental events.     

Millisecond accuracy video display using OpenGL under Linux

To measure people’s reaction times to the nearest millisecond, it is necessary to know exactly when a stimulus is displayed. This article describes how to display stimuli with millisecond accuracy on a normal CRT monitor, using a PC running Linux. A simple C program is presented to illustrate how this may be done within X Windows using the OpenGL rendering system. A test of this system is reported that demonstrates that stimuli may be consistently displayed with millisecond accuracy. An algorithm is presented that allows the exact time of stimulus presentation to be deduced, even if there are relatively large errors in measuring the display time.     

Windows executable software for the progressive demasking task

The progressive demasking (PDM) task is a perceptual identification task using degraded stimulus presentation procedures, but with the advantage of providing response time (RT) measures on each trial. Participants simply press a response button whenever they recognize a word. In this article, we present newly developed, easy-to-use freeware for PDM. Considering the difficulty of programming such a paradigm with standard experimental software, we developed a program based on high-level C libraries to be executed on Windows XP computers. The program execution is millisecond accurate forRT measurements and generates the fewest possible display duration errors.     

PyParse: A semiautomated system for scoring spoken recall data

Studies of human memory often generate data on the sequence and timing of recalled items, but scoring such data using conventional methods is difficult or impossible. We describe a Python-based semiautomated system that greatly simplifies this task. This software, called PyParse, can easily be used in conjunction with many common experiment authoring systems. Scored data is output in a simple ASCII format and can be accessed with the programming language of choice, allowing for the identification of features such as correct responses, prior-list intrusions, extra-list intrusions, and repetitions.     

FORTH,a software solution to real-time computing problems

Some common real-time computing problems are discussed, namely, the programming and testing of devices, problems relating to program execution speed, the timing of experimental events, and the programming of multiple parallel events. The programming language FORTH is shown to be capable of providing solutions to all of these problems. In addition, FORTH has the advantages of being an interpretive and extensible language with both high-level and low-level capabilities, while at the same time being efficient and fast in execution.     

Object-oriented millisecond timers for the PC

Object-oriented programming provides a useful structure for designing reusable code. Accurate millisecond timing is essential for many areas of research. With this in mind, this paper provides a Turbo Pascal unit containing an object-oriented millisecond timer. This approach allows for multiple timers to be running independently. The timers may also be set at different levels of temporal precision, such as 10⁻³ (milliseconds) or 10⁻⁵ sec. The object also is able to store the time of a flagged event for later examination without interrupting the ongoing timing operation.     

C language functions for millisecond timing on the IBM PC

This article describes four C language functions for programming the IBM PC and compatibles for timing with millisecond precision. The technique, which is based on a reprogramming of the PC’s real time clock, requires no additional hardware, no assembly language code, and no programming of machine or software interrupts. One function restores the PC’s time-of-day clock.     

A PC parallel port button box provides millisecond response time accuracy under Linux

For psychologists, it is sometimes necessary to measure peoples reaction times to the nearest millisecond. This article describes how to use the PC parallel port to receive signals from a button box to achieve millisecond response time accuracy. The workings of the parallel port, the corresponding port addresses, and a simple Linux program for controlling the port are described. A test of the speed and reliability of button box signal detection is reported. If the reader is moderately familiar with Linux, this article should provide sufficient instruction for him or her to build and test his or her own parallel port button box. This article also describes how the parallel port could be used to control an external apparatus     

Applications of multiprogramming software to real-time experiments in psychology

Multiprogramming operating systems are often advertised as solving the problem of competition among independent tasks operating on the same computer system. In real-time laboratories, multiprogramming systems are much more valuable for their ability to manage the relationships among asynchronous, cooperating tasks that are part of a single experiment. This cooperation allows the programming of paradigms that would otherwise require the use of faster and more expensive hardware. Examples are given from several languages and operating systems, including the small, home-built PSYCLE system and the commercially available VORTEX II system.     

Interval timing routines for the IBM PC/XT/AT microcomputer family

We describe a 1-msec software timer for measuring response latencies and controlling delays on the IBM PC/XT/AT without additional hardware requirements. To demonstrate the machine language routines, a short BASIC example program is included. In a simple experimental design, two different stimulus words are presented on screen and keypress response latencies are measured. Precise timing of stimulus presentation is accomplished by direct manipulation of the video controller. The principles of programming interrupt-controlled timing routines are addressed to be easily adapted to other problems or different programming languages.     

Time-stamping computer events to report .1-msec accuracy of events in the Micro Experimental Laboratory

A critical problem in most real-time programming tasks is the verification that timing of the presented displays and recorded events occur at the expected times. It is often difficult for psychologists to verify critical timing because of the lack of costly specialized tools and technicians to measure events with millisecond accuracy. Algorithms utilized in a new Time Audit mechanism, added to the Micro Experimental Laboratory, are described. This new mechanism involves the recording and time stamping of all input and output events with 0.1-msec accuracy. This allows the experimenter to determine the initiation, duration, and termination of each event and also makes it possible to relate these events to the synchronization of the screen refresh cycle. A customized user interface provides a time log and event-tracing feature that enables nonprogrammers to determine the duration of command execution to the .1-msec level, allowing rapid, precise assessment of program execution. The resulting time log provides a detailed specification of the experimental events for debugging and a permanent record of the experimental procedure.     

Dissecting the clock: understanding the mechanisms of timing across tasks and temporal intervals

Currently, it is unclear what model of timing best describes temporal processing across millisecond and second timescales in tasks with different response requirements. In the present set of experiments, we assessed whether the popular dedicated scalar model of timing accounts for performance across a restricted timescale surrounding the 1-second duration for different tasks. The first two experiments evaluate whether temporal variability scales proportionally with the timed duration within temporal reproduction. The third experiment compares timing across millisecond and second timescales using temporal reproduction and discrimination tasks designed with parallel structures. The data exhibit violations of the assumptions of a single scalar timekeeper across millisecond and second timescales within temporal reproduction; these violations are less apparent for temporal discrimination. The finding of differences across tasks suggests that task demands influence the mechanisms that are engaged for keeping time.     

COMPUTERIZED SYSTEMS FOR COLLECTING REAL-TIME OBSERVATIONAL DATA

Advances in computer technology have led to the development of a number of semiautomated systems for collecting real-time observational data. We conducted a survey of 15 developers of computerized systems and summarized the features of each system. Many of these systems have incorporated laptop or handheld computers as well as bar-code scanners. Most systems used IBM-compatible (DOS or Windows) software, although a few were designed for either the MacOS or some other operating system. The range in prices started from free to more complete systems costing over $1,500. Data analysis programs were included with most programs; however, only about a third of the systems included a program to compute interobserver agreement.     

Testing the accuracy of timing reports in visual timing tasks with a consumer-grade digital camera

This study tested the accuracy of a visual timing task using a readily available and relatively inexpensive consumer grade digital camera. A visual inspection time task was recorded using short high-speed video clips and the timing as reported by the task’s program was compared to the timing as recorded in the video clips. Discrepancies in these two timing reports were investigated further and based on display refresh rate, a decision was made whether the discrepancy was large enough to affect the results as reported by the task. In this particular study, the errors in timing were not large enough to impact the results of the study. The procedure presented in this article offers an alternative method for performing a timing test, which uses readily available hardware and can be used to test the timing in any software program on any operating system and display.