Frequency of measurement and data utilization as factors in standardized behavioral assessment of academic skill

A study was conducted to investigate the effects on students' spelling achievement of variations in teacher assessment procedures. Teachers measured student spelling performance at a constant level of task difficulty using different measurement frequencies and different rules to interpret the data. Each teacher wrote two consecutive 3-week goals for improved spelling performance for two sets of 100 spelling words and then measured student performance either daily or weekly by dictating randomly selected words from each 100-word list. Teachers were trained to apply either a predetermined set of decision rules or their own judgment to the data to decide if the spelling program they had implemented for the student was effective. Ineffective programs were changed or modified. Results indicated that daily measurement was significantly more effective than weekly measurement in increasing spelling achievement and that, under certain conditions, decision rules were more effective than teacher judgment in determining when to make program changes or modifications.This research was conducted pursuant to Contract 300-77-0491 between the Bureau of Education for the Handicapped (now called Special Education Programs) and the University of Minnesota Institute for Research on Learning Disabilities.     

Real-time generation of representations for cognitive models

Computerized models of human cognition and behavior have been used to help researchers refine their theories of cognitive processes. Two difficulties inherent in using these cognitive models are that building representations of to-be-modeled groups or individuals is (1) time-consuming and (2) prone to error when performed manually. The use of computerized tools to build representations for cognitive models could alleviate these problems. Such a set of tools has been developed and applied to the task of modeling individual aviation pilots performing flight maneuvers in real time. A study conducted with 27 pilots confirms the accuracy of the computer-generated representations.     

Task rule-congruency and Simon-like effects in switching between spatial tasks

In task switching, a response indicated as correct by both task rules is executed more quickly than one for which the rules disagree. This rule-congruency, so far demonstrated unequivocally only in nonspatial tasks, shows that the currently irrelevant task set is kept active. However, in spatial task-switching, rule-congruency could potentially reflect a preexperimental tendency that contributes to a Simon-like effect. In the present study, participants switched between RIGHT-LEFT and UP-DOWN tasks with either a standard key arrangement (e.g., upper key = UP) or a mapping-reversed arrangement (e.g., up = DOWN), which reverses the direction of the potential Simon-like effect but leaves potential rule-congruency effects unchanged. Mapping-reversal did not modulate any other effect, including rule-congruency, and therefore indicated rule-congruency unequivocally. Finally, implications concerning generality versus domain specificity of control processes in task switching are discussed.     

Task switching in video game players: Benefits of selective attention but not resistance to proactive interference

Research into the perceptual and cognitive effects of playing video games is an area of increasing interest for many investigators. Over the past decade, expert video game players (VGPs) have been shown to display superior performance compared to non-video game players (nVGPs) on a range of visuospatial and attentional tasks. A benefit of video game expertise has recently been shown for task switching, suggesting that VGPs also have superior cognitive control abilities compared to nVGPs. In two experiments, we examined which aspects of task switching performance this VGP benefit may be localized to. With minimal trial-to-trial interference from minimally overlapping task set rules, VGPs demonstrated a task switching benefit compared to nVGPs. However, this benefit disappeared when proactive interference between tasks was increased, with substantial stimulus and response overlap in task set rules. We suggest that VGPs have no generalized benefit in task switching-related cognitive control processes compared to nVGPs, with switch cost reductions due instead to a specific benefit in controlling selective attention.     

CLASSI: A classification model for the study of sequential processes and individual differences therein

In psychological research, one often aims at explaining individual differences in S-R profiles, that is, individual differences in the responses (R) with which people react to specific stimuli (S). To this end, researchers often postulate an underlying sequential process, which boils down to the specification of a set of mediating variables (M) and the processes that link these mediating variables to the stimuli and responses under study. Obviously, a crucial task is to chart how the individual differences in the S-R profiles are caused by individual differences in the S-M link and/or by individual differences in the M-R link. In this paper we propose a new model, called CLASSI, which was explicitly designed for this task. In particular, the key principle of CLASSI consists of reducing the S, M, and R nodes of a sequential process to a few mutually exclusive types and inducing an S-M and an M-R person typology from the data, with the S-M person types being characterized in terms of if S type then M type rules and the M-R person types in terms of if M type then R type rules. As such, the S-M and M-R person types and their associated if–then rules represent the important individual differences in the S-M and M-R links of the sequential process under study. An algorithm to fit the CLASSI model is described and evaluated in a simulation study. An application of CLASSI to data from the behavioral domain of anger and sadness is discussed. Finally, we relate CLASSI to other methods and discuss possible extensions. The first author is a post-doctoral fellow of the Fund for Scientific Research—Flanders (Belgium). The research reported in this paper was partially supported by the Research Council of K.U. Leuven (GOA/05/04).     

Comparing sequential associations within a single dyad

We present a new application of sampled permutation testing to examine whether two sequential associations are different within a single dyad (e.g., a teacher and a student). A Monte Carlo simulation with the same (i.e., 100 vs. 100) or a different (100 vs. 400) number of event pairs was used to simulate designs that use time-based (typicallyproducing equal-length comparisons) and event-based (typically producing different-length comparisons) data, respectively. For these pairs of simulated data streams, we compared the Type I error rates and the kappa for agreement on significance decisions, using the sampled permutation tests and the more traditional asymptotic log linear analysis. The results provide the first evidence relevant to evaluating the accuracy of log linear analysis and sampled permutation testing for the purpose of comparing sequential associations within a single dyad.     

Parallel central processing between tasks: Evidence from lateralized readiness potentials

The present dual-task study used lateralized readiness potentials (LRPs) and behavioral measures to determine whether response activation for Task 1 and Task 2 can occur in parallel. We also examined whether task similarity (known as dimensional overlap) increases parallel central processing by making it difficult to selectively activate one task set. With dimensional overlap, the behavioral data replicated previous findings of backward correspondence effects: The Task 1 response was influenced by its compatibility with the Task 2 response. This finding suggests parallel response activation. The LRP data supported this conclusion: Task 2 response activation (indexed by the LRP) began before Task 1 central operations had finished. When there was no dimensional overlap, backward correspondence effects could not be measured, but the LRP data confirmed that parallel response activation still occurred. We argue that parallel response activation does occur, perhaps due to accidental activation of Task 2 mapping rules when the intention is to selectively execute Task 1 mapping rules.     

Use of evidence in a categorization task: analytic and holistic processing modes

Category learning performance can be influenced by many contextual factors, but the effects of these factors are not the same for all learners. The present study suggests that these differences can be due to the different ways evidence is used, according to two main basic modalities of processing information, analytically or holistically. In order to test the impact of the information provided, an inductive rule-based task was designed, in which feature salience and comparison informativeness between examples of two categories were manipulated during the learning phases, by introducing and progressively reducing some perceptual biases. To gather data on processing modalities, we devised the Active Feature Composition task, a production task that does not require classifying new items but reproducing them by combining features. At the end, an explicit rating task was performed, which entailed assessing the accuracy of a set of possible categorization rules. A combined analysis of the data collected with these two different tests enabled profiling participants in regard to the kind of processing modality, the structure of representations and the quality of categorial judgments. Results showed that despite the fact that the information provided was the same for all participants, those who adopted analytic processing better exploited evidence and performed more accurately, whereas with holistic processing categorization is perfectly possible but inaccurate. Finally, the cognitive implications of the proposed procedure, with regard to involved processes and representations, are discussed.     

A computer-oriented task analysis method for mathematics instruction

Components of a computer solution for fraction problems in arithmetic have an analog in a new approach to educational task analysis. The components may be called goal-setting or planning elements, in contrast to the goal-satisfying or behavioral steps emphasized in much contemporary task analysis. A hypothetical tutorial dialogue in which a student is asked to tell a tutor what step to perform next is presented as an example of the emphasis of the planning side of computation. Explicit identification of goal-setting elements of a task is illustrated by analysis of the task of converting a pair of fractions to a new pair with a common denominator. The decomposition thus produces is simulated with two computer programs: (1) a LISP program using a set of production rules consisting of conditions to be met and subsequent actions to be taken and (2) a PROLOG program stating goals and solving goals explicitly.     

A neuro-fuzzy network to generate human-understandable knowledge from data

Neuro-fuzzy networks have been successfully applied to extract knowledge from data in the form of fuzzy rules. However, one drawback with the neuro-fuzzy approach is that the fuzzy rules induced by the learning process are not necessarily understandable. The lack of readability is essentially due to the high dimensionality of the parameter space that leads to excessive flexibility in the modification of parameters during learning. In this paper, to obtain readable knowledge from data, we propose a new neuro-fuzzy model and its learning algorithm that works in a parameter space with reduced dimensionality. The dimensionality of the new parameter space is necessary and sufficient to generate human-understandable fuzzy rules, in the sense formally defined by a set of properties. The learning procedure is based on a gradient descent technique and the proposed model is general enough to be applied to other neuro-fuzzy architectures. Simulation studies on a benchmark and a real-life problem are carried out to embody the idea of the paper.     

Revisiting the effects of Parkinson's disease and frontal lobe lesions on task switching: The role of rule reconfiguration

This study investigated the hypothesis that rule reconfiguration in task switching can isolate aspects of intact and impaired control at different stages of Parkinson's disease (PD) by comparing switches between concrete naming rules pertaining to stimulus selection, to switches between abstract rules which allocate categorization responses to these stimuli. Based on previous findings, it was hypothesized that attentional switches, where task set competition emerges at the stimulus but not response set level, highlights striatal dopaminergic function. Conversely, increasing the degree of task set competition to encompass reconfiguration of response set when switching between abstract rules, represents a condition which engages the prefrontal cortex (PFC) and renders this manipulation sensitive to frontal damage. To this end, we investigated task switching with concrete and abstract rules in unilaterally (Hoehn & Yahr stage I) and bilaterally (Hoehn & Yahr stage II) affected PD patients, as well as striatally intact frontal lesion patients. As predicted, frontal lesion patients demonstrated switching deficits only with abstract categorization rules, where switching engendered complete task set reconfiguration and a new response, as did stage II PD patients with presumed frontal cortical pathology. Replicating previous findings, stage I PD patients with relatively circumscribed striatal pathology demonstrated no such impairment. Disease severity also impacted on attentional switching indexed by naming rules, since medicated stage II but not stage I patients demonstrated switching deficits emerging from stimulus set reconfiguration, suggesting that the ameliorative efficacy of dopaminergic medication is inversely related to the severity of the striatal deficit. These findings illustrate that the nature of the rules that are switched, and its implication in terms of reconfiguring different task set elements, highlights different neural characters of cognitive flexibility. These manipulations may help decipher the differential effects of progressive neurodegeneration on parkinsonian cognition, and provide a framework in which to conceptualize the contributions of cortical and subcortical regions to cognitive control.     

Development of decision making: sequential versus integrative rules

Decisions can be made by applying a variety of decision-making rules-sequential rules in which decisions are based on a sequential evaluation of choice dimensions and the integrative normative rule in which decisions are based on an integration of choice dimensions. In this study, we investigated the developmental trajectory of such decision-making rules. In the Gambling Machine Task, participants choose between options that differ in three dimensions: frequency of loss, amount of loss, and certain gain. The task was administered to 231 children and adolescents (age range=8-17 years). Latent group analysis of their performance allowed precise classification of the underlying decision rules. The majority of participants used sequential decision rules, and the number of evaluated dimensions in these rules increased with age. Integrative rule use decreased with age. We discuss these results in light of traditional developmental theories and fuzzy trace theory.     

A transitive-chain theory of syllogistic reasoning

A new theory of syllogistic reasoning, called the transitive-chain theory, is presented. The transitive-chain theory proposes that information about set relations is represented in memory by pairs of informational components. The theory further proposes that information about set relations is integrated by applying a small set of rules to transitive chains of these set relations. The rules that are applied to these chains are specified in detail. The theory is cast in terms of information-processing models for variants of the syllogistic reasoning task, and then mathematical models that quantify each of these information-processing models are presented. In a series of five experiments, the transitive-chain theory provides a good account of the response-choice data for syllogisms with various types of content, quantifiers, and logical relations (categorical and conditional). The results of these experiments offer tentative answers to four issues in the theory of syllogistic reasoning: (a) representation of premise information; (b) combination of premise information; (c) sources of difficulty in syllogistic reasoning; and (d) generality of the processes used in syllogistic reasoning.     

A mixed-effects expectancy-valence model for the Iowa gambling task

The Iowa gambling task (IGT; Bechara, Damasio, Damasio, & Anderson, 1994) was developed to simulate real-life decision making under uncertainty. The task has been widely used to examine possible neurocognitive deficits in normal and clinical populations. Busemeyer and Stout (2002) proposed the expectancy-valence (EV) model to explicitly account for individual participants’ repeated choices in the IGT. Parameters of the EV model presumably measure different psychological processes that underlie performance on the task, and their values may be used to differentiate individuals across different populations. In the present article, the EV model is extended to include both fixed effects and subject-specific random effects. The mixed-effects EV model fits the nested structure of observations in the IGT naturally and provides a unified procedure for parameter estimation and comparisons among groups of populations. We illustrate the utility of the mixed-effects approach with an analysis of gender differences using a real data set. A simulation study was conducted to verify the advantages of this approach.     

Multi-layer architecture for adaptive fuzzy inference system with a large number of input features

The Sugeno adaptive fuzzy neural network using training data is a good approximation to model different systems. The large number of adaptive neuro-fuzzy inference system (ANFIS) input features is a major challenge in using ANFIS and is not applicable with increased parameters. We present a solution for many input features solving modular problems; we created a multi-layer architecture of SUB-ANFIS (MLA-ANFIS) for this purpose. Different topologies were created with various combinations of multiple input features, and an error indicator was calculated for each combination of topologies. Finally, the best topology was chosen among the states with the highest possible performance. We implemented a multi-layered approach based on 365-day concrete compressive strength data with eight input features and the optimized MLA-ANFIS topology (5-3-1) for this purpose from different ANFIS topologies and neural networks. Finally, the results from five other datasets prove the impact of the proposed MLA-ANFIS approach compared to the neural network method.     

Concrete problems in the abstract deontic selection task - and how to solve them

The abstract deontic selection task was introduced by Cheng and Holyoak with the aim of demonstrating that people possess abstract reasoning schemas for processing deontic rules about what an individual must, must not, may, or need not do. Solving this task requires people to detect possible rule violators. The average solution rate across several studies, while being substantially higher than that with abstract nondeontic tasks, did not reach the level obtained with concrete deontic tasks. A task analysis based on the deontic principles by Beller uncovers several problems with the formulation of the original task. They concern the presentation of the deontic rule as well as the instructions (focusing on rule following) and result in a specific selection behaviour. Three experiments replicate the difficulties with the original task and show that task performance increases when the formulation problems are resolved. The best performance was obtained with a task that combined a genuine violation detection instruction with a genuine permission rule. Interestingly, permissions are weak deontic rules that, if taken literally, cannot be violated in a deontic sense. Therefore, people must interpret these rules as implying a strong deontic constraint (i.e., a ban), which then constitutes the basis for solving the task. The results provide novel insights into the interpretation of deontic rules and into the role that these content-specific, but abstract, tasks can play for the study of reasoning processes.     

TOWARD THE DEVELOPMENT OF STRUCTURED CRITERIA FOR INTERPRETATION OF FUNCTIONAL ANALYSIS DATA

Using functional analysis results to prescribe treatments is the preferred method for developing behavioral interventions. Little is known, however, about the reliability and validity of visual inspection for the interpretation of functional analysis data. The purpose of this investigation was to develop a set of structured criteria for visual inspection of multielement functional analyses that, when applied correctly, would increase interrater agreement and agreement with interpretations reached by expert consensus. In Study 1, 3 predoctoral interns interpreted functional analysis graphs, and interrater agreement was low (M = .46). In Study 2, 64 functional analysis graphs were interpreted by a panel of experts, and then a set of structured criteria were developed that yielded interpretive results similar to those of the panel (exact agreement = .94). In Study 3, the 3 predoctoral interns from Study 1 were trained to use the structured criteria, and the mean interrater agreement coefficient increased to .81. The results suggest that (a) the interpretation of functional analysis data may be less reliable than is generally assumed, (b) decision-making rules used by experts in the interpretation of functional analysis data can be operationalized, and (c) individuals can be trained to apply these rules accurately to increase interrater agreement. Potential uses of the criteria are discussed.     

Assessing Children's Homework Performance: Development of Multi-Dimensional, Multi-Informant Rating Scales

Efforts to develop interventions to improve homework performance have been impeded by limitations in the measurement of homework performance. This study was conducted to develop rating scales for assessing homework performance among students in elementary and middle school. Items on the scales were intended to assess student strengths as well as deficits in homework performance. The sample included 163 students attending two school districts in the Northeast. Parents completed the 36-item Homework Performance Questionnaire — Parent Scale (HPQ-PS). Teachers completed the 22-item teacher scale (HPQ-TS) for each student for whom the HPQ-PS had been completed. A common factor analysis with principal axis extraction and promax rotation was used to analyze the findings. The results of the factor analysis of the HPQ-PS revealed three salient and meaningful factors: student task orientation/efficiency, student competence, and teacher support. The factor analysis of the HPQ-TS uncovered two salient and substantive factors: student responsibility and student competence. The findings of this study suggest that the HPQ is a promising set of measures for assessing student homework functioning and contextual factors that may influence performance. Directions for future research are presented.     

Components of task switching: A closer look at task switching and cue switching

Research using the diffusion model to decompose task-switching effects has contributed to a better understanding of the processes underlying the observed effect in the explicit task cueing paradigm: Previous findings could be reconciled with multiple component models of task switching or with an account on compound-cue retrieval/repetition priming. In the present study, we used two cues for each task in order to decompose task-switch and cue-switch effects. Response time data support previous findings that comparable parts of the switching effect can be attributed to cue-switching and task-switching. A diffusion model analysis of the data confirmed that non-decision time is increased and drift rates are decreased in unpredicted task-switches. Importantly, it was shown that non-decision time was selectively increased in task-switching trials but not in cue-switching trials. Results of the present study specifically support the notion of additional processes in task-switches and can be reconciled with broader multiple component accounts.