Raters who pursue different goals give different ratings

J. N. Cleveland and K. R. Murphy (1992) suggested that phenomena such as rater errors and interrater disagreements could be understood in terms of differences in the goals pursued by various raters. We measured 19 rating goals of students at the beginning of a semester, grouped them into scales, and correlated these with teacher evaluations collected at the end of the semester. We found significant multiple correlations, both within classes and in an analysis of the pooled sample (adjusting for instructor mean differences, incremental R2 =.08). Measures of rating goals obtained after raters had observed a significant proportion of ratee performance accounted for variance (incremental R2 =.07) not accounted for by measures of goals obtained at the beginning of the semester.     

Faking personality questionnaires: Fabricating different profiles for different purposes

Over fifty subjects completed four personality measures used in personnel selection. Based on a latin-square design they were asked to fill them in as they would if they were trying to present themselves as ideal candidates for the job of librarian, advertising executive or banker, while on one questionnaire they gave “honest,” actual responses. The results indicated, as previous research in this area has shown, that the questionnaires are all highly susceptible to faking, and that a quite different prototypic profile arose for each of the three different jobs. The results are discussed in terms of the templates of fakers for specific jobs; the methods of detecting fakers, and what dissimulation studies tell us about theories of both occupations and personality.     

Modeling item responses when different subjects employ different solution strategies

A model is presented for item responses when different subjects employ different strategies, but only responses, not choice of strategy, can be observed. Using substantive theory to differentiate the likelihoods of response vectors under a fixed set of strategies, we model response probabilities in terms of item parameters for each strategy, proportions of subjects employing each strategy, and distributions of subject proficiency within strategies. The probabilities that an individual subject employed the various strategies can then be obtained, along with a conditional estimate of proficiency under each. A conceptual example discusses response strategies for spatial rotation tasks, and a numerican example resolves a population of subjects into subpopulations of valid responders and random guessers.The first author's work was supported by Contract No. N00014-85-K-0683, project designation NR 150-539, from the Cognitive Science Program, Cognitive and Neural Sciences Division, Office of Naval Research. We are grateful to Murray Aitkin, Isaac Bejar, Neil Dorans, Frederiksen, and Marklyn Wingersky for their comments and suggestions, and to Alison Gooding, Maxine Kingston, Donna Lembeck, Joling Liang, and Kentaro Yamamoto for their assistance with Example 2.     

Faking personality questionnaires: Fabricating different profiles for different purposes

Over fifty subjects completed four personality measures used in personnel selection. Based on a latin-square design they were asked to fill them in as they would if they were trying to present themselves as ideal candidates for the job of librarian, advertising executive or banker, while on one questionnaire they gave “honest,” actual responses. The results indicated, as previous research in this area has shown, that the questionnaires are all highly susceptible to faking, and that a quite different prototypic profile arose for each of the three different jobs. The results are discussed in terms of the templates of fakers for specific jobs; the methods of detecting fakers, and what dissimulation studies tell us about theories of both occupations and personality.     

Diagnosing children's writing disabilities: different tests give different results

Writing problems are common in children with clinical disorders. However, there are significant intra-individual differences between the ability to spell words, construct sentences, and compose text. Therefore, achievement tests measuring different writing skills may not be consistent in identifying children who have these various writing disabilities. Our study compared scores on the Woodcock-Johnson Written Language subtests (which measure the ability to produce single words and single sentences) with scores on the Wechsler Individual Achievement Test Written Expression subtest (which assesses compositional writing skills) in 54 children referred by their teachers to a school psychologist for writing problems. The Woodcock-Johnson only identified 35% of children as having significant writing problems, whereas the Wechsler test identified 78%. Our study suggests that the latter is more likely than the Woodcock-Johnson to identify students who have problems in compositional writing.     

Expecting different categories at different locations and spatial selective attention

An experiment is reported testing the idea that spatial selective attention operates like the beam of a spotlight. Subjects made speeded orientation judgements to alphanumeric characters that could appear at an upper or lower display location. Letters tended (p = 0.8) to occur at one location, while digits tended to occur at the other. Alphanumeric category was pre-cued (p = 0.8). Response times were faster for cued than uncued items. For cued items subjects were faster at the likely location for a cued item. For uncued items this was reversed, with faster responding at the likely location for an uncued item. Hence, location selectivity reflected location probability for each category regardless of short-term expectancy concerning stimulus category. These data are incompatible with a simple spotlight view of spatial selective attention. Alternative explanations, including a modified two-stage spotlight model, were considered.     

Component analysis with different sets of constraints on different dimensions

Many of the classical multivariate data analysis and multidimensional scaling techniques call for approximations by lower dimensional configurations. A model is proposed, in which different sets of linear constraints are imposed on different dimensions in component analysis and classical multidimensional scaling frameworks. A simple, efficient, and monotonically convergent algorithm is presented for fitting the model to the data by least squares. The basic algorithm is extended to cover across-dimension constraints imposed in addition to the dimensionwise constraints, and to the case of a symmetric data matrix. Examples are given to demonstrate the use of the method.The work reported in this paper has been supported by the Natural Sciences and Engineering Research Council of Canada, grant number A6394, and by the McGill-IBM Cooperative Grant, both granted to the first author. The research of H. A. L. Kiers has been made possible by a fellowship of the Royal Netherlands Academy of Arts and Sciences. We thank Michael Hunter for his helpful comments on earlier drafts of this paper.     

Aristotle's different possibilities

Marxism is often claimed to be incompatible with any kind of ethical theory, because of its assumptions of economic determinism, of the class character of morals, and of the subordination of morality to politics. But the author proposes that these assumptions can be interpreted in such a flexible way as not to rule out the freedom of choice and responsibility, die relative independence of morals from economic conditions and political ends, and concepts of universal human value and a specifically moral ideal. A humanist philosophy, centered in Marx's analysis of alienation, provides a sufficiently rich theoretical basis for the solution of both ethical and meta‐ethical’ problems.     

Behavioral contrast in rats with different reinforcers and different response topographies

Experiment I demonstrated positive behavioral contrast in rats when one of two qualitatively different reinforcers (milk and pellets) was removed from a component of a multiple schedule. The contrast effect was larger and more enduring when milk was removed. Experiment II showed that the rats spent more time on the side of a shuttle-box on which milk was freely available than on the side on which pellets were freely available. Experiment III, a partial replication of Experiment I, failed to demonstrate the contrast effect of Experiment I. Experiment IV demonstrated contrast when two topographically distinct responses, nose-key poking and lever pressing, were required in different components of a multiple schedule. These results extend the conditions that generate behavioral contrast in rats.     

Proprioception plays a different role for sensorimotor adaptation to different distortions

If proprioceptive feedback is degraded by agonist-antagonist muscle vibration, then adaptation to rotated vision remains intact while adaptation to a velocity-dependent force field worsens. Here we evaluate whether this differential effect of vibration is related to the physical nature of the distortion - visual versus mechanical - or to their kinematic coupling to the subjects’ hand - velocity versus position dependent. Subjects adapted to a velocity-dependent visual distortion, to a position-dependent force, or to a velocity-dependent force; one half of the subjects adapted with, and the other half without agonist-antagonist vibration at the wrist, elbow, and shoulder. We found, as before, that vibration slowed down adaptation to a velocity-dependent force. However, vibration did not modify adaptation to the other two distortions, nor did it influence the aftereffects of any distortion. From this we conclude that intact proprioception supports strategic compensatory processes when proprioceptive signals agree with visual ones, and provide relevant (dynamic) information not available to the visual system.     

Becoming literate in different languages: similar problems, different solutions

The teaching of reading in different languages should be informed by an effective evidence base. Although most children will eventually become competent, indeed skilled, readers of their languages, the pre-reading (e.g. phonological awareness) and language skills that they bring to school may differ in systematic ways for different language environments. A thorough understanding of potential differences is required if literacy teaching is to be optimized in different languages. Here we propose a theoretical framework based on a psycholinguistic grain size approach to guide the collection of evidence in different countries. We argue that the development of reading depends on children's phonological awareness in all languages studied to date. However, we propose that because languages vary in the consistency with which phonology is represented in orthography, there are developmental differences in the grain size of lexical representations, and accompanying differences in developmental reading strategies across orthographies.     

Different components of working memory have different relationships with different mathematical skills

A comprehensive working memory battery and tests of mathematical skills were administered to 90 children-41 in Year 1 (5-6 years of age) and 49 in Year 3 (7-8 years of age). Working memory could explain statistically significant variance in number writing, magnitude judgment, and single-digit arithmetic, but the different components of working memory had different relationships with the different skills. Visual-spatial sketchpad (VSSP) functioning predicted unique variance in magnitude judgments and number writing. Central executive functioning explained unique variance in the addition accuracy of Year 1 children. The unique variance explained in Year 3 multiplication explained by phonological loop functioning just missed conventional levels of significance (p=.06). The results are consistent with the VSSP having a role in the development of number writing and magnitude judgments but a lesser role in early arithmetic.     

Observer recorder error as affected by different tasks and different expectancy inducements

Four experiments were designed to investigate the effects of systematic observer/recorder error on experimental results where experimenters had been induced to expect certain results. In the first experiment, experimenters were induced to expect certain outcomes for three different tasks. No observer error nor experimenter-effects were observed. Postexperimental interviews indicated some suspiciousness as well as failure to understand the hypotheses on the part of experimenters. In the second experiment expectancies were induced in observers who did not act as experimenters. No observer effects were noted, and again postexperimental interviews indicated manipulation failure. In the third experiment experimenters were asked to attempt to influence subjects' responses. Three tasks were used, but in two of these tasks observer error was controlled rather than measured. No differences in subjects' responses were noted for any of the measures. However, significant observer/recorder error was noted for both male and female experimenters in the task where such error was measurable. In the fourth experiment experimenters were again asked to influence subjects' responses. For the two tasks used observer error was measured. A predicted Task × Observer Error Interaction was obtained. Again, no differences in subjects' responses were noted.     

Students' conceptions of constructivist learning in different programme years and different learning environments

Background Constructivist views of learning have brought conceptions of learning to attention again. Conceptions are considered important determinants of effective learning. Students can differ in their conceptions depending on their educational experience. Aims The present study investigated students' conceptions of constructivist learning. Do students with greater experience in their academic programme differ in their conceptions of constructivist learning compared to students with less experience/ In addition, to what extent are conceptions of constructivist learning different in a conventional, lecture‐based curriculum compared to a constructivist, problem‐based learning curriculum/ Samples Three groups (i.e. first‐year, second‐year, and third‐year students) in two different curricula (i.e. conventional, lecture‐based and constructivist, problem‐based) were tested. Methods A cross‐sectional design was used. Students' conceptions of constructivist activities (i.e. knowledge construction, cooperative learning, self‐regulation, use of authentic problems, self‐perceived inability to learn, and motivation to learn) were measured by a questionnaire. Data were analyzed using a two‐way multivariate analysis of variance (MANOVA). Results A significant difference in questionnaire's scores between year 1 and year 2 (but not between year 2 and 3) was found with respect to conceptions about knowledge construction, self‐regulation, and the use of authentic problems, but not for cooperative learning and motivation to learn. For self‐perceived inability, an interaction effect was found. Furthermore, results showed significant differences between both curriculum groups on all dependent measures. Conclusions Differences in conceptions can be perceived between students who enter a new learning programme (i.e. higher education) and students who already have one year of experience in higher education. Among students with more than one year of educational experience, differences disappear. Furthermore, this study shows that the learning environment can make a difference with respect to students' conceptions of constructivist learning activities.     

Structural equation modeling of multitrait-multimethod data: different models for different types of methods

The question as to which structural equation model should be selected when multitrait-multimethod (MTMM) data are analyzed is of interest to many researchers. In the past, attempts to find a well-fitting model have often been data-driven and highly arbitrary. In the present article, the authors argue that the measurement design (type of methods used) should guide the choice of the statistical model to analyze the data. In this respect, the authors distinguish between (a) interchangeable methods, (b) structurally different methods, and (c) the combination of both kinds of methods. The authors present an appropriate model for each type of method. All models allow separating measurement error from trait influences and trait-specific method effects. With respect to interchangeable methods, a multilevel confirmatory factor model is presented. For structurally different methods, the correlated trait-correlated (method-1) model is recommended. Finally, the authors demonstrate how to appropriately analyze data from MTMM designs that simultaneously use interchangeable and structurally different methods. All models are applied to empirical data to illustrate their proper use. Some implications and guidelines for modeling MTMM data are discussed.