On the measurement of priming: What is the correct baseline?

Abstract

We argue, contrary to Ostergaard (this issue), that the baseline for measuring priming effects in “recognition/identification” experiments is correctly estimated by the subjects’ performance with non-primed items. There is no reason to assume, as Ostergaard (this issue) does, that priming that occurs in a single presentation of the target item is qualitatively different from priming that occurs, in the same experimental situation, in two or more presentations, or that empirical findings, such as stochastic independence between recognition and primed identification, are affected by the number of presentations of the target items at study.     

Statistical Tests of Conditional Independence Between Responses and/or Response Times on Test Items

Three plausible assumptions of conditional independence in a hierarchical model for responses and response times on test items are identified. For each of the assumptions, a Lagrange multiplier test of the null hypothesis of conditional independence against a parametric alternative is derived. The tests have closed-form statistics that are easy to calculate from the standard estimates of the person parameters in the model. In addition, simple closed-form estimators of the parameters under the alternatives of conditional dependence are presented, which can be used to explore model modification. The tests were applied to a data set from a large-scale computerized exam and showed excellent power to detect even minor violations of conditional independence.     

Checking the Assumptions of Rasch's Model for Speed Tests

Two new tests for a model for the response times on pure speed tests by Rasch (1960) are proposed. The model is based on the assumption that the test response times are approximately gamma distributed, with known index parameters and unknown rate parameters. The rate parameters are decomposed in a subject ability parameter and a test difficulty parameter. By treating the ability as a gamma distributed random variable, maximum marginal likelihood (MML) estimators for the test difficulty parameters and the parameters of the ability distribution are easily derived. Also the model tests proposed here pertain to the framework of MML. Two tests or modification indices are proposed. The first one is focused on the assumption of local stochastic independence, the second one on the assumption of the test characteristic functions. The tests are based on Lagrange multiplier statistics, and can therefore be computed using the parameter estimates under the null model. Therefore, model violations for all items and pairs of items can be assessed as a by-product of one single estimation run. Power studies and applications to real data are included as numerical examples.     

A Unified Nonparametric IRT Model for <Emphasis Type="Italic">d</Emphasis>-Dimensional Psychological Test Data (<Emphasis Type="Italic">d</Emphasis>-ISOP)

The (univariate) isotonic psychometric (ISOP) model (Scheiblechner, 1995) is a nonparametric IRT model for dichotomous and polytomous (rating scale) psychological test data. A weak subject independence axiom W1 postulates that the subjects are ordered in the same way except for ties (i.e., similarly or isotonically) by all items of a psychological test. A weak item independence axiom W2 postulates that the order of the items is similar for all subjects. Local independence (LI or W3) is assumed in all models. With these axioms, sample-free unidimensional ordinal measurements of items and subjects become feasible. A cancellation axiom (Co) gives, as a result, the additive isotonic psychometric (ADISOP) model and interval scales for subjects and items, and an independence axiom (W4) gives the completely additive isotonic psychometric (CADISOP) model with an interval scale for the response variable (Scheiblechner, 1999). The d-ISOP, d-ADISOP, and d-CADISOP models are generalizations to d-dimensional dependent variables (e.g., speed and accuracy of response). The author would like to thank an Associate Editor and two anonymous referees and also Professor H.H. Schulze for their very valuable suggestions and corrections.     

A Multicomponent Latent Trait Model for Diagnosis

This paper presents a noncompensatory latent trait model, the multicomponent latent trait model for diagnosis (MLTM-D), for cognitive diagnosis. In MLTM-D, a hierarchical relationship between components and attributes is specified to be applicable to permit diagnosis at two levels. MLTM-D is a generalization of the multicomponent latent trait model (MLTM; Whitely in Psychometrika, 45:479–494, 1980; Embretson in Psychometrika, 49:175–186, 1984) to be applicable to measures of broad traits, such as achievement tests, in which component structure varies between items. Conditions for model identification are described and marginal maximum likelihood estimators are presented, along with simulation data to demonstrate parameter recovery. To illustrate how MLTM-D can be used for diagnosis, an application to a large-scale test of mathematics achievement is presented. An advantage of MLTM-D for diagnosis is that it may be more applicable to large-scale assessments with more heterogeneous items than are latent class models.     

Estimation of careless error and lucky guess probabilities for dichotomous test items: A psychometric application of a biometric latent class model with random effects

Medical research has extensively dealt with the estimation of the accuracy (sensitivity and specificity) of a diagnostic test for screening individuals. In this paper we apply the biometric latent class model with random effects by Qu, Tan, and Kutner [(1996). Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. Biometrics, 52, 797-810] to estimate the response error (careless error and lucky guess) probabilities for dichotomous test items in the psychometric theory of knowledge spaces. The approach is illustrated with simulated data. In particular, we extend this approach to give a generalization of the basic local independence model in knowledge space theory. This allows for local dependence among the indicators given the knowledge state of an examinee and/or for the incorporation of covariates.     

Relationship between creativity and right,left, and integrated thinking styles

ABSTRACT:

The purpose of the present study was to explore whether the so‐called “right thinking style” is related to creativity as measured by two types of creativity tests. The How Do You Think (HDYT) inventory evaluates personality and biographical information related to creativity, and the Sounds and Images is a type of divergent thinking test that assesses the originality of images elicited by abstract sounds. As a second purpose, this study permitted an analysis of the relationship between these two types of creativity measures. The two creativity measures and the Your Style of Learning and Thinking (SOLAT) test were administered to 109 undergraduates. Results indicated that the HDYT scores were positively correlated with right thinking scores on the SOLAT (r = .48) and negatively correlated with left thinking scores (r = ‐.70). Results from Sounds and Images were not as clear. Similar relationships were found when overlapping items were removed from the various measures.     

A method for judging measures of stochastic dependence: further comments on the current controversy.

A large number of reports have been published on stochastic independence between implicit and explicit measures of memory. This is often taken to imply that different memory systems mediate implicit and explicit memory performance. In these cases, stochastic independence is inferred from contingency analysis of overall success rates in two memory tasks when performance in one or both of the tasks is, to a large extent, mediated by factors other than memory. Typically, the difference between performance with studied and nonstudied items is not large in implicit memory tasks. It is argued that this must be taken into account when evaluating the contingency analysis. A method is presented for estimating the relevant joint and conditional probabilities, assuming that the aspects of performance in the two tasks that are related to memory are dependent to the maximum possible extent. The method is applied to a number of published studies, and it is shown that the difference between these estimated probabilities and those given by stochastic independence is too small to allow any conclusion to be drawn about memory systems from contingency analysis of data reported in these studies.     

Two new test statistics for the rasch model

It is shown that the presently available statistical tests for the Rasch model are insensitive to violation of the unidimensionality axiom. Two new test statistics are presented. The first one,Q ₁, is sensitive to the same effects as the presently available statistics, but has some desirable properties of a nonstatistical nature. The second statistic,Q ₂, is sensitive to violation of local stochastic independence and unidimensionality and thus fills an existing gap.     

The adolescent friendship structure inventory (AFSI): A review and empirical consolidation of existing measures

Abstract

The aim of this study was to empirically consolidate previous friendship measures in order to identify a set of items to include in a tool for assessing positive and negative qualities of adolescents’ friendships. Existing self-report measures were identified and 91 items were selected from the measures. Following a pilot study undertaken to reduce the number of items for use in a larger study, the primary study included 415 Australian adolescents (249 boys and 166 girls, aged 12–17 years) who completed a questionnaire at school. The questionnaire included a reduced set of 47 of the 91 items. Exploratory factor analysis was conducted with 50% of the sample (n = 208). In the final factor analysis, 26 items loaded highly on one of four factors. The factors were labeled emotional support, positive interactions, coercive power, and conditional social regard. The factor structure was confirmed with the other 50% of the sample (n = 207). This confirmatory factor model had a good fit to the data and the four subscales on the new Adolescent Friendship Structure Inventory (AFSI) had sound inter-item correlations. In addition, the AFSI subscales were validated against a measure of friendship satisfaction, with all four subscales associated with satisfaction in the expected direction.     

Bayesian Procedures for Identifying Aberrant Response-Time Patterns in Adaptive Testing

In order to identify aberrant response-time patterns on educational and psychological tests, it is important to be able to separate the speed at which the test taker operates from the time the items require. A lognormal model for response times with this feature was used to derive a Bayesian procedure for detecting aberrant response times. Besides, a combination of the response-time model with a regular response model in an hierarchical framework was used in an alternative procedure for the detection of aberrant response times, in which collateral information on the test takers’ speed is derived from their response vectors. The procedures are illustrated using a data set for the Graduate Management Admission Test^® (GMAT^®). In addition, a power study was conducted using simulated cheating behavior on an adaptive test.     

Item placement on a personality measure: effects on faking behavior and test measurement properties

Although personality tests are widely used to select applicants for a variety of jobs, there is concern that such measures are fakable. One procedure used to minimize faking has been to disguise the true intent of personality tests by randomizing items such that items measuring similar constructs are dispersed throughout the test. In this study, we examined if item placement does influence the fakability and psychometric properties of a personality measure. Study participants responded to 1 of 2 formats (random vs. grouped items) of a personality test honestly and also under instructions to fake or to behave like an applicant. Results indicate that the grouped item placement format was more fakable for the Neuroticism and Conscientiousness scales. The test with items randomly placed fit the data better within the honest and applicant conditions. These findings demonstrate that the issue of item placement should be seriously considered before administering personality measures because different item presentations may affect the incidence of faking and the psychometric properties of the measure.     

Autonomie en transition de carrière. Développement d’un modèle et validation d’une échelle originale

IntroductionBased on the observation that autonomy is a fundamental concept in a world of work characterized by many transitions, this paper presents the construction and validation of a model and a scale of autonomy in career transitions.ObjectiveThe scale was developed to measure autonomy during adult career transitions.MethodA series of items was first produced following interviews with experts and employees. Three successive surveys involving 336, 314 and 450 participants made it possible, through item analysis and factor analysis, to reduce the number of items and to build a model composed of 14 items and 4 oblique factors.ResultsAfter confirmatory factor analysis, the data fit the model well and a study of external validity shows strong convergences with other measures of autonomy.ConclusionThe model and the measure of autonomy in career transition can be of great help in supporting employees. It makes it possible to identify the people who are most in difficulty and offers benchmarks to support and train organizations to modulate career paths and strengthen the acquisition of certain resources.     

Evidence against Rank-Dependent Utility Theories: Tests of Cumulative Independence,Interval Independence,Stochastic Dominance,and Transitivity

This study tests between two modern theories of decision making. Rank- and sign-dependent utility (RSDU) models, including cumulative prospect theory (CPT), imply stochastic dominance and two cumulative independence conditions. Configural weight models, with parameters estimated in previous research, predict systematic violations of these properties for certain choices. Experimental data systematically violate all three properties, contrary to RSDU but consistent with configural weight models. This study also tests whether violations of stochastic dominance can be explained by violations of transitivity. Violations of transitivity may be evidence of a dominance detecting mechanism. Although some transitivity violations were observed, most choice triads violated stochastic dominance without violating transitivity. Judged differences between gambles were not consistent with the CPT model. Data were not consistent with the editing principles of cancellation and combination. The main findings are interpreted in terms of coalescing, the principle that equal outcomes can be combined in a gamble by adding their probabilities. RSDU models imply coalescing but configural weight models violate it, allowing configural weighting to explain violations of stochastic dominance and cumulative independence.     

A Longitudinal Investigation of Perceived Control and Cognitive Performance in Young,Midlife and Older Adults

ABSTRACT

Longitudinal associations between generalized control beliefs (one's perceived capacity to influence events) and cognitive test performance were examined in a population-based sample of young, midlife and older adults. Participants provided measures of perceived control, self-assessed health, education and depression and anxiety symptoms, and completed cognitive tests at two assessments, 4 years apart. For each age group, baseline (between-person) control was positively related to performance on tests of memory (immediate recall and digits backwards), speed (Symbol Digit Modalities Test and choice reaction time) and verbal intelligence (Spot-the-Word). Interaction effects indicated stronger associations of between-person control beliefs with indices of speed for the older age group relative to the younger groups. Within-person changes in control were not significantly associated with changes in cognitive test performance over the study interval. Implications of the findings for self-efficacy based interventions designed to promote cognitive functioning are discussed.     

Ordinal consistency and ordinal true scores

This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.Part of this work was done while the author was a Visiting Fellow at Macquarrie University. The paper has benefitted from discussions with Professors Thomas J. Reynolds and Roderick P. McDonald and from the comments of several anonymous reviewers.     

Resting autonomic activity in adults who stutter and its association with self-reports of social anxiety

PurposeThe purpose of this study was to investigate resting autonomic activity in adults who stutter (AWS) compared to adults who do not stutter (ANS) and the relationship this has on self-reports of social anxiety.MethodsThirteen AWS and 15 ANS completed the Social Interaction Anxiety Scale (SIAS; Mattick & Clark, 1998) and Brief Fear of Negative Evaluation (BFNE; Leary, 1983). Following this, measures of skin conductance levels (i.e. index of sympathetic activity) and respiratory sinus arrhythmia (i.e. index of parasympathetic activity) were taken during a 5-minute resting, baseline period. Independent sample t tests were used to assess differences between groups on self-reports of anxiety (SIAS, BFNE) and resting autonomic levels (SCL, RSA). Separate multiple regression analyses were performed in order to assess the relationship between self-reports of anxiety and autonomic measures.ResultsResults showed significantly higher mean SCL and lower mean RSA levels in the AWS compared to the ANS at resting, baseline. Regression analysis showed that self-reports from the SIAS had a significant effect on RSA levels for the AWS but not the ANS. No significant effects were found for BFNE on RSA. Nor was there a significant effect from SIAS or BFNE on SCL levels for either group.ConclusionFindings suggest that resting RSA levels may be a physiological marker for social anxiety levels in adults who stutter.