Empirical research on the identification of singular,multiple and subordinate argumentation

To examine to what degree argumentation skills are mastered by pupils who attend the vocational, general and academic streams in Dutch secondary education various subtests were constructed. The theoretical study of argumentation as exposed by Van Eemeren and Grootendorst was the basis for this test construction. In this article tests for the identification of singular, multiple and subordinate argumentation are described. Also an account is given of a pretest of these three subtests.     

Child and adolescent services research at the national institute of mental health: Research opportunities in an emerging field

This article describes an important new area of research on services for children and adolescents with mental disorders at the National Institute of Mental Health, the parameters of mental health services research for youth, and the opportunities that are available for grant-funded investigations in this area.     

Linking qualitative and quantitative methods in cross-cultural survey research: Techniques from cognitive science

Survey research with diverse cultural and ethnic minority groups is a complex and challenging endeavor that confronts the researcher with problems related to linguistic and conceptual equivalence and measurement as well as problems related to difficulties that respondents have with the sociocultural dimensions of the survey/interview process. One way to improve the quality of cross-cultural surveys and to insure that the findings are culturally relevant and accurate is to combine qualitative and quantitative methods. This paper proposes that certain qualitative techniques from cognitive science, specifically cognitive anthropology and cognitive psychology, are particularly well suited to being combined with survey research. These techniques provide information corresponding to the underlying thought processes of respondents and enable researchers to better understand how different cultural and ethnic groups construe the world. The information obtained can be used improve the formulation of survey questions, design and structure questionnaire formats to coincide with the way particular groups organize concepts, and help researchers understand difficulties respondents may have with the survey/interview process. In addition, the techniques produce data that are easily codifiable and more manageable than traditional qualitative techniques including participant observation and in-depth interviews.     

On the sampling theory roundations of item response theory models

Item response theory (IT) models are now in common use for the analysis of dichotomous item responses. This paper examines the sampling theory foundations for statistical inference in these models. The discussion includes: some history on the stochastic subject versus the random sampling interpretations of the probability in IRT models; the relationship between three versions of maximum likelihood estimation for IRT models; estimating versus estimating -predictors; IRT models and loglinear models; the identifiability of IRT models; and the role of robustness and Bayesian statistics from the sampling theory perspective.A presidential address can serve many different functions. This one is a report of investigations I started at least ten years ago to understand what IRT was all about. It is a decidedly one-sided view, but I hope it stimulates controversy and further research. I have profited from discussions of this material with many people including: Brian Junker, Charles Lewis, Nicholas Longford, Robert Mislevy, Ivo Molenaar, Donald Rock, Donald Rubin, Lynne Steinberg, Martha Stocking, William Stout, Dorothy Thayer, David Thissen, Wim van der Linden, Howard Wainer, and Marilyn Wingersky. Of course, none of them is responsible for any errors or misstatements in this paper. The research was supported in part by the Cognitive Science Program, Office of Naval Research under Contract No. Nooo14-87-K-0730 and by the Program Statistics Research Project of Educational Testing Service.     

Continuous and discrete latent structure models for item response data

Relations are examined between latent trait and latent class models for item response data. Conditions are given for the two-latent class and two-parameter normal ogive models to agree, and relations between their item parameters are presented. Generalizationss are then made to continuous models with more than one latent trait and discrete models with more than two latent classes, and methods are presented for relating latent class models to factor models for dichotomized variables. Results are illustrated using data from the Law School Admission Test, previously analyzed by several authors.     

Brief report: Complex postformal thought in skilled research administrators

Expert research administrators responded to questions to determine to what degree they had postformal thinking skills. Postformal theory suggests that the interpersonal and institutional complexity of work roles would make them likely to exhibit postformal thought. The tests included a questionnaire about the use of postformal operations on the job, and thinking-aloud taped interviews of responses to job-related and standard postformal problems. The respondents demonstrated the use of postformal thinking operations on job-related tests and a more moderate degree of use on the standard problems. The results support prior research and our working hypothesis that a high degree of interaction with people and complex problems is associated with adults' postformal thought.     

The use of the occupational stress indicator (OSI) in factory workers in China

The purpose of the study was to investigate occupational stress among Chinese factory workers (N=342), from three cities of South East China, using the shortened version of the Occupational Stress Indicator (OSI)-2. The results showed that the reliabilities and predictive validity of the OSI-2 subscales and other subscales used for the study were reasonably high. Quite a high percentage of workers perceived high work pressure. The main sources of stress were intrinsic to the job; and the coping strategies that were most frequently used to tackle stress were “control” methods. In addition, “satisfaction on environmental condition” seemed to be a common predictor for job satisfaction, and mental and physical well-being. The logical relationships between job satisfaction, mental well-being and physical well-being in Chinese workers have provided support to the findings obtained in Western countries.     

Use and misuse of metrics in research evaluation

This paper addresses some critical issues in the applicability of quantitative performance measures (including bibliometric, economic, and co-occurrence measures) to the assessment of basic research. The strengths and weaknesses of metrics applied as research performance measures are examined. It is concluded that metrics have a useful role to play in the evaluation of research. Each metric employed, whether bibliometric, economic, co-occurrence, or others, brings a new dimension of potential insight to the complex problem of research assessment. However, when used in a stand-alone mode, metrics can be easily misinterpreted and result in misleading conclusions. Metrics should be an integral part of a more comprehensive approach to research evaluation, in which the leading role is assumed by expert peer review. Under this structure, the reviewers must be broadly constituted, so that the technical correctness of the research approaches being taken (doing the job right) can be evaluated by technical experts and the correctness of the research target (doing the right job) can be evaluated as well. This comprehensive approach insures quality in the implementation of existing research paradigms and allows the introduction of revolutionary new research paradigms as well. The views contained in this paper are solely those of the author and do not represent the views of the Department of the Navy. Ronald Kostoff received a Ph.D. from Princeton University, performed technical and economic research at Bell Laboratories, managed energy programs at U.S. Department of Energy, was Director of Technical Assessment at the Office of Naval Research for many years and presently manages the Navy’s In-House Laboratory Independent Research Program.