General estimators for the reliability of qualitative data

We study a proportional reduction in loss (PRL) measure for the reliability of categorical data and consider the general case in which each ofN judges assigns a subject to one ofK categories. This measure has been shown to be equivalent to a measure proposed by Perreault and Leigh for a special case when there are two equally competent judges, and the correct category has a uniform prior distribution. We consider a general framework where the correct category is assumed to have an arbitrary prior distribution, and where classification probabilities vary by correct category, judge, and category of classification. In this setting, we consider PRL reliability measures based on two estimators of the correct category—the empirical Bayes estimator and an estimator based on the judges' consensus choice. We also discuss four important special cases of the general model and study several types of lower bounds for PRL reliability.Bruce Cooil is Associate Professor of Statistics, and Roland T. Rust is Professor and area head for Marketing, Owen Graduate School of Management, Vanderbilt University. The authors thank three anonymous reviewers and an Associate Editor for their helpful comments and suggestions. This work was supported in part by the Dean's Fund for Faculty Research of the Owen Graduate School of Management, Vanderbilt University.     

aPRIDIT Unsupervised Classification with Asymmetric Valuation of Variable Discriminatory Worth

Abstract

Sometimes one needs to classify individuals into groups, but there is no available grouping information due to social desirability bias in reporting behavior like unethical or dishonest intentions or unlawful actions. Assessing hard-to-detect behaviors is useful; however it is methodologically difficult because people are unlikely to self-disclose bad actions. This paper presents an unsupervised classification methodology utilizing ordinal categorical predictor variables. It allows for classification, individual respondent ranking, and grouping without access to a dependent group indicator variable. The methodology also measures predictor variable worth (for determining target behavior group membership) at a predictor variable category-by-category level, so different variable response categories can contain different amounts of information about classification. It is asymmetric in that a “0” on a binary predictor does not have a similar impact toward signaling “membership in the target group” as a “1” has for signaling “membership in the non-target group.” The methodology is illustrated by identifying Spanish consumers filing fraudulent insurance claims. A second illustration classifies Portuguese high school student’s propensity to alcohol abuse. Results show the methodology is useful when it is difficult to get dependent variable information, and is useful for deciding which predictor variables and categorical response options are most important.     

Recognition vs. classification of ill-defined category exemplars

In Experiment 1, subjects were trained on three ill-defined categories represented by 4, 8, and 12 exemplars. Learning was followed by either a classification test on old, new, and random exemplars or a recognition test in which each learning exemplar was presented with two foils, both equidistant to the learning exemplar, but one more similar to the category prototype than the other. In Experiment 2, category size was manipulated between subjects, followed by the recognition test. Recognition data indicated that increasing the number of learning exemplars did not increase the total amount of exemplar information retained for each category. In addition, the more prototypical foil was incorrectly recognized more often than the less prototypical foil. Classification data replicated previous findings that increasing category size in learning enhances classification of new exemplars. The results were interpreted in terms of the relative amounts of categorical and idiosyncratic information that were retained from the original learning stimuli. It was proposed that categorical information can become a more dominant determinant of performance as category size increases.     

Establishing derived categorical responding in children with disabilities using the PEAK‐E curriculum

The purpose of the study was to evaluate a procedure to generate derived categorical responding by three children with disabilities and to promote the emergence of untrained intraverbal categorical responses. In the study, three 4‐member equivalence classes including three stimuli (A, B, and C) and a category name (D) for each class were trained using a match‐to‐sample procedure. Test probes were conducted for categorical responding, including both a trained (D‐A) and two derived (D‐B, D‐C) relational responses, as well as the emergence of untrained intraverbal categorical responding (D‐A/B/C) throughout the study. Relational training was effective at promoting the emergence of categorical responding, and two of the three participants demonstrated the emergence of additional intraverbal responding without prior training. The results provide further evidence supporting the practical utility of stimulus equivalence as well as the PEAK‐E curriculum.     

Inter‐category versus intra‐category fit: when social categories match social context

The goal of the research reported in this article was to examine whether social categorization, commonly thought to be a function of accessibility and intra‐category fit, is also sensitive to changes in inter‐category fit. Intra‐category fit refers to the match between a target person's features and stored categorical knowledge, inter‐category fit to the extent to which category memberships and targets' features covary across perceived group members. In Experiments 1, 2, and 3, members of social categories of low, medium, and high accessibility, respectively, were shown in a group discussion. Inter‐category fit was manipulated in three steps. Category salience in memory increased as an additive function of accessibility and inter‐category fit. Experiment 4 replicated the effects of inter‐category fit while intra‐category fit and the information presented for individual discussants were held constant. The present studies are the first to demonstrate an effect of inter‐category fit on a relatively direct measure of spontaneous social categorization. Copyright © 2004 John Wiley & Sons, Ltd.     

Evaluating Structural Equation Models for Categorical Outcomes: A New Test Statistic and a Practical Challenge of Interpretation

This research is concerned with two topics in assessing model fit for categorical data analysis. The first topic involves the application of a limited-information overall test, introduced in the item response theory literature, to structural equation modeling (SEM) of categorical outcome variables. Most popular SEM test statistics assess how well the model reproduces estimated polychoric correlations. In contrast, limited-information test statistics assess how well the underlying categorical data are reproduced. Here, the recently introduced C₂ statistic of Cai and Monroe (2014) is applied. The second topic concerns how the root mean square error of approximation (RMSEA) fit index can be affected by the number of categories in the outcome variable. This relationship creates challenges for interpreting RMSEA. While the two topics initially appear unrelated, they may conveniently be studied in tandem since RMSEA is based on an overall test statistic, such as C₂. The results are illustrated with an empirical application to data from a large-scale educational survey.     

A framework for integrating dimensional and categorical classifications of personality disorder

Although empirical evidence strongly supports a dimensional representation of personality disorder, there is strong resistance to dimensional classification due in part to concerns about clinical utility. Acceptance of an evidence-based dimensional classification would be facilitated by information on how such a system would map onto existing diagnoses. With this objective in mind, an integrated framework is proposed that combines categorical and dimensional diagnoses. A two-component classification is adopted that distinguishes between the diagnosis of general personality disorder and the assessment of individual differences in the form the disorder takes. Then, the DSM definition of personality disorders is extended by defining individual disorders as categories of trait dimensions. This makes it possible to develop an integrated classification organized around a set of empirically derived primary traits. Assessments of these traits may then be combined to generate categorical and dimensional diagnoses. It is argued that this approach would introduce an etiological perspective into the classification of personality disorder and improve categorical classification by providing an explicit definition of each diagnosis. The clinical utility of incorporating a dimensional classification is discussed in terms of convenience and acceptability, value in predicting outcomes and treatment planning, and usefulness in organizing and selecting interventions.     

The world in black and white: Ostracism enhances the categorical perception of social information

In two experiments, ostracized individuals showed more pronounced categorical perception of inclusion- and exclusion-related stimuli. Specifically, ostracism enhanced the ability to distinguish between-category differences (e.g., between happy and angry faces) relative to within-category differences (e.g., between two happy expressions). Participants were socially included or excluded via Cyberball (a virtual ball-tossing task). In Experiment 1, ostracized participants showed greater perceptual acuity in distinguishing between subtly happy and angry expressions combined with a reduced ability to discriminate expressions within each expression category. Experiment 2 found analogous categorical perception effects for targets varying on the dimension of race. Importantly, this effect was specific to social information; categorical perception of non-social objects was not qualified by social exclusion. These results suggest that ostracism exacerbates categorical perception, attuning perceivers to the differences between various inclusion- and exclusion-related categories relative to within category acuity, making the world appear more ‘black-and-white’ than it might otherwise.     

The impact of category structure and training methodology on learning and generalizing within-category representations

When interacting with categories, representations focused on within-category relationships are often learned, but the conditions promoting within-category representations and their generalizability are unclear. We report the results of three experiments investigating the impact of category structure and training methodology on the learning and generalization of within-category representations (i.e., correlational structure). Participants were trained on either rule-based or information-integration structures using classification (Is the stimulus a member of Category A or Category B?), concept (e.g., Is the stimulus a member of Category A, Yes or No?), or inference (infer the missing component of the stimulus from a given category) and then tested on either an inference task (Experiments 1 and 2) or a classification task (Experiment 3). For the information-integration structure, within-category representations were consistently learned, could be generalized to novel stimuli, and could be generalized to support inference at test. For the rule-based structure, extended inference training resulted in generalization to novel stimuli (Experiment 2) and inference training resulted in generalization to classification (Experiment 3). These data help to clarify the conditions under which within-category representations can be learned. Moreover, these results make an important contribution in highlighting the impact of category structure and training methodology on the generalization of categorical knowledge.     

Adjusting risk of recidivism: do judicial departures worsen or improve recidivism prediction under the Federal Sentencing Guidelines?

A limited amount of research exists examining the ability of clinical or intuitive adjustments of formalistic methods of decision-making to improve upon predictive accuracy beyond that of the original measure. Using receiver operator characteristic (ROC) curves and the departure decisions of federal judges, the recidivism predictive utilities of two measures were compared. The two measures were the Criminal History category, based on the Federal Sentencing Guidelines (a formalistic procedure), and a re-coded measure of criminal history derived from the sentences actually imposed by judges after they departed from the Guidelines upwards or downwards for recidivism reasons (an adjustment to the formalistic procedure). Both a 10 year post-sentencing recidivism follow-up and a 6 year uniform follow-up period suggested that the Criminal History category performed poorly in predicting recidivism for this offender population, and that judicial departures not only failed to improve, but actually worsened, the predictive accuracy of pre-departure judgments. Policy implications of the findings are discussed.     

Learning and transfer of category knowledge in an indirect categorization task

Knowledge representations acquired during category learning experiments are ‘tuned’ to the task goal. A useful paradigm to study category representations is indirect category learning. In the present article, we propose a new indirect categorization task called the “same”–“different” categorization task. The same–different categorization task is a regular same–different task, but the question asked to the participants is about the stimulus category membership instead of stimulus identity. Experiment 1 explores the possibility of indirectly learning rule-based and information-integration category structures using the new paradigm. The results suggest that there is little learning about the category structures resulting from an indirect categorization task unless the categories can be separated by a one-dimensional rule. Experiment 2 explores whether a category representation learned indirectly can be used in a direct classification task (and vice versa). The results suggest that previous categorical knowledge acquired during a direct classification task can be expressed in the same–different categorization task only when the categories can be separated by a rule that is easily verbalized. Implications of these results for categorization research are discussed.     

A dissociation between two implicit conceptual tests supports the distinction between types of conceptual processing

Subjects studied words in a classification task (to what categories does it belong?) or in a production task (producing associates to the word) and then completed one of two implicit memory tests—category association or free association. The classification study task and the category-association test emphasize categorical relations between concepts, and the production study task and the free-association test draw on associative relations. As predicted by the transfer-appropriate processing principle, priming in the category-association test was larger for words studied under the classification task than for those studied under the production task, whereas the opposite was true in the free-association test. This crossover dissociation is the first obtained between two implicit conceptual tests, and it provides support for the claim that it is necessary to distinguish between different types of conceptual processing.     

Category-based induction: An effect of conclusion typicality

Category-based induction involves the willingness of a thinker to project some newly learned property of one or more classes of objects to another class on the basis of their shared membership in a common superordinate category. Previous research has established that the perceived strength of arguments of the form "Class A has Property P; therefore, Class B has Property P" is influenced by the similarity of A to B and by the typicality or representativeness of A in a shared category, superordinate to both A and B. (The nature of P is also crucial, but we do not examine it in this study.) There is, however, no prior evidence that the relation between B and the category is influential. Three experiments were designed to test whether the typicality of B in the superordinate category also has an effect on inductive argument strength. By using multiple regression (Experiment 1) and an experimental design (Experiment 3), an effect of conclusion typicality was found, so that people are more willing to project properties to more typical conclusions. Experiment 2 ruled out conclusion familiarity as a potential confounding variable. The results are interpreted in the light of current models of category-based induction.     

Dispersion in models of categorical perception

A signal detection theory model of auditory discrimination with a nonlinear mapping from stimulus continuum to perceptual continuum can account for the enhanced discrimination at the category boundary found in categorical perception. Properties of this transformation are specified by a unimodal “dispersion function”. Furthermore, it is shown that a system consisting of two acoustic feature detectors with an associated decision function is also a dispersive system, which models categorical perception of a stimulus continuum as well as boundary shifts under adaptation. The effect of detector adaptation on discrimination is discussed in view of three different types of decision variable and different types of detector noise.     

部分呈现条件下类别分类学习中的信息表征机制

先前研究者普遍认为, 类别推理学习条件下可以同时表征诊断性信息和非诊断性信息, 而类别分类学习条件下中只能表征诊断性信息, 不能表征非诊断性信息。而最近又有研究者发现部分呈现条件下的类别分类学习可以表征非诊断性信息。本研究通过两个实验系统比较了全部呈现和部分呈现条件下类别分类学习的结果, 进一步探讨了分类学习条件下信息的表征情况, 并进一步探讨了部分呈现条件下的分类学习能够表征非诊断性信息的原因。实验1发现全部呈现6个特征、缺失1个特征(即部分呈现5个特征)、缺失2个特征(即部分呈现4个特征)3种条件下都能表征诊断性信息, 但只有部分呈现条件下能表征非诊断性信息。实验2发现全部呈现7个特征、缺失2个特征(即部分呈现5个特征)、全部呈现5个特征3种条件下都能表征诊断性信息, 但只有部分呈现条件下能表征非诊断性信息。总的实验结果表明：全部呈现条件下的分类学习只能表征诊断性信息, 而部分呈现条件下的分类学习能够同时表征诊断性信息和非诊断性信息, 并且部分呈现条件下表征非诊断性信息的原因是被试进行了推理学习, 而非注意广度的变化。     

Influence of manipulated category knowledge on prototype classification and recognition

The recognition and classification of category members was explored, following a variable number of learning trials. In Experiment 1, subjects received 1 or 9 learning trials, followed by a recognition-then-classification test containing old, new, prototype, and foil patterns. In Experiment 2, subjects received 1, 6, or 12 trials, and made either classification or recognition judgments. In each experiment, classification accuracy for all item types was at near-chance -performance after a single trial but steadily increased with increased learning trials. On the transfer test, oldness judgments were highest for the category prototype after a single trial. However, with increased learning trials, oldness judgments increased for old instances and decreased for the category prototype and new instances. We suggest that false recognition of the category prototype, especially after a single learning trial, need not reflect an abstraction process. We discuss the possibility that an abstracted prototype may emerge with additional learning as an unfamiliar, ideal point.     

嵌套范式下视空工作记忆对基于规则类别学习

基于COVIS模型与认知加工阶段假设,通过2个实验探讨嵌套范式下, 视空工作记忆对基于规则类别学习的影响。实验1采用类别学习中嵌套视空工作记忆的范式,结果发现视空工作记忆削弱基于规则类别学习成绩,与COVIS模型预测相一致。实验2则采用视空工作记忆中嵌套类别学习任务的范式,结果却发现视空工作记忆对基于规则类别学习的影响消失。实验结果表明嵌套范式下视空工作记忆的位置影响基于规则类别学习,初步验证了类别学习存在多个认知加工阶段的假设,视空工作记忆主要影响基于规则类别学习中规则的发现和检验阶段。     

Learning about the internal structure of categories through classification and feature inference

Previous research on category learning has found that classification tasks produce representations that are skewed toward diagnostic feature dimensions, whereas feature inference tasks lead to richer representations of within-category structure. Yet, prior studies often measure category knowledge through tasks that involve identifying only the typical features of a category. This neglects an important aspect of a category's internal structure: how typical and atypical features are distributed within a category. The present experiments tested the hypothesis that inference learning results in richer knowledge of internal category structure than classification learning. We introduced several new measures to probe learners' representations of within-category structure. Experiment 1 found that participants in the inference condition learned and used a wider range of feature dimensions than classification learners. Classification learners, however, were more sensitive to the presence of atypical features within categories. Experiment 2 provided converging evidence that classification learners were more likely to incorporate atypical features into their representations. Inference learners were less likely to encode atypical category features, even in a “partial inference” condition that focused learners' attention on the feature dimensions relevant to classification. Overall, these results are contrary to the hypothesis that inference learning produces superior knowledge of within-category structure. Although inference learning promoted representations that included a broad range of category-typical features, classification learning promoted greater sensitivity to the distribution of typical and atypical features within categories.     

Categorical versus dimensional status of borderline personality disorder

One of the many controversies concerning the borderline personality disorder (BPD) diagnosis is whether the construct refers to a categorical or dimensional variable. The current study used Meehl's (1973) maximum covariance analysis to investigate this issue. The charts of 409 psychiatric inpatients were systematically reviewed for the presence of BPD and dysthymic symptoms. Charts of 244 inpatients were also reviewed to assess the presence of indicators of male sex, a categorical variable. The results for BPD and dysthymia were consistent with a dimensional model, whereas those for male sex were consistent with a categorical model. A dimensional model of classification of BPD is recommended, and suggestions for future research are provided.     

The effects of search criteria and retrieval cue availability on memory for words

The effects of search criteria upon recall performance following a word classification task were investigated. Rival hypotheses were tested, one based on the number of cues available at retrieval and the other on the amount of processing carried out during classification. After being presented with two-component category names, subjects classified words on the basis of either category membership or the number of semantic components the words shared with a target category, and then they received an unexpected recall test. Experiment 1 favored an amount-of-processing explanation, but it was felt that redundancy in the stimulus materials might invalidate this conclusion. Experiment 2 confirmed this suspicion and clearly supported an explanation in terms of retrieval cue availability. The notion of partial semantic processing of words is discussed, and it is argued that the number of congruent encodings formed during classification can be used as an index of elaboration and subsequent memory performance.