Statistical classification of black-capped (Poecile atricapillus) and mountain chickadee (Poecile gambeli) call notes

Both black-capped (Poecile atricapillus) and mountain chickadees (Poecile gambeli) produce a chick-a-dee call that consists of several distinct note types. In some regions, these 2 species live sympatrically, and it has been shown that 1 species will respond weakly to songs of the other. This suggests that chickadee song, and potentially other of their vocalizations, contains species-specific information. We tested the possibility that call notes were acoustically sufficient for species identification. Black-capped and mountain non-D notes were summarized as a set of 9 features and then analyzed by linear discriminant analysis. Linear discriminant analysis was able to use these notes to identify species with 100% accuracy. We repeated this approach, but with black-capped and mountain D notes that were summarized as a set of 4 features. Linear discriminant analysis was able to use these notes to identify species with 94% accuracy. This demonstrates that any of the note types in these chickadee calls possesses sufficient information for species classification.     

Do complex models increase prediction of complex behaviours? Predicting driving ability in people with brain disorders

Prediction of complex behavioural tasks via relatively simple modelling techniques, such as logistic regression and discriminant analysis, often has limited success. We hypothesized that to more accurately model complex behaviour, more complex models, such as kernel-based methods, would be needed. To test this hypothesis, we assessed the value of six modelling approaches for predicting driving ability based on performance on computerized sensory-motor and cognitive tests (SMCTests?) in 501 people with brain disorders. The models included three models previously used to predict driving ability (discriminant analysis, DA; binary logistic regression, BLR; and nonlinear causal resource analysis, NCRA) and three kernel methods (support vector machine, SVM; product kernel density, PK; and kernel product density, KP). At the classification level, two kernel methods were substantially more accurate at classifying on-road pass or fail (SVM 99.6%, PK 99.8%) than the other models (DA 76%, BLR 78%, NCRA 74%, KP 81%). However, accuracy decreased substantially for all of the kernel models when cross-validation techniques were used to estimate prediction of on-road pass or fail in an independent referral group (SVM 73-76%, PK 72-73%, KP 71-72%) but decreased only slightly for DA (74-75%) and BLR (75-76%). Cross-validation of NCRA was not possible. In conclusion, while kernel-based models are successful at modelling complex data at a classification level, this is likely to be due to overfitting of the data, which does not lead to an improvement in accuracy in independent data over and above the accuracy of other less complex modelling techniques.     

Do complex models increase prediction of complex behaviours? Predicting driving ability in people with brain disorders

Prediction of complex behavioural tasks via relatively simple modelling techniques, such as logistic regression and discriminant analysis, often has limited success. We hypothesized that to more accurately model complex behaviour, more complex models, such as kernel-based methods, would be needed. To test this hypothesis, we assessed the value of six modelling approaches for predicting driving ability based on performance on computerized sensory–motor and cognitive tests (SMCTests?) in 501 people with brain disorders. The models included three models previously used to predict driving ability (discriminant analysis, DA; binary logistic regression, BLR; and nonlinear causal resource analysis, NCRA) and three kernel methods (support vector machine, SVM; product kernel density, PK; and kernel product density, KP). At the classification level, two kernel methods were substantially more accurate at classifying on-road pass or fail (SVM 99.6%, PK 99.8%) than the other models (DA 76%, BLR 78%, NCRA 74%, KP 81%). However, accuracy decreased substantially for all of the kernel models when cross-validation techniques were used to estimate prediction of on-road pass or fail in an independent referral group (SVM 73–76%, PK 72–73%, KP 71–72%) but decreased only slightly for DA (74–75%) and BLR (75–76%). Cross-validation of NCRA was not possible. In conclusion, while kernel-based models are successful at modelling complex data at a classification level, this is likely to be due to overfitting of the data, which does not lead to an improvement in accuracy in independent data over and above the accuracy of other less complex modelling techniques.     

Alternative Techniques for Predicting Success in Air Controller School

The Armed Services Vocational Aptitude Battery (ASVAB) is the principal cognitive test battery used for military classification-that is, for assignment of individuals to specific job categories. Successful entrance into a specific job category requires satisfactory completion of relevant training. Therefore, it is necessary that the ASVAB accurately predict the likelihood that individuals will complete training. Presently, the U.S. Navy classifies enlisted personnel based on a unit-weighted linear combination of scores derived from a subset of tests currently composing the ASVAB test battery. This research explored the development of new classification models estimated by using linear regression, logistic regression, discriminant analysis, and artificial neural networks, and it compared their classification accuracy with models developed using unit- weighted regression. Models were estimated and cross-validated using data from individuals admitted into the Navy's Air Controlman training during fiscal years 1988 and 1989. Accuracy was measured by the proportion of individuals correctly classified as graduates or as academic failures and by the ability to rank students-that is, to predict the highest scores for graduates and the lowest scores for attrites.     

Skew and internal consistency

The effects of skew on the standardized item alpha were examined with Monte Carlo techniques. Alphas computed from normal variables were compared with alphas from lognormal variables, ranks, and skewed versus normal Likert-type variables. The extent and direction of skew were varied, as was the size of the population interitem correlation (rho), the number of items, and the number of categories for Likert-type variables. Because the average interitem correlation affects alpha and skew affects the average interitem correlation, the effect of skew on the average interitem correlation also was examined. Results indicated that skew decreased the average interitem correlation and produced small decreases in alpha that were largest when skew was large, rho was small, items were skewed in opposite directions, and there were fewer items.     

Optimal Scaling of Interaction Effects in Generalized Linear Models

This study involved two phases: first, when classification was based on the calibration sample; and second, in a cross-validation setting. Computer generated data were used. Results obtained from rules based on probabilities of group membership were compared for accuracy when classifying in the discriminant space and in the predictor variable spaces. In the first phase accuracy was greater in the predictor variable spaces, while the reverse was true in the second phase. In general, rules based on probabilities of group membership were approximately equally accurate and more accurate than a rule related to a multiple regression analysis. Other findings are also discussed.     

Alternative Lens Model Equations for Dichotomous Judgments about Dichotomous Criteria

The Brunswik lens model typically represents a judge's accuracy using parameters derived from linear regression. This is not optimal if the judgment or the ecological criterion is dichotomous. Alternative approaches, modeling dichotomies using logistic regression, or linearizing judgments with confidence ratings, have not been compared with the same data. Four techniques for deriving lens model equation parameters were compared: (i) linear and (ii) logistic regression applied to dichotomous patient outcomes and judgments; (iii) linear regression with confidence‐adjusted judgments but dichotomous patient outcomes; and (iv) a hybrid with a linear model of the confidence‐adjusted judgments and a logistic model of the patient outcomes. Judgment accuracy (r_a) was slightly higher with confidence adjustment of the categorical judgments. The logistic lens model accounted for a higher proportion of r_a than the linear lens model; the confident‐linear and hybrid lens models were intermediate. For up to a quarter of participants, different methods identified different cues as most important. Display condition differences in achievement r_a and in lens model components are similar with all lens model methods. Each of the three alternative lens model equation methods improves on the linear lens model equation's decomposition of the accuracy of dichotomous judgments. Confidence adjustment improves achievement although it requires additional work from the subjects. The logistic lens model equation explains the highest proportion of achievement, but with a small stimulus set, it is more vulnerable to cue intercorrelations than either the linear or the confident linear lens model equation. Copyright © 2016 John Wiley & Sons, Ltd.     

分类视角下认知诊断测验项目区分度指标及应用

在认知诊断中还没有指标能在无作答数据情况下直接评价项目的属性分类准确率或属性判准率。项目水平上的属性分类准确率,与项目属性向量、项目参数、先验分布和作答反应等有关。综合各个影响因素定义了项目水平上的属性期望分类准确率指标,并将其用于组卷。模拟研究显示：新指标可十分准确地评价项目的属性判准率,新指标对于项目筛选十分重要;以模式分类准确率为评价指标,基于新指标的组卷方法与经典的组卷方法表现相当。     

Kernel principal component analysis combining rotation forest method for linearly inseparable data

Rotation forest (RoF) is an ensemble classifier combining linear analysis theories and decision tree algorithms. In recent existing works, RoF was widely applied to various fields with outstanding performance compared to traditional machine learning techniques, given that a reasonable number of base classifiers is provided. However, the conventional RoF algorithm suffers from classifying linearly inseparable datasets. In this study, a hybrid algorithm integrating kernel principal component analysis (KPCA) and the conventional RoF algorithm is proposed to overcome the classification difficulty for linearly inseparable datasets. The radial basis function (RBF) is selected as the kernel for the KPCA method to establish the nonlinear mapping for linearly inseparable data. Moreover, we evaluate various kernel parameters for better performance. Experimental results show that our algorithm improves the performance of RoF with linearly inseparable datasets, and therefore provides higher classification accuracy rates compared with other ensemble machine learning methods.     

Maximum information stratification method for controlling item exposure in computerized adaptive testing

The proposal for increasing the security in Computerized Adaptive Tests that has received most attention in recent years is the a-stratified method (AS - Chang and Ying, 1999): at the beginning of the test only items with low discrimination parameters ( a ) can be administered, with the values of the a parameters increasing as the test goes on. With this method, distribution of the exposure rates of the items is less skewed, while efficiency is maintained in trait-level estimation. The pseudo-guessing parameter ( c ), present in the three-parameter logistic model, is considered irrelevant, and is not used in the AS method. The Maximum Information Stratified (MIS) model incorporates the c parameter in the stratification of the bank and in the item-selection rule, improving accuracy by comparison with the AS, for item banks with a and b parameters correlated and uncorrelated. For both kinds of banks, the blocking b methods (Chang, Qian and Ying, 2001) improve the security of the item bank.     

Do logistic regression and signal detection identify different subgroups at risk? Implications for the design of tailored interventions

Identifying subgroups of high-risk individuals can lead to the development of tailored interventions for those subgroups. This study compared two multivariate statistical methods (logistic regression and signal detection) and evaluated their ability to identify subgroups at risk. The methods identified similar risk predictors and had similar predictive accuracy in exploratory and validation samples. However, the 2 methods did not classify individuals into the same subgroups. Within subgroups, logistic regression identified individuals that were homogeneous in outcome but heterogeneous in risk predictors. In contrast, signal detection identified individuals that were homogeneous in both outcome and risk predictors. Because of the ability to identify homogeneous subgroups, signal detection may be more useful than logistic regression for designing distinct tailored interventions for subgroups of high-risk individuals.     

四参数Logistic模型和传统模型对被试作答拟合能力的比较研究

针对测验中高能力被试答错容易试题的睡眠现象,可使用四参数Logistic模型分析数据。研究选取了来自心理测验和成就测验的实际数据,分别采用传统模型和四参数Logistic模型进行拟合,对不同模型的拟合指标及参数估计结果进行比较。结果表明,四参数Logistic模型能够提高拟合程度,增强估计结果的准确性,有效纠正高能力被试能力被低估的现象。建议在必要时使用四参数Logistic模型进行数据分析。     

Clinical and statistical approaches to the assessment of brain damage in children

In an attempt at innovation in the clinical versus statistical prediction controversy, the present research examined three methods of assessing organic brain damage in a sample of sixty-eight 9- to 14-year-old children. Methods compared were clinical judgment (CP); existing actuarial norms (EAN); and derived statistical prediction (DSP), based on three psychological tests. The results were as follows: (a) None of the methods classified subjects significantly better than chance (p<.05); (b) there was no significant difference (p<.05) among the methods in terms of their rates of accurate classification; (c) CP based on combined tests did not result in higher predictive accuracy than those based on an individual test; (d) CP made after receiving feedback on accuracy showed an increase in number of correct classifications over prior CP; and (e) CP based on knowledge of the statistical prediction in addition to test data showed a higher rate of correct classification than DSP alone.     

Content Analysis: A Comparison of Manual and Computerized Systems

Computerized content analysis is a technique for classifying texts. in personality research it has beer used to make inferences about psychological states and traits of the speaker or writer. Whereas computerized techniques have been increasing in both sophistication and ease of use, questions remain regarding their validity compared to hand-scored scales, in this study, we compare a relatively simple computerized technique; a more sophisticated, context-sensitive computerized approach; and a context-sensitive, human-scored system. Seventy-one subjects, representing four diagnostic groups, provided free-speech samples which were scored according to all three methods. Data from each method were then submitted to discriminant and classification analyses. In comparing the classification performance of the three methods, we found that both computerized methods were more accurate than the human-scored method. The time and cost benefits of computerized systems are discussed.     

assessing the effectiveness of statistical classification techniques in predicting future employment of participants in the temporary assistance for needy families program

Three classification techniques (Chi‐square Automatic Interaction Detection [CHAID], Classification and Regression Tree [CART], and discriminant analysis) were tested to determine their accuracy in predicting Temporary Assistance for Needy Families program recipients' future employment. Technique evaluation was based on proportion of correctly predicted cases and 6 other criteria. Sustained employment was associated with both recipients' subjective work‐related skills assessments and their objective test scores, and with their educational levels and marital status. Correct case classification by model was 74% for CHAID, 75% for CART, and 68% for discriminant analysis. Results suggest these classification tools can be useful supplements in assessments of clients' employment potentials.     

Feature utilization of continuously varying attributes in visual pattern classification

Feature utilization of continuously varying attributes was examined in the context of classification of random polygons, the collection of which contained no a priori classes based upon physical features. Ten Os sorted 60 eight-sided patterns into two through nine groups. One week later, each O repeated three sorts and then placed 30 new patterns into the groups of his previous five-group sort. Comparison of groups across sessions primarily showed repetition of initial classification. Stepwise discriminant analyses were performed on Os’ classes, predictors being physical measures of various pattern attributes. Significant predictors of all classifications were found, the two most prevalent predictor types being measures of compactness and jaggedness. LDFs based on original pattern classes showed     

Content analysis: a comparison of manual and computerized systems

Computerized content analysis is a technique for classifying texts. In personality research it has been used to make inferences about psychological states and traits of the speaker or writer. Whereas computerized techniques have been increasing in both sophistication and ease of use, questions remain regarding their validity compared to hand-scored scales. In this study, we compare a relatively simple computerized technique; a more sophisticated, context-sensitive computerized approach; and a context-sensitive, human-scored system. Seventy-one subjects, representing four diagnostic groups, provided free-speech samples which were scored according to all three methods. Data from each method were then submitted to discriminant and classification analyses. In comparing the classification performance of the three methods, we found that both computerized methods were more accurate than the human-scored method. The time and cost benefits of computerized systems are discussed.     

Psychosocial and organic variables as predictors of lung cancer,cardiac infarct and apoplexy: Some differential predictors

In a longitudinal study started in 1965 in Crvenka (Yugoslavia), a number of psychosocial and organic variables were measured in 1353 Ss and related to the incidence of cancer, circulatory and other diseases in the subsequent 10-yr period by multivariate statistical methods. The present article reports on an attempt at differential prediction of lung cancer and cardiac infarct or apoplexy, which were selected because smoking is usually held to be a common risk factor for them. In a discriminant analysis (minimizing the overall classification error), 86–87% of the lung cancer and infarct/apoplexy cases could be predicted correctly, as well as 97% of the cases showing neither of the diseases. Some psychosocial variables like rationality and anti-emotionality or long-lasting hopelessness were about as relevant as the strongest organic predictors like level and variability of blood cholesterol.     

Base-rate neglect as a function of base rates in probabilistic contingency learning

When humans predict criterion events based on probabilistic predictors, they often lend excessive weight to the predictor and insufficient weight to the base rate of the criterion event. In an operant analysis, using a matching-to-sample paradigm, Goodie and Fantino (1996) showed that humans exhibit base-rate neglect when predictors are associated with criterion events through physical similarity. In partial replications of their studies, we demonstrated similar effects when the predictors resembled the criterion events in terms of similarly skewed base rates. Participants' predictions were biased toward the more (or less) frequent criterion event following the more (or less) frequent predictor. This finding adds to the growing evidence for pseudocontingencies (Fiedler & Freytag, 2004), a framework that stresses base-rate influences on contingency learning.     

Sex differences in achievement/career motivation of Iranian boys and girls

The relationships of early socialization, socioeconomic status (SES), religion, community discrimination, and community resources (all as possible predictor variables) to achievement/career motivation (the criterion variables) for 200 Iranian high school senior girls and boys were studied. Multiple regression analyses were performed to identify how much variance was accounted for by each predictor variable. In addition, a discriminant function analysis based on four linear combinations of the predictors for high and low SES, and for males and females, was performed to identify the best and strongest predictors. Through content analysis of the projective protocols representing achievement motivation, no significant achieving imagery was found for these Iranian subjects. The measure did, however, provoke imagery indicative of the subjects' preoccupation with marriage and/or affiliation with the opposite sex. Consistent with research findings in the United States, Iranian females scored lower in career motivation and expressed less belief in sex discriminatory myths than did males. The upper SES subjects of both sexes showed a higher level of career aspiration, and perceived more community resources for their careers, than did the lower SES groups. Sex, SES, community discrimination, and heterosexual interest contributed more heavily than the other predictors in predicting the variance of career motivation. Possible implications of the results for improving Iranian women's achievement/career motivation are discussed. A need for a more cross-culturally appropriate measure of achievement motivation is suggested.This study was supported, in part, by the University of Illinois Center for International and Comparative Studies. The author is indebted to Drs. Martin L. Maehr and Helen S. Farmer of the University of Illinois for their direction and advice in conducting this study.