Action unit classification using active appearance models and conditional random fields

In this paper, we investigate to what extent modern computer vision and machine learning techniques can assist social psychology research by automatically recognizing facial expressions. To this end, we develop a system that automatically recognizes the action units defined in the facial action coding system (FACS). The system uses a sophisticated deformable template, which is known as the active appearance model, to model the appearance of faces. The model is used to identify the location of facial feature points, as well as to extract features from the face that are indicative of the action unit states. The detection of the presence of action units is performed by a time series classification model, the linear-chain conditional random field. We evaluate the performance of our system in experiments on a large data set of videos with posed and natural facial expressions. In the experiments, we compare the action units detected by our approach with annotations made by human FACS annotators. Our results show that the agreement between the system and human FACS annotators is higher than 90% and underlines the potential of modern computer vision and machine learning techniques to social psychology research. We conclude with some suggestions on how systems like ours can play an important role in research on social signals.     

Mixtures of conditional mean- and covariance-structure models

Models and parameters of finite mixtures of multivariate normal densities conditional on regressor variables are specified and estimated. We consider mixtures of multivariate normals where the expected value for each component depends on possibly nonnormal regressor variables. The expected values and covariance matrices of the mixture components are parameterized using conditional mean- and covariance-structures. We discuss the construction of the likelihood function, estimation of the mixture model with regressors using three different EM algorithms, estimation of the asymptotic covariance matrix of parameters and testing for the number of mixture components. In addition to simulation studies, data on food preferences are analyzed.The authors are grateful to Donald B. Rubin and Michael E. Sobel for critical reading of a first draft of this paper and to three anonymous reviewers ofPsychometrika for their helpful comments and suggestions. The research of the first and the third author was supported by a grant from the Deutsche Forschungsgemeinschaft.     

Reasoning with three types of conditional: Biases and mental models

Two experiments are reported which compare conditional reasoning with three types of rule. These consist of two types of rule that have been widely studied previously, if p then q and p only if q, together with a third type, q if p. In both experiments, the p only if q type of rule yields a different pattern of performance from the two other types of rule. Experiment 1 is an abstract rule-evaluation task and demonstrates differential effects of temporal order and of suppositional bias. Experiment 2 investigates rule generation, rephrasing, and comparison, and demonstrates differential effects of temporal order and of thematic content. An analysis of the results is offered in terms of biases and mental models. Effects of rule form and context can be explained as reflecting the different sequences in which mental models are created for each rule form. However, it is necessary to consider the internal structure of individual mental models to account for effects arising from temporal ordering of rules.     

Correspondence analysis and association models constrained by a conditional independence graph

The manner in which the conditional independence graph of a multiway contingency table effects the fitting and interpretation of the Goodman association model (RC) and of correspondence analysis (CA) is considered.Estimation of the row and column scores is presented in this context by developing a unified framework that includes both models. Incorporation of the conditional independence constraints inherent in the graph may lead to equal or additive scores for the corresponding marginal tables, depending on the topology of the graph. An example of doubly additive scores in the analysis of a Burt subtable is given.Thanks are due to anonymous referees who substantially improved the original draft of this paper.     

Interaction of dominant focus and conditional reflex as a functional unit of behavior organization

The interaction of two fundamental phenomena—the dominant focus and the conditional reflex—discovered and introduced by A. A. Ukhtomsky and I. P. Pavlov lay at the basis of behavior. According to E. A. Asratyan, the backward conditioned connection is a specialized dominant focus in the functional structure of the consolidated conditional reflex. It makes the behavior goal-directed and active. The dominant focus and conditioned reflex play the same role in the adaptive behavior of the individual as does variability and selection in the process of evolutional adaptation. That is why it is impossible to agree with Popper and Eccles that hypothesis theory has to replace Pavlov’s theory of the conditional reflex. Imprinting and psychonervous activity by images (I. S. Beritashvili) are two special exemplars of conditional reflexes after one coincidence. The so-called “elementary reasoning activity of animals” (according to L. V. Krushinsky) is a kind of the instinctive inherited behavior.     

Causal graphical views of fixed effects and random effects models

Despite the long-standing discussion on fixed effects (FE) and random effects (RE) models, how and under what conditions both methods can eliminate unmeasured confounding bias has not yet been widely understood in practice. Using a simple pretest–posttest design in a linear setting, this paper translates the conventional algebraic formalization of FE and RE models into causal graphs and provides intuitively accessible graphical explanations about their data-generating and bias-removing processes. The proposed causal graphs highlight that FE and RE models consider different data-generating models. RE models presume a data-generating model that is identical to a randomized controlled trial, while FE models allow for unobserved time-invariant treatment–outcome confounding. Augmenting regular causal graphs that describe data-generating processes by adding the computational structures of FE and RE estimators, the paper visualizes how FE estimators (gain score and deviation score estimators) and RE estimators (quasi-deviation score estimators) offset unmeasured confounding bias. In contrast to standard regression or matching estimators that reduce confounding bias by blocking non-causal paths via conditioning, FE and RE estimators offset confounding bias by deliberately creating new non-causal paths and associations of opposite sign. Though FE and RE estimators are similar in their bias-offsetting mechanisms, the augmented graphs reveal their subtle differences that can result in different biases in observational studies.     

Signal recognition models compared for random and Markov presentation sequences

Models of two-category signal recognition are compared to data from a variety of experimental conditions. For recognition, one of two signals (S₁, S₂) which vary slightly on some simple physical dimension is presented on each trial, and 0 is to identify (I₁, I₂ ) which signal was presented. In general, Os show a decrease in both Pr(I₁ /S₂) and Pr(I₂/S₁) for either greater Pr(S₂) or, sequentially, for greater S₂ recency. These effects are described as probability and sequential contrast, respectively. The memory state model (MS) describes a three-state threshold process with responses determined by a simple first-order Markov process which depends on the sensory state and response on the immediately preceding trial. The memory trace regression model (MTR) assumes that O compares the observed signal event with the memory trace of the previous signal event. When the difference is large, the sign of the difference determines the response; when it is small, the response depends on the preceding response. The memory trace is assumed to regress toward the mean signal value. Both models accurately predict the observed bias changes as a function of signal probabilities and of the subsequence of events on the previous trial. The response axioms of the MTR model are modified to predict the results for individual Os for Markov chains of signal events. The response axioms of the MS model are modified to predict responding when information is given to O concerning signal probabilities. Although both models do well under all conditions, the MS model uses fewer parameters and correctly predicts the direction of higher order sequential dependencies.     

Comparing exemplar-retrieval and decision-bound models of speeded perceptual classification

The authors compared the exemplar-based random-walk (EBRW) model of Nosofsky and Palmeri (1997) and the decision-bound model (DBM) of Ashby and Maddox (1994; Maddox & Ashby, 1996) on their ability to predict performance in Garner’s (1974) speeded classification tasks. A key question was the extent to which the models could predict facilitation in the correlated task and interference in the filtering task, in situations involving integral-dimension stimuli. To obtain rigorous constraints for model evaluation, the goal was to fit the detailed structure of the response time (RT) distribution data associated with each individual stimulus in each task. Both models yielded reasonably good global quantitative fits to the RT distribution and accuracy data. However, the DBM failed to properly characterize the interference effects in the filtering task. Apparently, a fundamental limitation of the DBM is that it predicts that the fastest RTs in the filtering task should be faster than the fastest RTs in the control task, whereas the opposite pattern was observed in our data.     

Mixed models,random utilities,and the triangle inequality

A new probabilistic binary choice model, which we call the mixed model is proposed. This model is used to give an interpretation of probabilistic choice behavior in terms of complete and partial orders. The model is used as a guide to disprove Jacob Marschak's conjecture that the binary random utility model is equivalent to the triangle inequality. Another result is presented concerning the relation between moderate stochastic transitivity and the triangle inequality.     

Fixed and random address models for storing order in short-term memory

Two models have been proposed for storing order in short-term memory. Conrad's (1965) fixed address model assumes that items are kept in a set of fixed addresses, while Crossman's (1961) random address model assumes that items are tagged for order, and assigned to a set of random addresses. Experiment I tested the models in a task where six items were presented visually and either watched silently or shadowed. Memory for the position of one of them was tested with a probe. Neither model fitted the data, and it was suggested that order might pass through two stages in memory, the first being a buffer store with properties like the fixed address model, and the second being a longer-term store which behaves like a set of random addresses. Simulations of a combined fixed and random address model produced results like those of Experiment I, and in Experiment II response latencies were used to separate the buffer and longer term store stages. Responses with short latencies produced results like those from a buffer with fixed addresses, while responses with long latencies conformed more to the predictions of a random address model.     

Exemplars, prototypes, and the flexibility of classification models

J. P. Minda and J. D. Smith (2001) showed that a prototype model outperforms an exemplar model, especially in larger categories or categories that contained more complex stimuli. R. M. Nosofsky and S. R. Zaki (2002) showed that an exemplar model with a response-scaling mechanism outperforms a prototype model. The authors of the current study investigated whether excessive model flexibility could explain these results. Using cross-validation, the authors demonstrated that both the prototype model and the exemplar model with a response-scaling mechanism suffered from overfilling in the linearly separable category structure. The results illustrate the need to make sure that the best-fitting model is not chasing error variance instead of variance attributed to the cognitive process it is supposed to model.     

Classifying the variability in impact and active peak vertical ground reaction forces during running using DFA and ARFIMA models

The vertical ground reaction force (VGRF) during rear-foot striking running typically exhibits peaks referred to as the impact peak and the active peak; their timings and magnitudes have been implicated in injury. Identifying the structure of time-series can provide insight into associated control processes. The purpose here was to detect long-range correlations associated with the time from first contact to impact peak (TIP) and active peak (TAP); and the magnitudes of impact (IPM) and active peaks (APM) using a Detrended Fluctuation Analysis, and Auto-Regressive Fractionally Integrated Moving Average models. Twelve subjects performed an 8 min trial at their preferred running speed on an instrumented treadmill. TIP, TAP; IPM, and APM were identified from the VGRF profile for each footfall. TIP and TAP time-series did not demonstrate long-range correlations, conversely IPM and APM time-series did. Short range correlations appeared as well as or instead of long range correlations for IPM. Conversely pure powerlaw behaviour was demonstrated in 11 of the 24 time series for APM, and long range dependencies along with short range correlations were present in a further 9 time series. It has been hypothesised that control mechanisms for IPM and APM are different, these results support this hypothesis.     

Twice random,once mixed: Applying mixed models to simultaneously analyze random effects of language and participants

Psychologists, psycholinguists, and other researchers using language stimuli have been struggling for more than 30 years with the problem of how to analyze experimental data that contain two crossed random effects (items and participants). The classical analysis of variance does not apply; alternatives have been proposed but have failed to catch on, and a statistically unsatisfactory procedure of using two approximations (known as F ₁ and F ₂) has become the standard. A simple and elegant solution using mixed model analysis has been available for 15 years, and recent improvements in statistical software have made mixed models analysis widely available. The aim of this article is to increase the use of mixed models by giving a concise practical introduction and by giving clear directions for undertaking the analysis in the most popular statistical packages. The article also introduces the djmixed add-on package for SPSS, which makes entering the models and reporting their results as straightforward as possible.     

Additive and multiplicative models for gamma distributed random variables,and their application as psychometric models for response times

A class of models for gamma distributed random variables is presented. These models are shown to be more flexible than the classical linear models with respect to the structure that can be imposed on the expected value. In particular, both additive, multiplicative, and combined additive-multiplicative models can be formulated. As a special case, a class of psychometric models for reaction times is presented, together with their psychological interpretation. By means of a comparison with existing models, this class of models is shown to offer some possibilities that are not available in existing methods. Parameter estimation by means of maximum likelihood (ML) is shown to have some attractive properties, since the models belong to the exponential family. Then, the results of a simulation study of the bias in the ML estimates are presented. Finally, the application of these models is illustrated by an analysis of the data from a mental rotation experiment. This analysis is preceded by an evaluation of the appropriateness of the gamma distribution for these data.     

The online processing of only if and even if conditional statements: Implications for mental models

A sentential connective like only if or even if merges two simple propositions into a complex statement. This study used a visual world paradigm experiment to explore how this merging process proceeds online. We first presented participants with a short animation, illustrating different simple propositions that are possible to be merged by the sentential connectives. We then auditorily played an only if or an even if statement and recorded participants' eye movements on the concurrent test image. We observed that hearing the sentential connective results in more fixations on the tokens of the appropriate propositions that are eligible to be merged by the sentential connective. Each sentential connective elicited anticipatory effect suggests that once they heard the sentential connective, participants knew which propositions could be merged. We then discussed the implications of our results to the mental model theory of conditionals and the experimental studies reported in literature.     

Power and sample size calculations for multivariate linear models with random explanatory variables

This article considers the problem of power and sample size calculations for normal outcomes within the framework of multivariate linear models. The emphasis is placed on the practical situation that not only the values of response variables for each subject are just available after the observations are made, but also the levels of explanatory variables cannot be predetermined before data collection. Using analytic justification, it is shown that the proposed methods extend the existing approaches to accommodate the extra variability and arbitrary configurations of the explanatory variables. The major modification involves the noncentrality parameters associated with the F approximations to the transformations of Wilks likelihood ratio, Pillai trace and Hotelling-Lawley trace statistics. A treatment of multivariate analysis of covariance models is employed to demonstrate the distinct features of the proposed extension. Monte Carlo simulation studies are conducted to assess the accuracy using a child’s intellectual development model. The results update and expand upon current work in the literature.The author wishes to thank the associate editor and the referees for comments which improve the paper considerably. This research was partially supported by a grant from the Natural Science Council of Taiwan.     

The meaning(s) of conditionals: conditional probabilities,mental models,and personal utilities

Five experiments were conducted to test the hypothesis that people understand conditional statements ("if p then q") as indicating a high conditional probability P(q/p). Participants estimated the probability that a given conditional is true (Experiments 1A, 1B, and 3) or judged whether a conditional was true or false (Experiments 2 and 4) given information about the frequencies of the relevant truth table cases. Judgments were strongly influenced by the ratio of pq to p not q cases, supporting the conditional probability account In Experiments 1A, 1B, and 3, judgments were also affected by the frequency of pq cases, consistent with a version of mental model theory. Experiments 3 and 4 extended the results to thematic conditionals and showed that the pragmatic utility associated with believing a statement also affected the degree of belief in conditionals but not in logically equivalent quantified statements.