Analyzing sequential categorical data: Individual variation in markov chains

Markov chains are probabilistic models for sequences of categorical events, with applications throughout scientific psychology. This paper provides a method for anlayzing data consisting of event sequences and covariate observations. It is assumed that each sequence is a Markov process characterized by a distinct transition probability matrix. The objective is to use the covariate data to explain differences between individuals in the transition probability matrices characterizing their sequential data. The elements of the transition probability matrices are written as functions of a vector of latent variables, with variation in the latent variables explained through a multivariate regression on the covariates. The regression is estimated using the EM algorithm, and requires the numerical calculation of a multivariate integral. An example using simulated cognitive developmental data is presented, which shows that the estimation of individual variation in the parameters of a probability model may have substantial theoretical importance, even when individual differences are not the focus of the investigator's concerns.Research contributing to this article was supported by B.R.S. Subgrant 5-35345 from the University of Virginia. I thank the DADA Group, Bill Fabricius, Don Hartmann, William Griffin, Jack McArdle, Ivo Molenaar, Ronald Schoenberg, Simon Tavaré, and several anonymous reviewers for their discussion of these points.