A discrete time model for detection of randomly presented stimuli

Considered in this paper is a discrete-time Markov decision process with unobservable states, which occurs in equipment checking, medical diagnosis, signal detection, and reaction time studies. The process is characterized by (i) a state S₀ that at some unknown time t, which is determined probabilistically, turns into an absorbing state S₁; (ii) observations that provide fallible information about the true state and which are taken sequentially to determine whether or not the change from S₀ to S₁ has occurred; and (iii) losses for a false alarm and for delays in detecting the change. A response model is proposed, prescribing a terminal decision as soon as the posterior probability that S₁ obtains reaches or exceeds a fixed probability boundary. Predictions are derived concerning the probability of false alarm and several conditional distributions of the number of observations taken.