Using natural-language responses in microcomputer research

A microcomputer natural-language interface is described that will identify limited natural-language input, such as that obtained in psychological research. In the task studied, subjects made natural-language requests involving 13 DOS commands, A Bayesian algorithm is used to calculate a vector of 13 probabilities expressing the relative likelihood of various DOS tasks that the user was attempting to specify from keywords extracted from the user’s input. The algorithm’s first-guess accuracy in the limited domain studied is approximately 90%; it will identify the user’s request with 99% accuracy in its first three guesses. This performance is considerably better than that of human experts performing the same task with the same input. Also discussed are issues of cross-validation, robustness, conditional independence, and the calibration of the algorithm’s probabilistic output.