Generation and evaluation of user tailored responses in multimodal dialogue

When people engage in conversation, they tailor their utterances to their conversational partners, whether these partners are other humans or computational systems. This tailoring, or adaptation to the partner takes place in all facets of human language use, and is based on a mental model or a user model of the conversational partner. Such adaptation has been shown to improve listeners’ comprehension, their satisfaction with an interactive system, the efficiency with which they execute conversational tasks, and the likelihood of achieving higher level goals such as changing the listener’s beliefs and attitudes. We focus on one aspect of adaptation, namely the tailoring of the content of dialogue system utterances for the higher level processes of persuasion, argumentation and advice-giving. Our hypothesis is that algorithms that adapt content for these processes, according to a user model, will improve the usability, efficiency, and effectiveness of dialogue systems. We describe a multimodal dialogue system and algorithms for adaptive content selection based on multi-attribute decision theory. We demonstrate experimentally the improved efficacy of system responses through the use of user models to both tailor the content of system utterances and to manipulate their conciseness.     

A versatile electronic switch with memory

This noise-conditioned switch provides a convenient interface between S response switches and logic circuitry and is adaptable to many experimental situations.     

The basis of Stroop interference involving the multimodal correlates of auditory pitch

A pure auditory tone has a range of multimodal qualities that are determined by its pitch. A reaction-time task was used to demonstrate that subjects respond immediately and automatically to these qualities. Subjects were required to press one of two keys depending on which word, from a limited set, appeared on a microcomputer screen. The words were antonyms that represented multimodal stimulus qualities, and they were assigned to alternative responses so that the two words that shared the same response were correlated in the same way with pitch. As an incidental stimulus, either a 50 Hz tone or a 5500 Hz tone accompanied the presentation of each word. Subjects were found to respond more slowly when the multimodal qualities of the tone were incongruent with the qualities represented by the test word. When the stimulus-response mapping rules were changed, however, the Stroop effect did not occur; suggesting that a polarised semantic code of the incidental tone, that embraces its multimodal features, accesses the same semantic register as the equivalent code for the test word itself.     

On the emotions that accompany autobiographical memories: Dysphoria disrupts the fading affect bias

Participants in two studies recalled autobiographical events and reported both the affect experienced at event occurrence and the affect associated with event recollection. The intensity of affect associated with a recalled event generally decreased, but the affective fading was greater for negative events than for positive events. The magnitude of this fading affect bias also varied with participants' dysphoria levels: Dysphorics showed a smaller fading affect bias than non-dysphorics. Additional analyses suggested that the fading affect bias is not a product of: (1) distorted retrospective memory for the affect originally accompanying events; (2) differences in the initial affect intensity of positive and negative events; or (3) differences in the ages of positive and negative events. Other variables that might be related to the fading affect bias are discussed.