When half a face is as good as a whole: Effects of simple substantial occlusion on visual and audiovisual speech perception

The visible movement of a talker's face is an influential component of speech perception. However, the ability of this influence to function when large areas of the face (~50%) are covered by simple substantial occlusions, and so are not visible to the observer, has yet to be fully determined. In Experiment 1, both visual speech identification and the influence of visual speech on identifying congruent and incongruent auditory speech were investigated using displays of a whole (unoccluded) talking face and of the same face occluded vertically so that the entire left or right hemiface was covered. Both the identification of visual speech and its influence on auditory speech perception were identical across all three face displays. Experiment 2 replicated and extended these results, showing that visual and audiovisual speech perception also functioned well with other simple substantial occlusions (horizontal and diagonal). Indeed, displays in which entire upper facial areas were occluded produced performance levels equal to those obtained with unoccluded displays. Occluding entire lower facial areas elicited some impairments in performance, but visual speech perception and visual speech influences on auditory speech perception were still apparent. Finally, implications of these findings for understanding the processes supporting visual and audiovisual speech perception are discussed.     

Perceived causality occurs with stroboscopic movement of one or both stimulus elements

It was shown by magnitude estimation that the perception of causality first described by Michotte (1946/1963) also occurs consistently and strongly with stroboscopic, ie apparent or phi, movement. This is so when the 'causal' movement is stroboscopic and the 'caused' movement real, when these movements are reversed, and when both movements are stroboscopic. The effect is not due to prior experience with Michotte-type displays.     

Apparent movement detection in the pigeon

Pigeons were trained to discriminate between the presence and absence of apparent movement in visual displays. Generalization gradients obtained on the dimension of speed indicated a broad range of movement detection. Extremely accurate discrimination performance on displays varying in direction of movement suggested formation of a "movement concept".     

Effects of living in the up-down inverted visual world on apparent movement

The purpose of this study was to examine whether a change of perceptual framework may affect the occurrence of apparent movement. Apparent movement was observed by one subject living over four days in the prismatically inverted visual world, because this situation was considered as the operation in which the subject was forced to change the perceptual framework to adapt himself to the novel environment. Apparent movement with two points was measured in vertical and horizontal configurations before wearing and after removing the prism as well as while wearing it. Analysis showed significant effects of prism-wearing on the occurrence of apparent movement in both vertical and horizontal configurations. Although further elaboration is required, an hypothesis was suggested from the viewpoint of the loss of visual position constancy.     

Launching the effect: Representations of causal movements are influenced by what they lead to

We investigated whether the representation of an observed causal movement is influenced by its observed effect. Subjects watched displays showing collisions between two objects. In this 'launching event' (Michotte, 1946/1963), one of the two objects (Object A) started to move and set a second, initially stationary, object (Object B) into motion, which gave a strong impression of apparent causality. The apparent effectiveness of A's movement was manipulated by varying the velocities of A and B. When the velocity of B was higher than that of A, the effectiveness of the collision was high; when it was smaller it was low. Then, subjects were asked to reproduce the velocity of the causal movement. Reproduced velocity followed the velocity of both Object A and Object B, which supports the hypothesis that the effect of a movement is integrated with its apparent cause. However, when apparent causality was reduced by changing the direction of motion of B or by covering the point of collision, the influence of the effect on the representation of the cause persisted, suggesting that retroactive interference may account for the findings. The interference effect could not be reduced to temporal recency or spatial integration and was not obtained in the reverse temporal order (proactive interference). Rather, the two successive movements were blended in memory.     

The prevalence effect in two-dimensional stimulus-response compatibility is a function of the relative salience of the dimensions

When stimulus-response (S-R) sets vary along horizontal and vertical dimensions, a right-left prevalence effect is often obtained in which the horizontal compatibility effect is larger than the vertical compatibility effect. Vu and Proctor (2001) showed that the prevalence effect varies as a function of the dimension made salient by the response configuration. A salient features coding interpretation of this result implies that manipulating the salience of the stimulus display should produce similar results and that S-R translation should be fastest when salient features of the stimulus and the response sets correspond. Experiment 1 manipulated spatial proximity to make the vertical or the horizontal stimulus dimension salient. Neutral displays yielded a typical right-left prevalence effect, and this effect was enhanced by horizontal-salient displays and eliminated by vertical-salient displays. Experiments 2 and 3 showed that the benefit for horizontal (or vertical) compatibility was larger when the salient features of both the stimulus and the response sets emphasized the horizontal (or the vertical) dimension than when only one did. The results support salient features coding as an explanation for the prevalence effect obtained with two-dimensional S-R arrangements.     

The subliminal perception of movement and the course of autokinesis

The course of autokinesis is shown to be sensitive to the real movement of a surrounding stimulus. With the supraliminal presentation of this stimulus, apparent movement in a direction opposite to that of the real movement is induced. With the subliminal presentation of the same stimulus the real movement serves to inhibit autokinesis by inducing brief periods of stationarity between the phases of upward and downward apparent movement. The results confirm previous findings that the movement of a stimulus may be discriminated without there being any perceptual (phenomenal) adjunct.     

The effect of spatial orientation on the perception of moving tactile stimuli

Previous studies have shown that the perception of spatial patterns, such as letters, presented to the hand is affected by the spatial orientation of the hand. The present study investigated how the perception of direction of motion across the fingerpads changes with the position of the hand in space. The moving stimuli were generated on two displays. In one condition, the displays were placed horizontally in front of the subject, with the subject’s thumb (target site) and index finger (nontarget site) placed flat on the displays. In a second condition, the displays were vertically oriented and gripped between the thumb and index finger. Using a selective-attention paradigm in which subjects are instructed to respond only to the direction of motion at the target site, performance was still affected by the direction of motion at the nontarget site. Changing the orientation of the displays changed the effectiveness of the nontarget in interfering with the identification of the target movement. Nontarget stimuli that produced no interference in the horizontal orientation did so in the vertical, and vice versa. It appears that subjects are not using the local direction of movement across the fingerpads to judge the relative direction of movement at the two sites; rather, they are using the external direction of movement.     

Sensory and cognitive factors in the processing of visual velocity

A symmetrical 6 x 6 factorial design of distances and durations served to produce either 36 different moving stimuli (real movement condition) or 36 static displays separately containing the respective stimulus components (cognitive movement condition). Different metric rules underlay the two types of velocity judgments: Perceptual estimations of real movement obeyed a ratio model, whereas conscious estimations of implied movement obeyed an additive model. Valuation operations differed, too; the scales underlying real velocity were nonlinearly related to the even more compressive scales that underlay cognitive velocity. Implications of these results for velocity research are discussed.     

Perceived orientation in depth from line-of-sight movement

Twelve college students viewed computer-generated displays of a cross comprised of two orthogonal dotted lines, and judged the apparent in-depth orientation of the horizontal arm by positioning a horizontal bar mounted on a rotary potentiometer. The vertical arm of the simulated cross was always in the observer’s frontal plane, but the randomly textured horizontal arm was in one of nine orientations relative to the line of sight. Each observer viewed displays in which the simulated cross was, alternately, (a) stationary, (b) approaching the viewer, and (c) stationary but expanding in size. The static texture density gradient in the horizontal arm of the simulated stationary cross mediated perceived orientation in depth. Further, when motion perspective was added to the detail perspective, the impression of depth was enhanced, with the greatest enhancement obtaining at the near viewing distance. When dynamic magnification was added to the detail perspective, the impression of depth was attenuated; this effect was interpreted as an illusory case of motion perspective.     

The perceptual fate of letters in two kinds of apparent movement displays

Two experiments investigated the importance of figurai relationships among stimulus elements in the determination of the kind of motion perceived in bistable apparentovement displays. Stimulus frames containing the words MITE and ITEM were constructed in such a way that the letters I, T, and E appeared in corresponding locations while the M was in disparate locations between frames. When the frames were alternated with a 20-msec ISI, observers maintained local figurai identity in their percepts of apparent movement. However, when the frames were alternated with an 80-msec ISI, observers reported apparent movement of the entire groups of letters and thereby ignored figurai relationships in their percepts of apparent movement. In a second experiment, it was found that figurai nonidentity between elements of successive frames always resulted in apparent movement of a perceptual group or “blob” regardless of ISI. It is suggested that short-range apparent-movement percepts rely strongly upon figurai identity among corresponding stimulus elements, whereas long-range apparent-movement percepts do not.     

Magnitude of luminance modulation specifies amplitude of perceived movement

A compelling impression of movement, which is perceptually indistinguishable from a real displacement, can be elicited by patterns containing no spatially displaced elements. An apparent oscillation, w-movement, was generated by a stationary pattern containing a large number of horizontal pairs of spatially adjacent dots modulated in brightness. The observer’s task was to adjust the perceived amplitude of the w-motion to match the amplitude of a real oscillation. All of the data can be accounted for by a simple rule: If the relative change in the luminance,W = αL/L, between two adjacent stationary dots is kept constant, the distance over which these dots appeared to travel in space comprises a fixed fraction of the total distance by which they are separated. The apparent amplitude of the w-motion increases strictly in proportion with luminance contrast, provided that the contrast is represented in the motion-encoding system by a rapidly saturating compressive Weibull transformation. These findings can be explained in terms of bilocal motion encoders comparing two luminance modulations occurring at two different locations.     

Nodding in dis/agreement: a tale of two cultures

Head movement is commonly used to communicate positive versus negative response. However, whereas in US culture, vertical head movement denotes positivity (nodding to say "yes") and horizontal head movement is associated with negativity (shaking heads to say "no"), in Bulgaria, this response pattern is reversed, that is, horizontal head movement means "yes" and vertical head movement means "no." Thus, these two cultures spatially "embody" agreement via different movement directions. We examined the effect of such cultural differences on cognitive processing that has no communicative intent by comparing ratings of likeability, brightness, and positive feeling associated with different color moving dots. Participants followed the dots' movement with their heads in a 2 (direction: vertical vs. horizontal) by 2 (speed: fast vs. slow) design. We found a two-way country by speed of movement interaction such that Bulgarian participants associated colors with more positive feeling when those were perceived in combination with slower head movement irrespective of movement direction. US participants, on the other hand, rated color dots as better mood-enhancing in combination with faster head movement. There was a similar two-way country by movement speed interaction for likeability ratings but none for brightness. Findings are discussed in terms of variability in gestural meaning and culture-specific embodiment patterns.     

Illusory continuous motion from oscillating positive-negative patterns: implications for motion perception

A black and white (positive) grating pattern was superimposed in exact register on its own photographic negative. Four operations were repetitively applied to this positive pattern so that it moved fractionally to the right, grew dimmer, moved back to the left, and grew brighter again. This sequence produced a strong illusion of continuous apparent motion to the right for as long as the cycle was repeated. The small relative motion between the two patterns generated two new illusory effects: enhanced real movement (ERM) and reversed real movement (RRM). The dimming and brightening phases gave rise to reversed apparent movement (RAM). All three effects are attributed to spatial filtering by neural mechanisms, which shifts the effective position of the positive-negative contours.     

An augmented reality home-training system based on the mirror training and imagery approach

Mirror training and movement imagery have been demonstrated to be effective in treating several clinical conditions, such as phantom limb pain, stroke-induced hemiparesis, and complex regional pain syndrome. This article presents an augmented reality home-training system based on the mirror and imagery treatment approaches for hand training. A head-mounted display equipped with cameras captures one hand held in front of the body, mirrors this hand, and displays it in real time in a set of four different training tasks: (1) flexing fingers in a predefined sequence, (2) moving the hand into a posture fitting into a silhouette template, (3) driving a “Snake” video game with the index finger, and (4) grasping and moving a virtual ball. The system records task performance and transfers these data to a central server via the Internet, allowing monitoring of training progress. We evaluated the system by having 7 healthy participants train with it over the course of ten sessions of 15-min duration. No technical problems emerged during this time. Performance indicators showed that the system achieves a good balance between relatively easy and more challenging tasks and that participants improved significantly over the training sessions. This suggests that the system is well suited to maintain motivation in patients, especially when it is used for a prolonged period of time.     

Spatial correspondence and relation correspondence: grouping factors that influence perception of the Ternus display

We cite two different perceptual-correspondence principles whose emphasis can help to disambiguate the otherwise ambiguous Ternus display in apparent movement (a display that can alternately be seen in one of two possible configurations). One of these principles is spatial correspondence, which emphasizes the maintenance of similar stimulus elements in given locations over time. The other principle is relation correspondence, which emphasizes the maintenance of the inter-organization of stimulus elements across frames. Each of four experiments reported here made use of a different stimulus feature (eg element color or element texture) to emphasize these different correspondence principles in Ternus displays. As predicted, an emphasis upon spatial correspondence resulted in an increase in reports of element movement compared to a featureless control condition. Emphasis on relation correspondence resulted in an increase in reports of group movement. Extensions and limitations of the use of these principles to explain apparent movement are discussed.     

Displacement of the path of perceived movement by intersection with static contours

Observation of a moving dot gives rise to a perceived movement path, which has properties similar to those of real contours. If the dot crosses a line inclined to a horizontal direction of movement, it appears to undergo a vertical displacement. This displacement was found to be greatest for a line orientation of around 15° with respect to the movement. At other relative orientations, the size of the perceived displacement varied in the same manner as the perceived expansion of angles formed by intersection between static contours. Movement path distortions were measured with background fields like those that produce the Hering and Zöllner illusions with continuous lines. Illusory displacements of perceived movement were found to be equivalent to the static forms. The subjective contour formed by observation of movement can therefore give rise to illusions similar to those obtained with real lines.     

Conditional discrimination learning in the pigeon

Four pigeons received conditional discrimination training in which reinforcement contingencies were related to specific combinations of color and form, but were unrelated to either color or form considered separately. During discrete-trial training, each response in the presence of two of four color-form displays produced reinforcement and terminated the trial; responding to the other two displays was never reinforced, and each such response prolonged the particular trial on which it occurred. Subsequently, the subjects received multiple-schedule training in which responding to either of the displays previously associated with reinforcement was now reinforced on a variable-interval schedule, and extinction was the schedule again correlated with the other two displays. After differential responding to the stimuli was clearly evident, intensity of the combination displays was changed in subsequent training sessions. Complex stimulus control was generally maintained across variation in intensity, although there were temporary disruptions in performance associated with onset of some of the intensity changes. Finally, a component-stimulus test revealed considerably more responding to the forms than to the colors.     

Nodding in dis/agreement: a tale of two cultures

Head movement is commonly used to communicate positive versus negative response. However, whereas in US culture, vertical head movement denotes positivity (nodding to say “yes”) and horizontal head movement is associated with negativity (shaking heads to say “no”), in Bulgaria, this response pattern is reversed, that is, horizontal head movement means “yes” and vertical head movement means “no.” Thus, these two cultures spatially “embody” agreement via different movement directions. We examined the effect of such cultural differences on cognitive processing that has no communicative intent by comparing ratings of likeability, brightness, and positive feeling associated with different color moving dots. Participants followed the dots’ movement with their heads in a 2 (direction: vertical vs. horizontal) by 2 (speed: fast vs. slow) design. We found a two-way country by speed of movement interaction such that Bulgarian participants associated colors with more positive feeling when those were perceived in combination with slower head movement irrespective of movement direction. US participants, on the other hand, rated color dots as better mood-enhancing in combination with faster head movement. There was a similar two-way country by movement speed interaction for likeability ratings but none for brightness. Findings are discussed in terms of variability in gestural meaning and culture-specific embodiment patterns.     

Position uncertainty and the perception of apparent movement

Two experiments compared the perception of apparent movement when the second of two successive stimuli always appeared in the same position and when it varied randomly between two spatial positions. The results of both experiments showed that foreknowledge of the position of the second stimulus does not facilitate the perception of apparent movement. Experiment 2 also clearly showed that the space-time relationships of Korte’s third law of apparent movement does not depend on foreknowledge of the position of the second stimulus. These findings imply that apparent movement in real time occurs after the second stimulus has been registered by the visual system. It suggests that apparent movement involves a delayed decision mechanism that stores the first stimulus, the interstimulus temporal interval, and the second stimulus, and then impletes a motion compatible with the stimulus information.