THE ROLE OF AMPLITUDE ENVELOPE IN AUDIO-VISUAL PERCEPTION: TESTING THE EFFECT OF AMPLITUDE ENVELOPE IN SPATIAL VENTRILOQUISM

Abstract

The world is filled with richly diverse sounds which we are able to perceptually distinguish using a variety of properties. One of these properties is the amplitude envelope, or the intensity of a sound over time. While it is common in the real world for sounds to have time-varying amplitude envelopes, the majority of sounds used in perceptual research have time-invariant or unspecified amplitude envelopes. The aim of the present thesis is twofold. Because many of the studies using time-invariant or undefined envelopes make use of very short sounds (below 100 msec), the first experiment aimed to determine the duration required for discriminating among three different envelopes: flat (invariant), ramped (increasing in intensity over time), and damped (decreasing in intensity over time). In Experiment 1, participants took part in a 2-alternative forced choice, psychophysical staircase paradigm in which they indicated which of two envelopes they thought they were listening to. Results showed that, when telling ramped tones apart from either flat or damped tones, participants showed discrimination thresholds below 50 msec, while they had thresholds of approximately 75-80 msec when differentiating flat from damped tones. Because amplitude envelope has been shown to impact audiovisual integration and the perceptual system is sensitive to interaural envelope differences when localizing sounds, the second experiment aimed to determine whether amplitude envelope could modulate the visual bias present in spatial ventriloquism, an audiovisual illusion where the perceived location of a sound is influenced by the location of a visual stimulus. In Experiment 2, participants performed a psychophysical staircase task which measured their accuracy in localizing sounds with flat and damped envelopes, with or without a simultaneous flash on the screen in front of them. Results showed that, at durations above the envelope discrimination thresholds found in Experiment 1 (83 msec), there was no visual bias on perceived location of the sound, while the bias was present at a duration below this threshold (16 msec). Together, these results add to the mounting evidence suggesting that amplitude envelope has profound and varied effects on our perception of sounds, and is an important property to consider when designing experiments.