by C. Zetzsche, F. Roehrbein, M. Hofbauer, K. Schill
Abstract:
We show that a mobile observer in a natural environment receives systematically co-varying signals in his different sensory modalities. An independent, modality-specific processing - as assumed in classical theories of perception - would hence be sub-optimal. Rather, information theory predicts that the system should use a statistically optimised joint processing strategy. We tested this by measuring the two-dimensional just-noticeable difference (jnd) curves for basic visual-auditory stimulus configurations (a patch of light combined with a 1kHz tone). The forcedchoice task was to detect any change in this configuration, irrespective of modality. The resulting two-dimensional jnd-curve cannot be explained by an independent, modality-specific processing. In particular, the sensitivity increase for the "ecologically relevant" joint auditoryvisual increments or decrements is much higher than the usual probability summation effects. This points to a direct neural integration of visual and auditory information at an early stage.
Reference:
Audio-Visual Sensory Interactions and the Statistical Covariance of the Natural Environment (C. Zetzsche, F. Roehrbein, M. Hofbauer, K. Schill), In Proc. Forum Acousticum 2002, 2002.
Bibtex Entry:
@InProceedings{Zetzsche2002,
author = {C. Zetzsche and F. Roehrbein and M. Hofbauer and K. Schill},
title = {Audio-Visual Sensory Interactions and the Statistical Covariance of the Natural Environment},
booktitle = {Proc. Forum Acousticum 2002},
year = {2002},
abstract = {We show that a mobile observer in a natural environment receives systematically co-varying
signals in his different sensory modalities. An independent, modality-specific processing - as
assumed in classical theories of perception - would hence be sub-optimal. Rather, information
theory predicts that the system should use a statistically optimised joint processing strategy. We
tested this by measuring the two-dimensional just-noticeable difference (jnd) curves for basic
visual-auditory stimulus configurations (a patch of light combined with a 1kHz tone). The forcedchoice
task was to detect any change in this configuration, irrespective of modality. The
resulting two-dimensional jnd-curve cannot be explained by an independent, modality-specific
processing. In particular, the sensitivity increase for the "ecologically relevant" joint auditoryvisual
increments or decrements is much higher than the usual probability summation effects.
This points to a direct neural integration of visual and auditory information at an early stage.},
}