Audiovisual speech recognition : correspondence between brain and behavior / / topic editor Nicholas Altieri.

• Author: Nicholas Altieri
• TeilnehmendeR: Altieri, Nicholas,
• Place / Publishing House: Lausanne, Switzerland : : Frontiers Media SA,, 2014.; ©2014
• Year of Publication: 2014
• Language: English
• Series: Frontiers Research Topics
• Physical Description: 1 online resource (101 pages) :; illustrations, charts; digital, PDF file(s).
• Notes: Bibliographic Level Mode of Issuance: Monograph; Published in Frontiers in Psychology.

Perceptual processes mediating recognition, including the recognition of objects and spoken words, is inherently multisensory. This is true in spite of the fact that sensory inputs are segregated in early stages of neuro-sensory encoding. In face-to-face communication, for example, auditory information is processed in the cochlea, encoded in auditory sensory nerve, and processed in lower cortical areas. Eventually, these “sounds” are processed in higher cortical pathways such as the auditory cortex where it is perceived as speech. Likewise, visual information obtained from observing a talker’s articulators is encoded in lower visual pathways. Subsequently, this information undergoes processing in the visual cortex prior to the extraction of articulatory gestures in higher cortical areas associated with speech and language. As language perception unfolds, information garnered from visual articulators interacts with language processing in multiple brain regions. This occurs via visual projections to auditory, language, and multisensory brain regions. The association of auditory and visual speech signals makes the speech signal a highly “configural” percept. An important direction for the field is thus to provide ways to measure the extent to which visual speech information influences auditory processing, and likewise, assess how the unisensory components of the signal combine to form a configural/integrated percept. Numerous behavioral measures such as accuracy (e.g., percent correct, susceptibility to the “McGurk Effect”) and reaction time (RT) have been employed to assess multisensory integration ability in speech perception. On the other hand, neural based measures such as fMRI, EEG and MEG have been employed to examine the locus and or time-course of integration. The purpose of this Research Topic is to find converging behavioral and neural based assessments of audiovisual integration in speech perception. A further aim is to investigate speech recognition ability in normal hearing, hearing-impaired, and aging populations. As such, the purpose is to obtain neural measures from EEG as well as fMRI that shed light on the neural bases of multisensory processes, while connecting them to model based measures of reaction time and accuracy in the behavioral domain. In doing so, we endeavor to gain a more thorough description of the neural bases and mechanisms underlying integration in higher order processes such as speech and language recognition.

Table of Contents

Audiovisual Integration: An Introduction to Behavioral and Neuro-Cognitive Methods / Nicholas Altieri -- Speech Through Ears and Eyes: Interfacing the Senses With the Supramodal Brain / Virginie van Wassenhove -- Neural Dynamics of Audiovisual Speech Integration Under Variable Listening Conditions: An Individual Participant Analysis / Nicholas Altieri and Michael J. Wenger -- Gated Audiovisual Speech Identification in Silence vs. Noise: Effects on Time and Accuracy / Shahram Moradi, Björn Lidestam and Jerker Rönnberg -- Susceptibility to a Multisensory Speech Illusion in Older Persons is Driven by Perceptual Processes / Annalisa Setti, Kate E. Burke, Rose Anne Kenny and Fiona N. Newell -- How Can Audiovisual Pathways Enhance the Temporal Resolution of TimeCompressed Speech in Blind Subjects? / Ingo Hertrich, Susanne Dietrich and Hermann Ackermann -- Audio-Visual Onset Differences are used to Determine Syllable Identity for Ambiguous Audio-Visual Stimulus Pairs / Sanne ten Oever, Alexander T. Sack, Katherine L. Wheat, Nina Bien and Nienke van Atteveldt -- Brain Responses and Looking Behavior During Audiovisual Speech Integration in Infants Predict Auditory Speech Comprehension in the Second Year of Life / Elena V. Kushnerenko, Przemyslaw Tomalski, Haiko Ballieux, Anita Potton, Deidre Birtles, Caroline Frostick and Derek G. Moore -- Multisensory Integration, Learning, and the Predictive Coding Hypothesis / Nicholas Altieri -- The Interaction Between Stimulus Factors and Cognitive Factors During Multisensory Integration of Audiovisual Speech / Ryan A. Stevenson, Mark T. Wallace and Nicholas Altieri -- Caregiver Influence on Looking Behavior and Brain Responses in Prelinguistic Development / Heather L. Ramsdell-Hudock.