We investigate the general principles of how the human brain processes auditory information, especially in regard to our ability to combine sound with vision and touch (“multisensory integration”). Our working hypothesis for human audition is that the auditory information necessary for recognizing a sound-source (e.g., a ringing telephone) is processed along different cortical pathways in the brain than those pathways that process the spatial aspects of what we hear (e.g. where in the room the ringing telephone is located). This putative organization is analogous to the visual system, which is largely organized along “what is it” and “how” pathways for object recognition and a “where is it” pathway for localization.

The goals of our current research using functional magnetic resonance imaging (fMRI) are threefold. First, we explore brain regions responsible for recognition of natural or environmental sounds (in contrast to speech sounds, which tend to activate specialized language-related brain areas in humans).

Fig. 1: A rotating 3D model of the human brain (left cortical hemisphere) illustrating cortex (yellow) that is more responsive to hearing and recognizing environmental sounds than hearing, but not recognizing, the same sounds played backwards (n=24 participants Alpha <0.05).

For our second goal, we examine where and how the information processed along the auditory pathways becomes integrated with visual and somatosensory/motor processing pathways in the brain, providing us with a unified percept of the “multisensory” objects we experience every day.

Towards our third goal, we investigate the brain systems responsible for our spatial perception of sound, including the localization of sounds in three-dimensional space and the perception of a moving sound-source.

Fig. 2: Red depicts brain regions preferentially activated when hearing and processing hand-manipulated tool sounds, while blue depicts regions more responsive to animal vocalizations. Green shows cortex activated while participants (20 in each group) made dominant hand movements as if manipulating a variety of different tools. Note that the temporal parietal junction ("TPJ," yellow overlap region) was strongly activated only in the hemisphere opposite the dominant hand, being very responsive when recognizing tool-related sounds. ( Click to Enlarge )

These findings fit well with clinical studies, in that lesions to the TPJ in the hemisphere opposite the dominant hand can lead to a severe disruption in one`s understanding of how to appropriately use tools or common objects (termed "Ideational Apraxia"). For instance, when asked to manipulate objects of common use, such patients might produce a hammering gesture with a set of keys, or attempt to use scissors to write.