Spatial auditory displays: Reducing cognitive load and improving driver reaction times

Advances in spatial auditory displays are showing a reduction in driver cognitive load and an increase in both and safety and convenience.

6Spatial auditory displays can offer drivers a safer and more agreeable driving experience. By providing auditory cues that associate activities and instructions with specific locations, spatial auditory displays help drivers intuitively identify voices, sources of instructions, and even the location and relative trajectory of danger. These displays can also improve reaction times to application prompts, as well as to potentially hazardous roadway objects or events.

Cognitive load is the amount of work a person’s brain is doing at any given time. It affects the ability to learn, respond to and interpret stimuli, and perform commonplace tasks such as negotiating a turn through a busy intersection.

Even straightforward tasks such as listening to instructions given by a GPS device can increase a driver’s cognitive load and thus adversely affect driving. Spatial auditory displays can reduce this cognitive load to improve driver performance as well as increase driver satisfaction with in-vehicle systems and applications.

Spatial auditory displays

Today, in-vehicle audio systems are designed so that users perceive sounds emitted by most non-music applications as coming from a single location. In contrast, spatial auditory displays cause users to perceive sound as coming from different locations in a 3D space. These locations may be inside or outside the vehicle.

A standard and a spatial auditory display are both illustrated in Figures 1 and 2. Figure 1 shows a speech recognition system with all auditory prompts coming from the same location.Figure 2 shows the same prompts using a spatial auditory display.

Figure 1: Auditory prompts sound like they’re coming from the same location in a standard auditory display for phone and climate control applications.
(Click graphic to zoom by 1.9x)

Figure 2: Auditory prompts sound like they’re coming from different locations in a spatial auditory display for phone and climate control applications.
(Click graphic to zoom by 1.9x)
Spatial auditory displays can help a driver associate an application or prompt with a specific location and effectively communicate dynamic spatial information associated with an object (for instance, current location and trajectory). This improves both the user experience of in-vehicle applications and driver performance.

Spatial cues used by listeners

Spatial auditory displays deliver acoustic cues that the human auditory system uses to localize sounds. These cues include:

  • Onset disparities: Differences of when a sound begins at the left ear and at the right ear
  • Interaural time differences: Differences in phase of a sound between the listener’s two ears
  • Interaural level differences: Differences in sound level at the ear directly in the path of the sound compared to the ear in the sound shadow produced by the listener’s head
  • Spectrum: Distribution of intensity of the sound across its frequencies
  • Overall level: Intensity of the sound being heard
  • Direct-to-reflected sound ratio: Intensity of the sound that reaches the listener directly relative to the intensity of the sound that is reflected off objects before reaching the listener
  • Deltas from reference objects: Differences in the aforementioned cues between the sound and a reference sound

Different types of spatial auditory displays

The degree to which a spatial auditory display can cause a user to perceive a sound at the desired location depends on its implementation. This implementation depends on hardware (essentially the number, type, and location of speakers) and capabilities of the signal processing (for instance, wideband versus standard signal). Figure 3 lists vehicle audio system displays, ranked by accuracy. Note that even with an ideal display, the limits of human perception hinder the precision with which location can be used to identify and differentiate prompts.

Figure 3: Vehicle audio system hardware implementations, in descending order of localization accuracy.
(Click graphic to zoom by 1.9x)

Improvements to driver performance

Spatial auditory displays can improve driver performance because they improve the human-machine interface the driver uses to communicate with the vehicle. Better task performance is enabled by improvements in areas such as error reduction, shortened response times, and better comprehension. This helps lessen the effort required to perform a task and reduces judgment errors, allowing for faster and more accurate performance of driving and secondary tasks.

Error reduction

Spatial auditory displays help reduce identification, location, and navigation maneuver errors.

Identification errors

Identification errors are errors the user makes when identifying the application, object, or event that produced a prompt, instruction, warning, and so on. Though users often eventually figure out the correct source of a sound, this identification requires effort and costs time.

Spatial location is a mnemonic device that helps users remember items by their location. The human auditory system automatically processes acoustic localization cues at a lower cognitive level to identify position; thus, users in a vehicle will remember that sounds from a specific location belong to the associated application, object, or event. Because it is a reliable clue to the source of the sound and hence to the action required, spatial location helps reduce the time and effort the driver needs to correctly interpret a prompt and perform the required action.

Location errors

Location errors occur when the user’s judgment of a sound’s location differs significantly from its true location. For example, if an ambulance is approaching from the right, but the left window is open and there is a building on the left reflecting sound from the siren, then a driver might incorrectly judge the ambulance as approaching from the left instead of the right. By controlling the acoustic cues delivered to the user, spatial auditory displays can help the driver correctly locate the ambulance (see Figure 4).

Figure 4: A spatial auditory display can help a driver locate and track an emergency vehicle.
(Click graphic to zoom by 1.9x)

Navigation maneuver errors

Navigation maneuver errors occur when the driver fails to maneuver the vehicle as prompted. Spatial auditory displays help decrease these errors in two ways. First, they provide redundant information on the location of the maneuver. Instead of just verbally prompting the driver to “turn right in 100 meters,” spatial auditory displays provide redundant information about the location of the maneuver by localizing the prompt in space. Second, they can convey trajectory information about the maneuver using apparent motion of the auditory prompt to convey, for instance, a slight right as opposed to a sharp right.

Improved driver response times

Spatial auditory displays can reduce the time a driver needs to identify the application that is providing a prompt and to fix an object’s location. With sounds emitted by a device such as the telephone always coming from one location, and those emitted by the climate control system always coming from another, a driver can instantly recognize the source of a sound or instruction. A spatial auditory display providing spatial cues to an object’s location and trajectory, such as an approaching vehicle at a blind intersection (see Figure 5), reduces the time required by the driver to correctly locate the object. Especially if the object is another vehicle on a collision path, time gained can mean the difference between a close call and an accident.

Figure 5: A spatial auditory display can alert a driver about an approaching vehicle at a blind intersection.
(Click graphic to zoom by 1.9x)

Improved speech comprehension

Spatial auditory displays can also improve speech comprehension in noisy vehicle environments by placing speech at a different location (or locations if there are multiple voices on the far end of the conversation) than noise sources such as the tires, engine, and so on. This placement produces a better sense of presence for far-end voices, which can be crisply localized directly in front, for example, instead of existing as a smeared spatial image down in the foot well. Playing back voices from different locations than noise sources helps drivers improve speech comprehension by taking advantage of the “cocktail party effect” – a listener’s enhanced ability to focus on a specific voice while ignoring other voices and noises because of information provided by the directionality of the sounds.

The benefits of improved speech comprehension are not limited to phone conversation. Especially if combined with wideband telephony, spatial auditory displays can help decrease driver distraction because the easier it is to understand who is speaking and what is being said, the fewer shared cognitive resources are required for the conversation.

Impact on driver performance

Ultimately, spatial auditory displays in vehicles help the driver easily, rapidly, and correctly identify who or what is speaking or emitting a sound, its location, and its meaning. This improved interaction between the driver and in-vehicle applications (navigation, phone, and infotainment), the vehicle itself, and its context (notices and warnings) speeds reaction times and reduces the cognitive load on the driver, leaving more cognitive resources available for the business of driving.

Auditory cues for intuitive interpretation

Spatial auditory displays will prove invaluable both for facilitating human-machine interaction within vehicles and as an enabling technology for new in-vehicle applications. These displays provide auditory cues like those humans have been using for millions of years to intuitively identify and understand object and event locations, trajectories, and meanings, enhancing the driver’s awareness of what is going on around the vehicle and reducing cognitive load to free resources for driving.

Scott Pennock is senior standards specialist at QNX Software Systems.

QNX Software Systems

Follow: Twitter Facebook Blog LinkedIn YouTube