Sound design inducing attention in the context of audiovisual immersive environments

Inês Salselas, Rui Penha, Gilberto Bernardes

Research output: Contribution to journalArticlepeer-review

Abstract

Sound design has been a fundamental component of audiovisual storytelling in linear media. However, with recent technological developments and the shift towards non-linear and immersive media, things are rapidly changing. More sensory information is available and, at the same time, the user is gaining agency upon the narrative, being offered the possibility of navigating or making other decisions. These new characteristics of immersive environments bring new challenges to storytelling in interactive narratives and require new strategies and techniques for audiovisual narrative progression. Can technology offer an immersive environment where the user has the sensation of agency, of choice, where her actions are not mediated by evident controls but subliminally induced in a way that it is ensured that a narrative is being followed? Can sound be a subliminal element that induces attentional focus on the most relevant elements for the narrative, inducing storytelling and biasing search in an immersive non-linear audiovisual environment? Herein, we present a literature review that has been guided by this prospect. With these questions in view, we present our exploration process in finding possible answers and potential solution paths. We point out that consistency, in terms of coherency across sensory modalities and emotional matching may be a critical aspect. Finally, we consider that this review may open up new paths for experimental studies that could, in the future, provide new strategies in the practice of sound design in the context of non-linear media.
Original languageEnglish
Pages (from-to)737-748
Number of pages12
JournalPersonal and Ubiquitous Computing
Volume25
Issue number4
DOIs
Publication statusPublished - Aug 2021

Keywords

  • Audiovisual attention
  • Cross-modal perception
  • Sound design

Fingerprint

Dive into the research topics of 'Sound design inducing attention in the context of audiovisual immersive environments'. Together they form a unique fingerprint.

Cite this