EMORSION: Examining the Impact of Audio Parameters on Emotional Responses and Immersion in Film

What Happened

Researchers published EMORSION, a proof-of-concept study on arXiv that systematically examines how specific audio parameters—such as volume, pitch, tempo, and spectral balance—influence emotional responses and immersion in film audiences. The study selected four film scenes spanning horror and drama genres, balanced between mainstream and independent productions, and measured participant reactions in a controlled cinema setting. By isolating audio variables rather than treating sound as a monolithic element, the work aims to quantify which acoustic features most strongly correlate with fear, sadness, tension, or absorption.

Why It Matters

This research addresses a persistent blind spot in both media psychology and AI-driven content analysis. Most affective computing models focus on visual cues—facial expressions, scene composition, or color palettes—while treating audio as a secondary or aggregated feature. EMORSION’s granular approach suggests that parameters like attack time or frequency range can independently modulate emotional states, which has direct implications for automated film scoring, adaptive sound design, and recommendation systems that optimize for user engagement.

For AI practitioners, the study underscores the value of feature-level audio analysis over high-level genre tags. A horror scene with a slow, low-frequency drone may induce dread differently than one with sharp, high-pitched stabs, yet both are often collapsed into a single “scary” label in training datasets. EMORSION provides a methodological template for decomposing such categories into measurable, manipulable audio dimensions.

Implications for AI Practitioners

First, the findings can improve multimodal emotion recognition systems. Current architectures often fuse video and audio embeddings at a late stage, losing the nuanced interplay between, say, a character’s calm expression and a tense musical score. Incorporating parameter-level audio features could yield more robust predictions of viewer affect, particularly in ambiguous scenes where visual and auditory cues conflict.

Second, generative AI tools for sound design—such as text-to-audio models or automated Foley systems—could benefit from parameter-level conditioning. Instead of prompting “sad music,” a creator could specify “slow tempo, minor key, low spectral centroid,” enabling more precise emotional targeting. EMORSION’s data could serve as a validation benchmark for such systems.

Third, the study highlights the need for domain-specific datasets. Most publicly available audio emotion datasets are built on short clips of music or isolated sounds, not continuous film scenes with narrative context. Expanding these resources with parameter annotations would accelerate progress in affective audio AI.

Key Takeaways

• EMORSION demonstrates that discrete audio parameters (e.g., tempo, pitch, spectral balance) independently influence audience emotion and immersion, beyond high-level genre or mood labels.
• The research provides a methodological framework for decomposing film audio into measurable features, which can enhance multimodal AI models for emotion recognition and content recommendation.
• Generative audio tools and automated sound design systems can leverage parameter-level conditioning to achieve more nuanced emotional outcomes.
• The study underscores the need for richer, film-specific audio datasets with granular feature annotations to support future affective computing research.