PA: Development of an Augmented Reality Interface for Intuitive Spatial Audio Control with Haptic Feedback

Offizieller Titel Deutsch

Entwicklung einer Augmented-Reality-Schnittstelle zur intuitiven Steuerung von 3D-Audio Systemen mit haptischem Feedback

Official Titel English

Development of an Augmented Reality Interface for Intuitive Spatial Audio Control with Haptic Feedback

Task Description / Aufgabenstellung

Coreknowledge to bring: pre-experiences with AR/VR or really strong intrinsive motivation to self-study it

The Institute for Mechatronics in Mechanics, in collaboration with the interdisciplinary ligeti zentrum, conducts research on the application of vibrotactile actuators in artistic contexts. Vibrotactile actuators are resonating devices that oscillate within the vibrotactile frequency range, offering unique opportunities for interactive and sensory applications.

A new collaborative project aims to develop an augmented reality (AR) interface for controlling a spatial 3D audio system installed at the ligeti zentrum. Current spatial audio tools, such as SpatRevolution and IRCAM's OpenSource Spat/Spatialisateur, provide advanced sound rendering algorithms but lack user-friendly interfaces for real-time interaction in 3D environments. Additionally, these systems often fail to effectively integrate hardware and AR technologies, limiting their usability in dynamic, interactive setups.

The proposed thesis focuses on designing and implementing an AR interface that enables intuitive interaction with sound sources in a 3D room environment. Sound sources should be represented as "sound balls" within the AR space, allowing users to manipulate their positions and properties dynamically. For instance, users can move these sound balls to different locations in the virtual room, and the system should update the spatial audio configuration accordingly.

To enhance user experience, the system should incorporate haptic feedback through vibrotactile actuators. A pivoting handle (f.ex. something like ) should provide tactile sensations corresponding to the audio track or instrument being manipulated. Additionally, users should be able to control key sound parameters—such as volume, equalizer settings, and user-defined configurations—through gesture-based interactions. Changes in these settings should also be reflected in the haptic feedback, creating a multi-sensory experience and usability.

The spatial audio system should start with a default configuration where all “sound balls” are positioned on the floor. Users can pick them up and place them in new positions, enabling personalized spatial audio setups. This dynamic interaction will allow for a highly customizable and immersive sound experience. To facilitate user interaction, the system should include a control over key sound parameters such as volume, equalizer settings, and a user-defined setting implemented by gestures, enabling users to interact with the sound balls directly through hand movements. These change of setting should the user also get mirrored in the haptic feedback as well. The user position of spatial systems are normally in the optimal listening position of the spatial audio system, located at the center of the setup, ensuring a balanced audio experience.

The research will address challenges such as integrating spatial audio algorithms with AR hardware, achieving low-latency audio rendering, and designing effective gestures for sound manipulation. The final system will include a fully functional AR spatial audio interface, a graphical user interface (GUI) for sound control and visualization, and gesture-based interaction mechanisms. Comprehensive documentation will detail the algorithms, system architecture, and user interaction design.

This work will advance AR-based audio systems and provide valuable insights into intuitive sound manipulation in virtual environments.

Reference Person: @Juliana Lüer supported by Jacob from the ligeti zentrum

Zeitplanung:

Checklist

• Introduction / tour in M4

• Urheberrechtsvereinbarung signed: https://www.tuhh.de/t3resources/sls/pdf/ZPA/Formulare_oeffentlich/Rechte_an_Abschlussarbeiten.pdf

• if applicable: signed confidential agreement

• official registration

Helpful links:

• Thesis Work / Studentische Arbeiten 

• Criteria for the evaluation of Bachelor's and Master's theses, handout version

• Laborordnung

• Spotlights / Kurzvorträge

• Templates (Latex, ppt etc.)

Document Upload Final Thesis / Dokumentenabgabe Abschlussdokument

File of final presentation / Dokumentenabgabe Abschlusspräsentation

Link for further files / Link für weitere Dokumente