MA - Analysis of human music interpretation into paintings for the design of a painting model. - HfMT

Analysis of human music interpretation into paintings for the design of a painting model.

Analyse der menschlichen Musikinterpretation in Gemälden für den Entwurf eines Gemäldemodells.

Taks description

Topic

This topic is part of the overarching project “Exploring the Multimodality of Arts with Robotics”.

The idea for this project arose from the Ligeti Center, founded in 2023, which promotes interdisciplinary collaboration between art, science, medicine, and technology. The current focus is on the development of a robot capable of interpreting music to create a work of art. Originally developed as part of the Applied Design Methodology for Mechatronics (ADMM) course, this project has evolved through the efforts of several students and has resulted in the design of a robotic arm, controlled in ROS, that can create paintings based on music interpretation. The robot’s brush can move across a canvas, adjust its pressure, and mix a variety of colors, all while responding to specific musical parameters such as BPM, loudness, and mood.

The project integrates mechatronics , music theory , and human artistic interpretation , exploring different aspects of musical interpretation, from machine learning to human gestures to specific parameters of the music, in order to control the robot. By comparing the different outputs and processes with those of humans, the aim is to gain a better understanding of human music interpretation.

Workshops and public events are planned, where the robot will perform live and engage audiences in exploring the intersection of art and science.

Ongoing improvements to the robot’s design, control, and machine learning amongst others will be carried out through student theses and collaborations with various labs.

Task Description

Due to the specific nature of this task, it would be carried out at the HfMT, with one of their professors as the 1st examiner.

Along our workshops on music interpretation with children, we have collected a small database of paintings linked to some songs, accompanied by surveys. We want to analyse these in order to understand more about human music interpretation. One goal is to extract some parameters (such as the colours, the type of strokes, ...) to improve the control of our painting robot.

So far, the database includes:

We also plan to create a database of paintings made by a program from different sets of music parameters. A study on this database should be planned (and carried out if the database is available in time) to determine the best set of music parameters to control the robot.

Depending on the time available, an optional task would be to study the covers of singles/albums associated with their respective songs in order to extract some general properties that could be used to generate paintings from music.

Possible areas of work / skills

• Music analysis

• Painting analysis

• Machine learning?

• Database?

• English language is preferred

Desired starting date

As soon as possible - 6 months, full time

Contact person:

@Ornella Tortorici

Zeitplanung:

Checklist

• Introduction / tour in M4

• Urheberrechtsvereinbarung signed

• if applicable: signed confidential agreement

• Official registration

Helpful links:

• Thesis Work / Studentische Arbeiten 

• Laborordnung

• Spotlights / Kurzvorträge

• Templates (Latex, ppt etc.)

• https://imek.atlassian.net/wiki/spaces/MLP/pages/12419954

• https://imek.atlassian.net/wiki/spaces/MLP/pages/58589237

• Criteria for the evaluation of Bachelor's and Master's theses, handout version

Document Upload Final Thesis / Dokumentenabgabe Abschlussdokument

File of final presentation / Dokumentenabgabe Abschlusspräsentation

Link for further files / Link für weitere Dokumente