Bachelor-, Master- or Projekt-Work: Robot interpreting music to paint

Owned by Ornella Tortorici, created
Last updated: 12 Mar 2024

 

 

 

 

Task Description / Aufgabenstellung

In the context of the Ligeti Center, founded in 2023, and of which our institute is a part, our aim is to combine art and science. The Ligeti Center is an interdisciplinary transfer center that combines the fields of art, medicine, applied science and technology. This is the origin of the idea of a robot capable of interpreting music to create a work of art. This topic was already proposed as a project for the ADMM (Applied Design Methodology for Mechatronics) course in 2023. 4 groups of students developed a robot to accomplish this task, an overview of the results can be seen here.

We want to develop this project with improvements and different perspectives. For this we can propose different theses, in different fields. The robot will be used for performances during public events. Such an event already took place in the Zentralbibliothek of Hamburg in February 2024.

List of theses already done on this subject:

Different topics / tasks are available in this project:

  • Electronic integration and low level control of the system:

    • Develop the low level control of the system, including the robot arm and the color dispensers.

    • Manage the communication between the music interpretation algorithm and the sensors & actuators, ideally with ROS (arduino, raspberry or other board).

    • Integrate and calibrate a force sensor to measure the pressure of the brush.

    • Integrate sensors to get a position feedback of the motors (at least for the arm).

    • Add safety features: emergency stop, detect overpressure of the paint…

    • Build an HMI to control the robot and get feedback from it (either mobile app / website or screen and buttons on the system).

    • PCB design for the robot to integrate all functionalities and converters for the different power supplies.

  • Simulation:

    • Build a digital twin of the system.

    • Painting simulator taking into account the same parameters as the robot: color, speed of the brush, orientation of the brush, pressure of the brush to show real time digital painting.

  • Gesture control:

    • Implementation of gesture control of a UR robot from our Qualisys motion tracking system.

    • Implementation of gesture control of the painting robot from our Qualisys motion tracking system.

  • Music interpretation improvements:

    • Research on music interpretation in contact with artists (musicians, dancers, painters,…).

    • Categorize existing paintings depending on emotions/feelings (sad, joyful, fun, thrilling,…) to associate with the music styles.

    • Make the algorithm predictive and not only reactive.

    • Get inspiration from Synaesthesia (people perceiving letters with colours for example).

    • Get inspiration from graphic scores for music improvisation.

    • Deep learning from drawings of people on the same music.

    • Analysis / deep learning from album/single covers depending on music styles.

    • Recognize the musical instruments playing

One thesis would correspond to one or several of these points, but not all of them! Depending on the amount of tasks and their complexity, the thesis can be a BA, PA or MA.

contact person: @Ornella Tortorici

Name:

 

Thesis Type MA/BA/PA:

 

Student ID / Matrikelnummer:

 

Field of Study / Studiengang:

 

Official start-date / Offizieller Beginn:

 

Final-report-due /Abgabe:

 

Spotlight-presentations:

1.

2.

3.

Zweitprüfer / Second Examiner

 

Confidential / Vertraulich

 

Zeitplanung:

Checklist

Helpful links:

Document Upload Final Thesis / Dokumentenabgabe Abschlussdokument

File of final presentation / Dokumentenabgabe Abschlusspräsentation

 

Institut für Mechatronik im Maschinenbau (iMEK), Eißendorfer Straße 38, 21073 Hamburg