MA: Improving Human-Robot Skill Transfer via Mixed Reality and Virtual Haptic Feedback in Industrial Application.

Project Description:

This project aims to explore how mixed reality (MR) technology can enhance intuitive programming for industrial robots by enabling real-time measurement of position and virtual force during human-robot interactions. The focus will be on replicating human motion tasks with an industrial robot using MR-based position and virtual force sensing. Additionally, a user-friendly MR interface will be developed for real-time interaction, incorporating features such as defining home positions, recording trajectories, simulating dry runs, implementing emergency stops, and facilitating tool changes. The interface will provide visual feedback on both the position and force exerted by the operator or tool. To achieve this, a tracking system utilizing VIVE trackers attached to the operator's hand or tool will be used to accurately measure position, angle, and velocity during interaction. Force regulation will be addressed by creating a wireframe mesh with a defined tolerance thickness along the object's surface and estimating force based on mechanical properties within the MR environment. Excess force will be prevented through haptic feedback mechanisms. The efficacy of the MR system in reproducing 6-degree-of-freedom (6-DoF) trajectories demonstrated by human operators during motion tasks will be showcased, utilizing a Scale factor. Finally, the applied force and robot path will be compared and verified in a real polishing application for a complex surface as a pilot study.

Tasks and Duties:

 o   Research and Literature Review:

·       Conduct a comprehensive review of existing literature on mixed reality (MR) technology for human-robot interaction application

·       Review of Electrical Muscle Stimulation feedback mechanisms.

o   MR Interface Design and Development:

·       Design and develop a user-friendly MR interface for real-time interaction with industrial robots.

·       Implement features such as defining home positions, recording trajectories, simulating dry runs, implementing emergency stops, and facilitating tool changes, and scale factor.

·       Integrate visual feedback mechanisms for position and virtual force exerted by the operator into the MR interface.

o   Virtual Force Sensing and Regulation:

·       Develop algorithms for virtual force sensing within the MR environment.

·       Create a wireframe mesh with a defined tolerance thickness along the object's surface to estimate force based on the mechanical properties of the object/tool and the movement of the tool.

·       Implement Haptic-feedback mechanisms to prevent excess force exertion (VR and hand).

o   Hardware/Software Setup and Integration:

·       Set up VIVE trackers and necessary hardware components for tracking the operator's hand or tool accurately.

·       Integrate the tracking system with the MR interface to measure position, angle, and velocity during human-robot interactions.

·       Implement ROS (Robot Operating System) nodes in Python or C++ for communication between the MR interface, industrial robot, and tracking system.

·       Integrate the Unity application with ROS for real-time data exchange and control of the industrial robot.

o   Experimental verification:

·       Conduct an evaluation study, comparing the accuracy of positioning between two types of trackers at different speeds (Inside-Out Tracking and Lighthouse Tracking).

·       Conduct a pilot study to verify the ability of the proposed approach in a real polishing application.

o   Documentation and Presentation:

·       Document the design, implementation, and testing processes.

·       Prepare user manuals or guides for operating the system.

·       Create presentation materials summarizing the project objectives, methodologies, and outcomes for final reporting.

 Requirements:

o   Proficient in coding languages such as C#, Python, or C++

o   Experienced in using Unity for developing MR interfaces

o   Familiar with ROS for robot control

o   Knowledgeable in haptic systems

Contact Person: Dr. Mohammad Sadeghi (mohammad.sadeghi@tuhh.de)

Desired starting date: As Soon As possible