Project Description:
The project aims to utilize quaternion-based techniques to resolve enhance the functionality of the pre-designed/fabricated Head Camera mechanism by integrating closed-loop feedback control and a machine learning algorithm. The mechanism is capable of mimicking human head movement for extension, flexion, and lateral position. A multilayer perceptron Artificial Neural Network (ANN) is utilized to construct an identification system that establishes a relationship between desired camera poses and the angular configuration of the three motors controlling the mechanism, achieved by solving inverse kinematic models. The Proportional Integral Derivative (PID), Fractional Order Proportional Integral (FOPI), and FOPID are considered as control algorithms. To prevent gimbal lock issues in a 3-DOF actuator control system that interfaces , quaternion-based techniques should be employed, interfacing with a VIVE VR headset for immersive interaction. Quaternion representations offer advantages in representing depicting 3D rotations without the limitations associated with constraints of Euler angles, thus thereby mitigating gimbal lock while ensuring smooth and accurate translation of head movement to actuator rotation. To ensure the accuracy of the system's movement, closed-loop feedback should be considered. Additionally, incorporating voice control for user communication in the VR environment would be is desirable.
Tasks and Duties:
Literature Review:
Conduct a thorough review of the literature on gimbal lock mitigation techniques, camera control systems, quaternion-based orientation, and voice recognition technologies in VR environments.
Identify existing methods and solutions for addressing gimbal lock, integrating multiple cameras, implementing voice commands, Machine learning algorithms, and utilizing quaternions for orientation representation.Analyze the advantages and limitations of quaternion-based approaches compared to traditional Euler angle representations.
System Design:
Extend the system architecture to incorporate camera control mechanisms, voice recognition capabilities, and quaternion-based orientation techniques.
Define the specifications for integrating two cameras: one the camera on the 3 DOF mechanism and the other on the end-effector of the robot.
Design the voice command interface for switching between cameras, controlling zoom functions, and interacting with quaternion-based orientation.
Camera Integration:
Implement hardware and software components to integrate the cameras with the VR headset and Read the IMU/Temp sensor for monitoring purposes on VR.
Develop communication protocols and interfaces to ensure seamless interaction between the cameras and the control system.
Design and implement Implement the required sensor for feedback control (External IMU or embedded camera sensor).
Utilizing the ANN algorithm to solve the inverse kinematics and comparing it to the analytical solution.
Implement the PID/ FOPI/ FOPID controller for the close loop system and compare the results.
Calibrate the cameras and optimize their the positioning for optimal viewing angles.
Quaternion-based Orientation:
Develop algorithms for quaternion-based representation of head movements captured by the VR headset.
Implement quaternion transformations to translate head orientations into precise actuator rotations.
Ensure compatibility and synchronization with ROS middleware.
Voice Control System:
Research and select appropriate voice recognition technologies compatible with the VR headset and ROS.
Develop algorithms for processing voice commands related to camera switching, zoom in, zoom out, and orientation control (Scale control).
Integrate the voice control system within the VR headset, ensuring accuracy and responsiveness to user commands.
Testing and Validation:
Conduct comprehensive testing of the integrated system to verify the functionality of the camera switching, zoom control, and quaternion-based orientation.Evaluate the robustness and reliability of the voice control system and quaternion-based orientation under various environmental conditions and user scenarios.
Gather user feedback through usability testing to identify areas for improvement and refinement.
Documentation and Presentation:
Document the design, implementation, and testing processes for camera integration, quaternion-based orientation, and voice control system.
Prepare user manuals or guides for operating the VR headset and utilizing voice commands.
Create presentation materials summarizing the project objectives, methodologies, and outcomes for final reporting.
...
o Programming Skills (Arduino, C++, OR Python)
o Strong Good mathematical background (Quaternion)
o Basic knowledge of Machine learning (e.g., ANN)
o Understanding of Control Systems (PID)
o ROS Familiarity would be an advantageo Interest in Robotics and Virtual Reality
Contact Person: Mohammad Sadeghi (mohammad.sadeghi@tuhh.de)
...