A Novel Multi-Sensor Fusing using a Machine Learning based Human–Machine Interface and Its Application to Automate Industrial Robots
Date of Award
5-5-2024
Degree Name
M.S. in Electrical and Computer Engineering
Department
Department of Electrical and Computer Engineering
Advisor/Chair
Temesguen Messay-Kebede
Abstract
This thesis presents a novel method to control an industrial robotic arm using multiple sensors. This system consists of a hybrid brain activity and vision sensors that convey a human being’s intention and visual perception. We fuse and analyze the data from those sensors using a machine learning-based approach to automatically guide the manipulator to a designated location. We believe that this Brain–Machine–Interface (BMI) can greatly alleviate the burdensome traditional method used to program a robot (greatly aids the end-user). We experiment with different modular configurations for the brain activity information, i.e., parallelized models and what we refer to as a global model for fusing the information. We explore various machine learning and pattern recognition techniques as well as existing feature selection methods. Our experimental results show that the subject can control the robot to a destination of interest using a machine—robot–interface. We attain accuracy in the order of 99.6% when it comes to the desired motion and 99.8% for the case of deducing the desired characteristic (color) of the targeted object. These results outperform any similar existing approaches that we have researched. Moreover, in comparison to those similar operational systems, we present a unique modular configuration for brain activity interpretation and object detection mechanism that yields an overall system that is highly computationally efficient. Although, in this work, we implemented and demoed our approach using a simple pick and place demo, our work presents the basic structure underlying a system that can be efficiently used to benefit people with restricted ability to function physically (tetraplegic patients), and allowing them to perform complex and robotics related duties in an industrial setting.
Keywords
Brain Machine Interface, Machine Learning, Industrial Robotics, Sensors-Fusion, Classifiers, Similarity Distances.
Rights Statement
Copyright 2024, author
Recommended Citation
Behbenhi, Yasmeen W.A.H.A.A.E., "A Novel Multi-Sensor Fusing using a Machine Learning based Human–Machine Interface and Its Application to Automate Industrial Robots" (2024). Graduate Theses and Dissertations. 7568.
https://ecommons.udayton.edu/graduate_theses/7568
