This week Bibi worked on updated the project introduction (Home page of the website) with more concise application/environment, completing and updating the meeting minutes, converting the progress log from paragraph form to blog style, research new similar projects/ideas, and looking into how the code will need to be written for the execution of the ViBRA.
Similar Projects
Not many people have done a project that relies solely on vision for the human movement replication. One creator was found on YouTube who completed a vision based gesture-controlled hand, this does not involve the whole hand, only the wrist to fingers. In this project he utilizes an Arduino to program the hand which was 3D printed. Using the Arduino code and printed hand, the user puts up a certain hand movement in front of a camera, whether it be counting or the opening and closing on the palm, and the robotic hand would then mimic the action of the user.
This project resembles very closely to what the ViBRA should be able to do at the minimum standard.
There were also projects that mimicked a users actions using a glove or suit of some kind, these projects were of higher technical status. On the IEEE Xplore platform, an article titled "Mimicking Human Movement with Robots: Control of an Anthropomorphic Robotic Arm using a Glove-Based System as an Educational Tool" was found and focuses on the use of a glove and robotic arm to complete the hand mimicking function expected of the ViBRA. Finding these projects depicted that the idea is feasible and although it will take a lot of time and patience, it is not impossible.
How does it work?
Based on these projects that were found through research, the question arose as to how does this hand tracking mechanism work. This brought on further research that needed to be done to obtain a response to this question. It was found that two python modules were the key to getting the hand recognition software up and running, OpenCV and MediaPipe. These modules have to be installed to the python IDE of the user's choice. OpenCV is a library used for computer vision applications, it is mainly used for image and video processing. MediaPipe, which is used for building audio or video data, will be the one perform the tracking of the hand.
There is a hand-landmark model that is used to detect palm movements. The software uses this model to detect and keep track of the user's palm in order for the robotic hand to replicate in real time.
Comments