After figuring out an alternative way to send the python data to the Arduino for finger movements last week, the next task as we wait on the motors to be moved around a bit and the tension to be adjusted on the arm wiring, is to start figuring out how the wrist and elbow movements will be implemented.
Since the fingers are using a binary value of 1 or 0 to determine whether the finger is up or down and relay that information to the Arduino, adding the wrist movement proved to be a bit more complicated than just a one or zero. The fingers being up or down could have been determined by comparing the placement of the landmark points on the tip and base of the finger, but since the ViBRA's wrist will be rotation, we cannot use the same method of determination.
Therefore, after thinking and asking around to see how this can be done, we ended up back to using angle to send the wrist information to the Arduino, but this time, the angle will be calculated based on the fingers. The lines of code below shows the function that is written to calculate the wrist angle which will then be relayed to the Arduino for movements.
The wrist angle calculation in the most recent code is performed using the calculate_angle() function. This function takes three points as input: A, B, and C, which represent the coordinates of three landmarks: the wrist (point A), the middle finger (point B), and the pinky finger (point C).
To calculate the wrist angle, the function applies the cosine rule. The cosine rule is a mathematical formula used to determine the angle between two sides of a triangle when the lengths of all three sides are known. In this case, the function applies the cosine rule to the triangle formed by the line segments AB and BC.
By using the cosine rule, the function calculates the angle opposite the longest side, which corresponds to the angle at the wrist joint. This angle represents the rotation or flexion of the wrist. The calculated wrist angle is then sent to the Arduino to control the wrist servo motor. The Arduino code receives the angle value and adjusts the wrist servo accordingly to achieve the desired wrist movement.
In terms of the elbow movement, the editing of the TFLite google files within mediapipe to add a visual elbow landmark will take more time than what is allotted to get the project completed, therefore, in order to implement the elbow without having to edit these files, we are attempting to use some further calculations.
To approximate the movement of the elbow, we make an assumption that the position of the elbow lies alone the line of the wrist and middle finger landmarks. Based on this assumption, the code extracts the coordinates of the wrist and middle finger landmarks. Using these coordinates, the code estimates the position of the elbow by calculating the x and y coordinates of the estimated elbow position.
Once the estimated elbow position is determined, the code can analyze the relative movement of the elbow. For example, by comparing the x-coordinate of the estimated elbow with the x-coordinate of the wrist, the code can determine if the elbow is moving to the right or left. Similarly, by analyzing the y-coordinate, it can determine if the elbow is moving upward or downward. This information about the direction of elbow movement is then sent to the Arduino .
On the Arduino side, the code receives the data about the elbow movement and interprets it to control the elbow servo motor accordingly. The Arduino code can define specific angles or movements associated with each direction (e.g., up, down, left, right) and adjust the servo motor position accordingly. This also means that there will be two motors to control the elbow movements.
Note that instead of the Arduino looking for an angle to move the elbow, it should now be looking for a string output and then deciding how the servo should move, which once tested, will hopefully not cause any communication issues.
Reference used for cosine rule: https://www.revistaingenieria.unam.mx/numeros/2002/v03n4-03.pdf
Comments