have you ever tried waving your hand around like a magic wand and summoning a calculator? We would guess not considering that you’d probably look a little silly doing so. That is unless you had [Andrei’s] amazing gesture-controlled calculator. [Andrei] thought it would be valuable to use a calculator in his research lab without having to take his gloves off and the results are pretty cool.
His hardware consists of a PocketBeagle, an OLED, and an MPU6050 inertial measurement unit for capturing his hand motions using an accelerometer and gyroscope. The hardware is pretty straightforward, so the appeal of this project lies in its maker learning implementation.
[Andrei] first captured a few example datasets to train his algorithm by recreating the hand gestures for each number, 0-9, and recording the resulting accelerometer and gyroscope outputs. He processed the data first with a wavelet transform. The intent of the transform was two-fold. First, the transform allowed him to minimize the number of samples in his datasets while preserving the shape of the accelerometer and gyroscope signals, the crucial features in the maker learning classification. Secondly, he was able to increase the number of features for the classification considering that the wavelet transform resulted in both approximation and in-depth coefficients which can both be fed into the algorithm.
Because he had a small dataset, he used the Stratified Shuffle Split technique instead of the test train split method which is normally a lot more suited for larger datasets. The Stratified Shuffle Split made sure around the same number of train and test samples for each gesture. He was also very conscious of optimizing his model for running on a portable processing unit like the PocketBeagle. He spent some time optimizing the parameters of his algorithm and ultimately converted his model to a TensorFlowLite model using the built-in “TFLiteConverter” function within TensorFlow.
Finally, in true open-source fashion, all his code is available on GitHub, so feel complimentary to give it a go yourself. Calculatorium Leviosa!