Hardware and Software
The hardware we use includes:
Raspberry Pi Model 4B as platform.
A very fast embedded system.
Philips Hue White Filament Regular A60 LED as controlled device.
Full-wireless controlled light bulb.
MLX90640 Thermal Camera as input device.
32*24 pixels thermal camera, samples at 0.5 - 64 Hz. Camera connection:
Camera Box
This is the simulation of the camera part, which can fix the camera on the plane to reduce the wrong judgment caused by hardware movement. The PCB is fixed on the bottom plate, then the pins are soldered on the PCB, the DuPont wire is connected, and the other end is connected to the Raspberry Pi.
Raspberry Pi OS for the whole application, simple Hue communication functions can be also compiled and run on Ubuntu (Tested on Ubuntu 20.04).
cURL for the HTTPS methods.
I2C Protocol for camera data transmission.
Qt for sensor UI.
cmake for build, test and package.
Pi has very fast processing speed as an embedded system, which our MLX90640 camera is connected with via I2C bus. The camera, in default mode, works at 4Hz sampling rate. We use event driven methods for the camera driver, after mlx90640->start(), everything initializes, and will start the mlx90640->run() loop, once the data of a full frame is collected in run(), the Qt callBack mywidget->hasValue(float*) will be triggered.
In Qt event hasValue(float*), the object in the MLX90640 camera will be checked if it is a human part object by detecting if the temperature is between 30-40 Celsius degrees, than we do some pixel math and position calculation of the object to see if it is a controlling signal, note that no machine learning is used during the gesture recognition.
If a controlling signal is confirmed, then a new thread of Hue json HTTP command will be allocated to control the Hue bulb wirelessly.