We humans inhabit a world of sight, sound, smell, taste, and touch. We cannot directly perceive the electromagnetic fields powering many of our modern technologies without specialized tools. Technologies such as cameras and LiDAR enhance our visual perception but are limited to clear lines of sight. Medical imaging technologies have successfully overcome some line-of-sight constraints but are often limited by factors like high power consumption, large size, and substantial costs. Motivated by these limitations, we explored whether the relatively low frequency radio waves from an affordable, consumer-grade SDR (the LimeSDR Mini 2.0) could detect hand gestures and poses through occlusions like walls by leveraging the inherent properties of radio waves in the 2–4 GHz range, which experience lower attenuation through common building materials compared to higher-frequency waves.

More specifically, we used amplitude and phase data from a 3.4 GHz Linear Frequency Modulated Continuous Wave (LFMCW) carrier signal with ultra-low bandwidth (500 KHz) to train two 1D Convolutional Neural Networks: RF-HandMark, that estimates hand poses using 21 hand landmarks and RF-Gesture, which classifies three distinct hand gestures with 87% test accuracy across three occlusion levels:

• Clear line of sight
• A 0.75 cm foam board
• A 4.5 cm wood block

RF-Gesture architecture (130k parameters)

RF-HandMark architecture (4 M parameters)

Even though the low frequency / low bandwidth combination reduces the granularity of the information in the return signal from a theoretical standpoint, we found that modern neural network architectures are capable of extracting meaningful patterns from these limited-resolution signals, enabling accurate gesture and pose recognition even through occlusions. This demonstrates the potential of SDRs as cost-effective, energy-efficient, and accessible alternatives to traditional high-frequency, multi-antenna radar systems for gesture and pose recognition applications.

Future research could explore the effectiveness of similar systems at higher ranges, or with different frequencies. A system that could detect human gestures and full body landmarks at a range of 10-20 meters through multiple occlusions would be a very significant achievement. From a theory standpoint, we speculate that such a system is possible.

For more information, take a look at our video:

Adapted (by a human) from an upcoming paper by Ben Pearman and Anand Kumar, University of Calgary: “LoRF-Ha: Hand Pose Estimation and Gesture Detection Through Occlusions Using A Consumer SISO SDR.” Forthcoming, 2025.