Gesture Recognition with mmWave Wi-Fi Access Points: Lessons Learned

Nabeel Nisar Bhat, Rafael Berkvens Jeroen Famaey
University of Antwerp-imec
IEEE 24th International Symposium on a World of Wireless, Mobile and Multimedia Networks (IEEE WoWMoM),2023
Paper Supplementary Code arXiv
-->

Abstract

In recent years, channel state information (CSI) at sub-6 GHz has been widely exploited for Wi-Fi sensing, particularly for activity and gesture recognition. In this work, we instead explore mmWave (60 GHz) Wi-Fi signals for gesture recognition/pose estimation. Our focus is on the mmWave WiFi signals so that they can be used not only for high data rate communication but also for improved sensing e.g., for extended reality (XR) applications. For this reason, we extract spatial beam signal-to-noise ratios (SNRs) from the periodic beam training employed by IEEE 802.11ad devices. We consider a set of 10 gestures/poses motivated by XR applications. We conduct experiments in two environments and with three people. As a comparison, we also collect CSI from IEEE 802.11ac devices. To extract features from the CSI and the beam SNR, we leverage a deep neural network (DNN). The DNN classifier achieves promising results on the beam SNR task with stateof-the-art 96.7% accuracy in a single environment, even with a limited dataset.

Key Takeaways

  • Gesture recognition using cheap commercial off-the-shelf mmWave Wi-Fi access points.
  • Beam SNR as a signal feature extracted from beam sweeping, part of the IEEE 802.11ad standard.
  • Unlike CSI or CIR, beam SNR can be accessed without additional overhead as it is required to be reported in sector sweep frames.
  • Diverse range of gestures and poses considered, including head rotations.
  • Low-sampling frequency of beam SNR, not misemploying the concept of ISAC, occasional sensing. No effect on communication.
  • 96.7% accuracy in the best case using mmWave beam SNR.

BibTeX

@inproceedings{bhat2023gesture,
  title={Gesture recognition with mmWave Wi-Fi access points: Lessons learned},
  author={Bhat, Nabeel Nisar and Berkvens, Rafael and Famaey, Jeroen},
  booktitle={2023 IEEE 24th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM)},
  pages={127--136},
  year={2023},
  organization={IEEE}
}
}