RADIO ANALYTICS FOR HUMAN SENSING, RECOGNITION AND DETECTION

Abstract

With the widespread deployment of WiFi devices in indoor environments such as homes, buildings, and vehicles, WiFi has evolved far beyond its traditional role as a communication medium. By leveraging its expanding radio frequency (RF) spectrum, WiFi has facilitated numerous sensing applications aimed at monitoring and improving various aspects of daily life. As indoor human sensing and recognition become increasingly critical for enhancing safety and efficiency, it is essential to explore practical applications of RF sensing. This dissertation contributes to this evolving field by focusing on three key areas: in-car child presence detection, residential occupancy detection, and advance human sensing through human imaging. In the first part of this dissertation, we propose a novel and robust in-car child presence detection (CPD) system utilizing commodity WiFi. The proposed CPD system comprises three key modules: a presence detection module, a seat occupancy detection module, and a child-adult classification module. The presence detection module incorporates motion and breathing detectors. To enhance breathing detection, we introduce a novel approach that treats intermediate spectrograms used for breathing estimation as images, applying image enhancement techniques followed by effective false alarm removal methods. The seat occupancy detection module identifies the occupied seat by calculating confidence scores that reflect the quality of the detected breathing signal. Finally, the child-adult classification module leverages a convolution neural network (CNN)-based model to accurately differentiate between scenarios involving only a child and other occupancy scenarios.Extensive experimental results demonstrate the robustness of the proposed system in real in-car environments and its strong generalization capability across different car models. In the second part of this dissertation, we introduce a residential human occupancy detection system that leverages existing WiFi infrastructure in homes to achieve extensive coverage. Our approach incorporates an environment-independent neural network architecture featuring a shared CNN backbone and a device-order-agnostic transformer block.The system utilizes readily available IoT devices in homes, such as smart plugs, bulbs, hubs, and speakers, to collect channel state information (CSI). The collected CSI is processed through a stack of preprocessing layers to compute the environment-independent feature, the autocorrelation function (ACF). These features are then passed to a deep learning module that outputs occupancy status—present or non-present—at one-second intervals. Our design is robust to variations in the subject’s location, orientation, and environmental dynamics. Additionally, it is adaptable to different numbers and types of devices, operating across various bands and carrier frequencies (2.4 GHz or 5 GHz) without compromising performance. This flexibility ensures compatibility with diverse residential setups, making the system highly practical and scalable. In the final part of this dissertation, mmWave imaging techniques are explored for two major applications: whole-body imaging and facial reconstruction. For whole-body imaging, a high-resolution neural network, HRNet, is introduced to reconstruct the human body as a 2D image. HRNet, based on a conditional generative adversarial network (cGAN) architecture, produces high-resolution human silhouette images that are useful for applications such as human identification. The system takes radar spatial spectrum data, generated using a modified MUSIC algorithm, as input, with Kinect images serving as ground truth during training. One of the key challenges in this task is the weak reflections from the human body, which can hinder the accuracy of image reconstruction. To address this, the cGAN architecture is enhanced with multiple loss functions during training, enabling the extraction of critical structural information about the human body. Experimental evaluations conducted using a commodity mmWave radar device operating at 60 GHz, involving twelve participants across three environments, demonstrate that this system can accurately reconstruct human images. For facial reconstruction, the HRNet design is further refined to achieve detailed imaging of human faces using mmWave radar. Modifications to the perceptual loss function and the introduction of a landmark-based loss function significantly improve the quality and detail of the generated face images. The landmark-based loss ensures structural consistency by accurately aligning key facial features such as the eyes, nose, and mouth. Additionally, a facial image refinement step is introduced to improve the accuracy without altering the entire face. To ensure the system generalizes effectively, data is collected from individuals with diverse facial alignments and features.