Radio Analytics for Human Computer Interaction

Abstract

WiFi, as we know it, is no more a mere means of communication. Recent advances in research and industry have unleashed the sensing potential of wireless signals. With the constantly expanding availability of the radio frequency spectrum for WiFi, we now envision a future where wireless communication and sensing systems co-exist and continue to facilitate human lives. Radio signals are currently being used to ``sense" or monitor various human activities and vital signs. As Human-Computer Interaction (HCI) continues to form a considerable part of daily activities, it is interesting to investigate the potential of wireless sensing in designing practical HCI applications. This dissertation aims to study and design three different HCI applications, namely, (i) In-car driver authentication, (ii) Device-free gesture recognition through the wall, and (iii) Handwriting tracking by leveraging the radio signals. In the first part of this dissertation, we introduce the idea of in-car driver authentication using wireless sensing and develop a system that can recognize drivers automatically. The proposed system can recognize humans by identifying the unique radio biometric information embedded in the wireless channel state information (CSI) through multipath propagation. However, since the environmental information is also captured in the CSI, radio biometric recognition performance may be degraded by the changing physical environment. To this end, we address the problem of ``in-car changing environments” where the existing wireless sensing-based human identification system fails. We build a long-term driver radio biometric database consisting of radio biometrics of multiple people collected over two months. Machine learning (ML) models built using this database make the proposed system adaptive to new in-car environments. The performance of the in-car driver authentication system is shown to improve with extending multi-antenna and frequency diversities. Long-term experiments demonstrate the feasibility and accuracy of the proposed system. The accuracy achieved in the two-driver scenario is up to 99.13% for the best case compared to 87.7% achieved with the previous work. In the second part, we propose GWrite, a device-free gesture recognition system that can work in a through-the-wall scenario. The sequence of physical perturbations induced by the hand movement influences the multipath propagation and reflects in the CSI time series corresponding to the gesture. Leveraging the statistical properties of the EM wave propagation, we derive a relationship between the similarity of CSIs within the time series and the relative distance moved by the hand. Feature extraction modules are built on this relation to extract features characteristic of the gesture shapes. We built a prototype of GWrite on commercial WiFi devices and achieved a classification accuracy of 90.1% on a set of 15 gesture shapes consisting of the uppercase English alphabets. We demonstrate that a broader set of gestures could be defined and classified using GWrite as opposed to the existing systems that operate over a limited gesture set. In the final part of this dissertation, we present mmWrite, the first high-precision passive handwriting tracking system using a single commodity millimeter wave (mmWave) radio. Leveraging the short wavelength and large bandwidth of 60 GHz signals and the radar-like capabilities enabled by the large phased array, mmWrite transforms any flat region into an interactive writing surface that supports handwriting tracking at millimeter accuracy. mmWrite employs an end-to-end pipeline of signal processing to enhance the range and spatial resolution limited by the hardware, boost the coverage, and suppress interference from backgrounds and irrelevant objects. Experiments using a commodity 60 GHz device show that mmWrite can track a finger/pen with a median error of 2.8 mm close to the device and thus can reproduce handwritten characters as small as 1 cm X 1 cm, with a coverage of up to 8 m^2 supported. With minimal infrastructure needed, mmWrite promises ubiquitous handwriting tracking for new applications in HCI.