Location-Aided Activity Recognition from Channel State Information With Deep Cross-Modal Learning

Due to the omnipresence of Wifi signals, the Channel State Information (CSI) can offer an alternate source to image, video, and other high-dimensional streams in a great many Internet-of-Things (IoT) applications. Specifically, in indoor applications where the movement of subjects cause classifiable effects, the CSI can provide utility with little to no burden on the current infrastructure. As a result an ever increasing number of Machine Learning researchers have proposed to use passive CSI data for applications ranging from surveillance to smart operations in living, healthcare, and office spaces. In this work, a new deep learning architecture is proposed that is able to take advantage of the spatial and temporal diversity in multi-antenna, multi-channel CSI for activity classification while implicitly aligning cross modal representations in order to improve prediction. Our experiments verify that by including location information as a secondary modality, the activity classification performance and generalizibility increase substantially based on where the subject is in multi-room settings.