Department Publications

Growing concerns over privacy invasion due to video camera based monitoring systems have made way to non-invasive Wi-Fi signal sensing based alternatives. This paper introduces a novel end-to-end deep learning framework that utilizes the changes in orthogonal frequency division multiplexing (OFDM) sub-carrier amplitude information to simultaneously predict the identity, activity and the trajectory of a user and create a user profile that is of similar utility to a one made through a video camera based approach. The novelty of the proposed solution is that the system is fully autonomous and requires zero user intervention unlike systems that require user originated initialization, or a user held transmitting device to facilitate the prediction. Experimental results demonstrate over 95% accuracy for user identification and activity recognition, while the user localization results exhibit a ±12cm error, which is a significant …

We propose self-supervised deep algorithms to detect anomalies in heterogeneous autonomous systems using frontal camera video and IMU readings. Given that the video and IMU data are not synchronized, each of them are analyzed separately. The vision-based system, which utilizes a conditional GAN, analyzes immediate-past three frames and attempts to predict the next frame. The frame is classified as either an anomalous case or a normal case based on the degree of difference estimated using the prediction error and a threshold. The IMU-based system utilizes two approaches to classify the timestamps; the first being an LSTM autoencoder which reconstructs three consecutive IMU vectors and the second being an LSTM forecaster which is utilized to predict the next vector using the previous three IMU vectors. Based on the reconstruction error, the prediction error, and a threshold, the timestamp is classified as either an anomalous case or a normal case. The composition of algorithms won runners up at the IEEE Signal Processing Cup anomaly detection challenge 2020. In the competition dataset of camera frames consisting of both normal and anomalous cases, we achieve a test accuracy of 94% and an F1-score of 0.95. Furthermore, we achieve an accuracy of 100% on a test set containing normal IMU data, and an F1-score of 0.98 on the test set of abnormal IMU data.

In this paper, we propose a novel algorithm based on supervised learning for antenna array extrapolation for the purpose of super resolution DoA estimation. We use multiple signal classification (MUSIC) as a DoA estimation technique to estimate the DoA. In order to reduce the computational burden, existing approaches focus on interpolating the missing elements in a virtual array using a sparse array (or non-uniform linear arrays) or employ antenna selection within the same aperture. In contrast, here we utilize an uniform linear array (ULA) using a low number of antennas (within a small array aperture) to extrapolate another ULA with a higher number of antennas (within a bigger aperture). This approach is not restricted to any specific antenna array configuration, in fact it can be generalized to any array configuration. We propose an algorithm which utilizes the advances of supervised learning (dictionary learning …

A combined approach of low-complexity light field depth filtering and deep learning is proposed for object classification in the presence of partial occlusions. The proposed approach exploits depth information embedded in multi-perspective four-dimensional (4-D) light fields via low-complexity 4-D sparse depth filtering and deep-learning. The proposed 4-D depth filter, designed using numerical optimization techniques by formulating as an ℓ 1 - ℓ ∞ minimization problem, is shown to outperform typical light field refocusing based on 4-D shift-sum averaging filters. Experiments conducted using a light field dataset acquired by a Lytro camera verify 45% and 27% better performance in terms of object classification accuracy compared to the cases when no depth filtering is employed and standard shift-sum refocusing is employed, respectively.

Reinforcement learning algorithms have been successfully trained for games like GO, Atari, and Chess in simulated environments. However, in cue sport-based games like Carrom, real world is unpredictable unlike in Chess and GO due to the stochastic nature of the gameplay as well as the effect of external factors such as friction combined with multiple collisions. Hence, solely training in a simulated platform for games like Billiard and Carrom, which need precise execution of a shot, would not be ideal in actual gameplay. This paper presents a real-time vision based efficient robotic system to play Carrom against a proficient human opponent. We demonstrate the challenges of adopting a reinforcement learning algorithm beyond simulations in implementing a strategic gameplay for the robotic system. We currently achieve an overall shot accuracy of 70.6% by combining heuristic and reinforcement learning …

Light field saliency detection—important due to utility in many vision tasks—still lack speed and can improve in accuracy. Due to the formulation of the saliency detection problem in light fields as a segmentation task or a “memorizing” tasks, existing approaches consume unnecessarily large amounts of computational resources for (training and) testing leading to execution times is several seconds. We solve this by aggressively reducing the large light-field images to a much smaller three-channel feature map appropriate for saliency detection using an RGB image saliency detector. We achieve this by introducing a novel convolutional neural network based features extraction and encoding module. Our saliency detector takes 0.4 s to process a light field of size 9× 9× 512× 375 in a CPU and is significantly faster than existing systems, with better or comparable accuracy. Our work shows that extracting features from light fields through aggressive size reduction and the attention results in a faster and accurate light-field saliency detector.