Department Publications

Geometric information of scenes available with four-dimensional (4-D) light fields (LFs) paves the way for post-capture refocusing. Light field refocusing methods proposed so far have been limited to a single planar or a volumetric region of a scene. In this letter, we demonstrate simultaneous refocusing of multiple volumetric regions in LFs. To this end, we employ a 4-D sparse finite-extent impulse response (FIR) filter consisting of multiple hyperfan-shaped passbands. We design the 4-D sparse FIR filter as an optimal filter in the least-squares sense. Experimental results confirm that the proposed filter provides 63% average reduction in computational complexity with negligible degradation in the fidelity of multi-volumetric refocused LFs compared to a 4-D nonsparse FIR filter.

Low-complexity signal processing algorithms and reconfigurable digital hardware architectures are proposed for multi-depth filtering and occlusion suppression in 4-D light fields (LFs). The proposed multi-depth-pass (MDP) and multi-depth-reject (MDR) 4-D filters allow simultaneous enhancement of planar objects at multiple desired depths and attenuation of planar objects at multiple undesired depths, thereby achieving electronically tunable multi-depth focusing. Partial-separability and recursive nature of the proposed MDP and MDR transfer functions ensure ultra-low hardware complexity suitable for real-time processing of large volume of input samples encountered in 4-D LFs. Filter synthesis details and examples with synthetic and real LFs of size 15  ×  15 × 434  ×  625 are presented. A generic framework encapsulating a multitude of possible digital hardware realizations is described, employing J …

Four-dimensional (4-D) light fields (LFs) enable novel imaging technologies, which are traditionally based on two-dimensional images. In most of these applications, denoising of LFs is required as a preprocessing technique before any subsequent processing. We propose a real-time LF denoising method using a novel 4-D linear and shift-invariant hyperfan filter. The proposed method exploits sparsity of the spectrum of a LF and the 4-D hyperfan filter is implemented in the 4-D mixed-domain (i.e.,two-dimensional space and two-dimensional frequency) leading to significant reductions in computational and memory complexities. A software implementation of the proposed method provides better or comparable denoising performance for grayscale and color LFs with respect to the metrics peak-signal-to-noise ratio (PSNR) and structural similarity (SSIM) compared to previously reported linear LF denoising methods, while …

Growing concerns over privacy invasion due to video camera based monitoring systems have made way to non-invasive Wi-Fi signal sensing based alternatives. This paper introduces a novel end-to-end deep learning framework that utilizes the changes in orthogonal frequency division multiplexing (OFDM) sub-carrier amplitude information to simultaneously predict the identity, activity and the trajectory of a user and create a user profile that is of similar utility to a one made through a video camera based approach. The novelty of the proposed solution is that the system is fully autonomous and requires zero user intervention unlike systems that require user originated initialization, or a user held transmitting device to facilitate the prediction. Experimental results demonstrate over 95% accuracy for user identification and activity recognition, while the user localization results exhibit a ±12cm error, which is a significant …

The potential of drone based services is enormous with applications ranging from consumer product delivery to health services. When meeting this demand, one of the major challenges we face is the noise produced by drones, which not only contributes to listener discomfort, but also hinders the device's ability to effectively communicate via audio. Thus, there exists a pressing need for understanding the characteristics of drone related noise, which can then be suppressed using suitable methods. This paper presents a preliminary study on modeling the relationship between input motor current and acoustic noise produced by a drone. An experimental study is conducted indoors for a drone under hovering manoeuvre with a single active motor and propeller. The drone noise was measured by a single on-board microphone. We identify multiple tones or harmonics in the drone noise spectrum that vary proportionally …

A low-complexity codec and a hardware architecture are proposed for achieving real-time compression of four-dimensional (4-D) light field (LF) signals captured from camera/lenslet arrays. The proposed system employs the 4-D extension of the two-dimensional (2-D) 8×8 approximate discrete cosine transform (ADCT) that has recently appeared in the literature. Motivated by the partial separability of the multidimensional spectrum of LFs, the proposed 4-D ADCT is obtained by cascading 2-D inter-view and 2-D intra-view transform stages. Software simulations are provided to confirm the performance of the 4-D ADCT based compression and comparisons are made with respect to 2-D inter-view only and 2-D intra-view only ADCT-based compression. Proposed digital architectures are validated using stepped hardware co-simulation on a Xilinx Virtex-7 VC-707 FPGA platform verifying 597 MHz maximum possible …

A combined approach of low-complexity light field depth filtering and deep learning is proposed for object classification in the presence of partial occlusions. The proposed approach exploits depth information embedded in multi-perspective four-dimensional (4-D) light fields via low-complexity 4-D sparse depth filtering and deep-learning. The proposed 4-D depth filter, designed using numerical optimization techniques by formulating as an ℓ 1 - ℓ ∞ minimization problem, is shown to outperform typical light field refocusing based on 4-D shift-sum averaging filters. Experiments conducted using a light field dataset acquired by a Lytro camera verify 45% and 27% better performance in terms of object classification accuracy compared to the cases when no depth filtering is employed and standard shift-sum refocusing is employed, respectively.

A minimax design for 2-D complex-coefficient FIR filters having asymmetric frequency responses is proposed in this paper. We consider the general form of 2-D FIR filters with low group delay and formulate the minimax design as a semidefinite programming problem. The 2-D linear-phase FIR filters with conjugate-symmetric coefficients are a special case of the proposed design. Example filter designs having near-equiripple magnitude responses are presented to verify the effectiveness of the proposed design method.

A moving object in a five-dimensional (5-D) light field video (LFV) can be selectively enhanced using the depth and the velocity of the object. In this paper, a 5-D depth-velocity (DV) filter is proposed to enhance multiple moving objects at different depths and with different velocities in an LFV. The 5-D DV filter is designed as a cascade of an infinite-extent impulse response multi-depth filter and a finite-extent impulse response multi-velocity filter. Experimental results obtained with numerically generated and real LFVs indicate that more than 15 dB improvement in signal-to-interference ratio can be achieved with the proposed 5-D multi DV filter compared to previously proposed multi depth-only filters.