Department Publications

Five-dimensional (5-D) light field video (LFV) (also known as plenoptic video) is a more powerful form of representing information of dynamic scenes compared to conventional three-dimensional (3-D) video. In this paper, the 5-D spectrum of an object in an LFV is derived for the important practical case of objects moving with constant velocity and at constant depth. In particular, it is shown that the region of support (ROS) of the 5-D spectrum is a skewed 3-D hyperfan in the 5-D frequency domain, with the degree of skew depending on the velocity and depth of the moving object. Based on this analysis, a 5-D depth-velocity digital filter to enhance moving objects in LFVs is proposed, described and implemented. Further, by means of the commercially available Lytro light-field camera, LFVs of real scenes are generated and used to test and confirm the performance of the 5-D depth-velocity filters for enhancing such …

A 5-D IIR depth-velocity filter is proposed for enhancing objects moving on linear trajectories, with constant velocity and at constant depth, in light field videos (also known as plenoptic videos). The passband of the proposed filter is a plane in the 5-D frequency domain and is realized by cascading three first-order 5-D IIR filters having 4-D hyperplanar passbands of appropriate orientations. Numerical simulation results are presented to confirm the performance of the proposed filter.

The spectrum of an object moving in 5-D light field videos (also known as plenoptic videos) with constant velocity and at non-constant depth is analyzed. Based on this analysis, a novel 5-D IIR adaptive depth-velocity filter is proposed for enhancing such objects. The time-variant coefficients of the proposed 5-D IIR adaptive depth-velocity filter are derived in closed form. Numerical simulation results confirm the effectiveness of the proposed filter compared to 3-D IIR adaptive velocity filters and 4-D IIR adaptive depth filters.

Five-dimensional (5-D) light field video (LFV) (also known as plenoptic video) is a more powerful form of representing information of dynamic scenes compared to conventional three-dimensional (3-D) video. In this dissertation, the spectra of moving objects in LFVs are analyzed, and it is shown that such moving objects can be enhanced based on their depth and velocity by employing 5-D digital filters, what is defined as depth-velocity filters. In particular, the spectral region of support (ROS) of a Lambertian object moving with constant velocity and at constant depth is shown to be a skewed 3-D hyperfan in the 5-D frequency domain. Furthermore, it is shown that the spectral ROS of a Lambertian object moving at non-constant depth can be approximated as a sequence of ROSs, each of which is a skewed 3-D hyperfan, in the 5-D continuous frequency domain. Based on the spectral analysis, a novel 5-D finite-extent impulse response (FIR) depth-velocity filter and a novel ultra-low complexity 5-D infinite-extent impulse response (IIR) depth-velocity filter are proposed for enhancing objects moving with constant velocity and at constant depth in LFVs. Furthermore, a novel ultra-low complexity 5-D IIR adaptive depth-velocity filter is proposed for enhancing objects moving at non-constant depth in LFVs. Also, an ultra-low complexity 3-D linear-phase IIR velocity filter that can be incorporated to design 5-D IIR depth-velocity filters is proposed. To the best of the author’s knowledge, the proposed 5-D FIR and IIR depth-velocity filters and the proposed 5-D IIR adaptive depth-velocity filter are the first such 5-D filters applied for enhancing moving objects in …

Two novel computationally efficient implementations for 3-D FIR cone filters are proposed in this paper. In the proposed implementations, the well-known 1-D quadrature mirror cosine modulated and 1-D directly designed temporal filter banks are cascaded with 2-D FIR circularly symmetric lowpass spatial filters to approximate the cone-shaped passband. Furthermore, the 3-D FIR frustum filters are derived from the proposed 3-D FIR cone filter implementations. The proposed implementations provide additional 2 dB signal-to-interference-and-noise ratio improvement compared to previously reported 3-D FIR cone and frustum filter implementations with reduced or equivalent computational complexity.

The inverse transform function of the HEVC Decoder has grown greatly in complexity with the addition of larger transform sizes and recent works have focused on efficient architectures that can achieve the required throughput. In this work we make the observation that a majority of the coefficients in a typical transform operation is zero, and therefore has no impact on the final outcome. We propose an architecture that can efficiently operate on such sparse matrices and introduce a scheduling strategy which completes a 2D IDCT with bare minimum iterations, with the added advantage of being able to integrate seamlessly with the entropy decoder without a coefficient reordering buffer. Experiments show that although the performance of this approach is scalable with the bit rate, a 120 MHz operating frequency is sufficient to handle QHD @ 48 Mbps, which is less than one third of the frequency requirement of prior work.