Department Publications

The design and hardware implementation of a low-complexity signal processing algorithm is proposed for real-time depth-velocity filtering in 5-D light field videos (LFVs). The proposed design is based on a stable 5-D infinite impulse response digital filter having three cascaded sections, each synthesized using the concept of multidimensional passive network resonance. A novel semi-systolic hardware implementation is proposed. Each section of the filter is implemented and tested on a Xilinx Virtex-7 FPGA platform using MATLAB-based hardware co-simulation with both synthetic and real LFV signals. A real-time processing throughput of 467 LFV frames/s is implied with each section of the filter operating at 204, 164, and 115 MHz for input LFV frames of size 9 × 9 × 220 × 360.

Light field signal processing allows manipulation of a rich set of information captured from a scene to achieve real-time depth filtering and occlusion suppression. Low-complexity four-dimensional infinite impulse response digital filters for simultaneous depth filtering and occlusion suppression over multiple depths in light fields are proposed. A low-complexity two-dimensional separable approach is employed to design the proposed filters having multiple frequency-planar pass-bands/stopbands in the four-dimensional spatial frequency domain, that can be electronically tuned to enhance/reject planar objects at multiple depths in a light field. Filter synthesis details are provided with specific design examples corresponding to 2-passband and 1-stopband cases. Numerically generated and Lytro camera captured light fields are used to verify the effectiveness of the proposed multi-depth-pass and multi-depth-reject …

A 3-D finite-extent impulse response (FIR) frustum filter having sparse coefficients is proposed for enhancing broadband plane waves (PWs) received by uniform aperture arrays. Motivated by the design technique of 1-D interpolated FIR filters, the proposed filter is realized by cascading a spatially-upsampled 3-D FIR prototype filter having a frustum-shaped passband and a 3-D FIR masking filter having a parallelepiped passband. Sparsity of the filter coefficients is achieved through the spatial upsampling of the coefficients of the prototype filter and by hard-thresholding the coefficients of both prototype and masking filters. The proposed 3-D FIR frustum filter saves 39.5% multiplications and 66.6% additions compared to the state-of-the-art 3-D FIR frustum filters without degrading the performance in enhancing broadband PWs.

This paper presents the implementation of a two-dimensional (2-D) finite-impulse-response (FIR) trapezoidal filter based beamforming array receiver. A 2.4 GHz 16-element receive-mode IQ array is designed and used for physically measuring the beam patterns corresponding to 2-D FIR trapezoidal filters. The 16-element beamformer is implemented using the reconfigurable open architecture computing hardware version-2 (ROACH-2) field programmable gate array (FPGA) platform to perform analog to digital conversion and digital signal processing. The 2-D FIR trapezoidal filter is designed as a 16-spatial input 32-tap filter and is implemented targeting the ROACH-2’s Xilinx Virtex 6 (sx475t) FPGA chip as a filter-and-sum architecture. The receiver array is precisely rotated (keeping the transmitter fixed) to digitally measure the received energy by integrating the instantaneous power of the output of the 2-D FIR …

A 5-D depth–velocity filter is proposed for enhancing moving objects in noisy light field videos (LFVs) (also known as plenoptic videos). The proposed filter consists of an ultra-low complexity 5-D IIR depth filter and a 5-D FIR velocity filter. The 5-D IIR depth filter is employed to denoise a noisy LFV. The denoised LFV is then utilized to estimate the 3-D apparent velocity of the moving object of interest. The 5-D FIR velocity filter is designed based on the estimated 3-D apparent velocity and is used to enhance the moving object of interest while attenuating other interfering moving objects. Experimental results confirm the effectiveness of the proposed 5-D depth–velocity filter compared to previously reported 5-D depth–velocity filters.

A low-complexity 2-D FIR trapezoidal filter is proposed for enhancing broadband plane waves. Motivated by the design technique of 1-D interpolated FIR filters, the proposed filter is realized by cascading a low-complexity spatiallyupsampled 2-D FIR filter having a trapezoidal passband and a low-complexity 2-D FIR masking filter having a parallelogram passband. Experimental results confirm that the proposed filter provides 74% reduction of the computational-complexity compared to conventional 2-D FIR fan and conventional 2-D FIR trapezoidal filters without deteriorating the performance in enhancing 2-D broadband plane waves distorted by strong interference and noise signals.

Screen content coding (SCC) extension to High Efficiency Video Coding (HEVC) offers substantial compression efficiency over the existing HEVC standard for computer generated content. However, this gain in compression efficiency is achieved at the expense of further computational complexity with several resource hungry coding tools. Hence, extension of SCC to HEVC hardware encoders can be challenging. This paper presents resource efficient hardware designs for two key SCC tools, Intra Block Copy and Palette Coding. Moreover, a new hash search approach is proposed for Intra Block Copy, while a hardware friendly palette indices coding scheme is suggested for Palette Coding. These designs are targeted to achieve the throughput necessary for an 1080p 30 frames/s encoder, and incurs coding loss of 11.4% and 5.1% respectively in all intra configurations. The designs are synthesized for a Virtex-7 …

An ultra-low complexity three-dimensional (3-D) linear-phase infinite-extent impulse response (IIR) velocity filter is proposed for attenuating moving artifacts in videos while enhancing objects moving on smooth approximately-linear trajectories. The proposed 3-D linear-phase IIR velocity filter consists of an ultra-low complexity 3-D wide-angle linear-phase IIR cone filter bank between two two-dimensional (2-D) spatial variable-shift filters. The ultra-low complexity 3-D wide-angle linear-phase IIR cone filter bank consists of a one-dimensional (1-D) temporal modified discrete Fourier transform filter bank and 2-D spatial allpass, IIR highpass and allstop filters. A typical 3-D linear-phase IIR velocity filter of order , applied to enhance a heavily corrupted test video signal, requires only 26 real multiplications and 60 real additions to process a sample and provides a scene-complexity-independent signal-to …

A novel multidimensional spatio-temporal signal processing algorithm is proposed for depth-velocity filtering in 5-D light field videos along with a tutorial review of light field video spectra and the general architecture of associated 5-D depth-velocity filters. A 5-D FIR filter having a hyperfan-shaped passband which can selectively enhance the spectra of moving objects in light field videos based on their depth and motion trajectory is employed. A 5-D hyperfan-shaped passband is highly desired to achieve enhanced spatio-temporal selectivity over a wide range of lower frequencies. Experimental results are provided by using a light field videos captured using a Lytro light-field camera and mean structural similarity index is used to verify the effectiveness of the proposed 5-D FIR hyperfan depth-velocity filter.

Higher compression efficiency in HEVC encoders comes with increased computational complexity, making real time encoding of high resolution videos a challenging task. This challenge can be addressed by software, yet hardware solutions are more appealing due to their superior performance and low power consumption. This paper presents an FPGA based hardware implementation of an all intra HEVC encoder, which can encode 8 bits per sample, 1920×1080 resolution, 30 frames per second raw video, that is viable in real time even at low operating frequencies. A major obstacle to real time encoding in available architectures is the dependency created by reference generation. Moreover, each coding unit (CU) has to be processed in multiple configurations to determine the most efficient split and prediction mode representation, based on the bit stream generated. We propose a new three stage architecture to …

Five-dimensional (5-D) light field video (LFV) (also known as plenoptic video) is a more powerful form of representing information of dynamic scenes compared to conventional three-dimensional (3-D) video. In this paper, the 5-D spectrum of an object in an LFV is derived for the important practical case of objects moving with constant velocity and at constant depth. In particular, it is shown that the region of support (ROS) of the 5-D spectrum is a skewed 3-D hyperfan in the 5-D frequency domain, with the degree of skew depending on the velocity and depth of the moving object. Based on this analysis, a 5-D depth-velocity digital filter to enhance moving objects in LFVs is proposed, described and implemented. Further, by means of the commercially available Lytro light-field camera, LFVs of real scenes are generated and used to test and confirm the performance of the 5-D depth-velocity filters for enhancing such …

A 5-D IIR depth-velocity filter is proposed for enhancing objects moving on linear trajectories, with constant velocity and at constant depth, in light field videos (also known as plenoptic videos). The passband of the proposed filter is a plane in the 5-D frequency domain and is realized by cascading three first-order 5-D IIR filters having 4-D hyperplanar passbands of appropriate orientations. Numerical simulation results are presented to confirm the performance of the proposed filter.

The spectrum of an object moving in 5-D light field videos (also known as plenoptic videos) with constant velocity and at non-constant depth is analyzed. Based on this analysis, a novel 5-D IIR adaptive depth-velocity filter is proposed for enhancing such objects. The time-variant coefficients of the proposed 5-D IIR adaptive depth-velocity filter are derived in closed form. Numerical simulation results confirm the effectiveness of the proposed filter compared to 3-D IIR adaptive velocity filters and 4-D IIR adaptive depth filters.

Five-dimensional (5-D) light field video (LFV) (also known as plenoptic video) is a more powerful form of representing information of dynamic scenes compared to conventional three-dimensional (3-D) video. In this dissertation, the spectra of moving objects in LFVs are analyzed, and it is shown that such moving objects can be enhanced based on their depth and velocity by employing 5-D digital filters, what is defined as depth-velocity filters. In particular, the spectral region of support (ROS) of a Lambertian object moving with constant velocity and at constant depth is shown to be a skewed 3-D hyperfan in the 5-D frequency domain. Furthermore, it is shown that the spectral ROS of a Lambertian object moving at non-constant depth can be approximated as a sequence of ROSs, each of which is a skewed 3-D hyperfan, in the 5-D continuous frequency domain. Based on the spectral analysis, a novel 5-D finite-extent impulse response (FIR) depth-velocity filter and a novel ultra-low complexity 5-D infinite-extent impulse response (IIR) depth-velocity filter are proposed for enhancing objects moving with constant velocity and at constant depth in LFVs. Furthermore, a novel ultra-low complexity 5-D IIR adaptive depth-velocity filter is proposed for enhancing objects moving at non-constant depth in LFVs. Also, an ultra-low complexity 3-D linear-phase IIR velocity filter that can be incorporated to design 5-D IIR depth-velocity filters is proposed. To the best of the author’s knowledge, the proposed 5-D FIR and IIR depth-velocity filters and the proposed 5-D IIR adaptive depth-velocity filter are the first such 5-D filters applied for enhancing moving objects in …

Two novel computationally efficient implementations for 3-D FIR cone filters are proposed in this paper. In the proposed implementations, the well-known 1-D quadrature mirror cosine modulated and 1-D directly designed temporal filter banks are cascaded with 2-D FIR circularly symmetric lowpass spatial filters to approximate the cone-shaped passband. Furthermore, the 3-D FIR frustum filters are derived from the proposed 3-D FIR cone filter implementations. The proposed implementations provide additional 2 dB signal-to-interference-and-noise ratio improvement compared to previously reported 3-D FIR cone and frustum filter implementations with reduced or equivalent computational complexity.

The inverse transform function of the HEVC Decoder has grown greatly in complexity with the addition of larger transform sizes and recent works have focused on efficient architectures that can achieve the required throughput. In this work we make the observation that a majority of the coefficients in a typical transform operation is zero, and therefore has no impact on the final outcome. We propose an architecture that can efficiently operate on such sparse matrices and introduce a scheduling strategy which completes a 2D IDCT with bare minimum iterations, with the added advantage of being able to integrate seamlessly with the entropy decoder without a coefficient reordering buffer. Experiments show that although the performance of this approach is scalable with the bit rate, a 120 MHz operating frequency is sufficient to handle QHD @ 48 Mbps, which is less than one third of the frequency requirement of prior work.

A low-complexity 3D FIR filter is proposed for selectively enhancing severely corrupted linear trajectory spatio-temporal signals. The proposed 3D FIR filter is separable and consists of a 3D spatio-temporal wide-angle FIR cone filter between two 2D spatial variable-shift filters. Low complexity of the 3D spatio-temporal wide-angle FIR cone filter is achieved by exploiting the spatial-symmetry of the passband and by employing maximal decimation in the temporal dimension. Compared to existing techniques, the proposed 3D FIR filter provides significant reduction of the computational complexity for similar SINR improvement.

A multibeam filter bank (MBFB) is proposed for recovering the broadband far-field signals received from multiple directions while rejecting interfering signals and noise. The MBFB consists of two sections. The first section employs variable fractional delays and independently steers the directions of the multiple beams. It achieves low complexity by using the Farrow structure. The second section achieves low complexity by maximal decimation methods and by exploiting the spatial symmetry of a 3-D double-frustum-shaped passband. Superior performance and lower complexity compared with previous methods are numerically confirmed for a typical narrow-angle broadband multibeam beamformer of the type that could be used in radio astronomy.

Multi-view video has gained widespread popularity in the recent years. 3DTV, surveillance, immersive teleconferencing and free view-point television are few notable applications of multi-view video. Excessive storage and transmission bandwidth requirements are the major challenges faced by the industry in facilitating multi-view video applications. This paper presents efficient tools for coding of multi-view video based on the state of the art single view video coding standard H.265/HEVC (High Efficiency Video Coding). Our approach employs the LDI (Layered Depth Image) representation technique which is capable of compactly representing 3D scene content. We propose techniques and algorithms for LDI construction, view synthesis, efficient coding of LDI layers and associated auxiliary information. Subjective assessments indicate that our approach offers more than 50% reduction in bitrate compared to HEVC …

A low-complexity multirate 3-D spatio-temporal FIR cone and frustum filter structure is proposed having potential applications as a spatio-temporal directional filter. The cone filter structure employs a 1-D modified discrete Fourier transform (DFT) filter bank and 2-D spatial filters. The frustum filter having a double-frustum-shaped passband oriented along the temporal frequency axis is approximated by employing an appropriate subset of subbands. Low computational complexity is achieved by maximal decimation in the temporal dimension, and by employing the DFT-polyphase realization to implement the 1-D modified DFT filter bank. The cone and frustum filters are almost alias free, and provide near-perfect reconstruction. The reduction in computational complexity, relative to undecimated and under-decimated realizations, is numerically confirmed by means of a potential application involving the attenuation of …

The short-term electricity demand forecasting has become one of the major research area in power system engineering. By combining the smart metering to the short-term demand forecasting techniques, new features can be added to save on demand and electricity bill. This paper illustrates the methodology used to forecast electricity demand over short period of time which can be used with smart meters. Polynomial fitting with interpolation is used to forecast the demand by taking the apparent power sample points from smart meters. The outcome of this work will be beneficial to the residential or industrial electricity consumers to control the demand side loads. It will help the industrial consumers to save on maximum demand charge with the introduction of warning message or residential consumers to reduce their electricity bill by cutting down non-essential loads in peak hours.

In this thesis, multidimensional signal processing techniques to mitigate disturbances in synthetic aperture systems such as radio telescopes are investigated. Here, two computationally efficient three-dimensional (3D) spatio-temporal (ST) finite impulse response (FIR) cone filter bank structures are proposed. Furthermore, a strategy is proposed to design 3D ST FIR frustum filter banks, having double-frustum-shaped passbands oriented along the temporal axis, derived from appropriate 3D ST FIR cone filter banks. Both types of cone and frustum filter banks are almost alias free and provide near-perfect reconstruction. In the proposed cone and frustum filter banks, both temporal and spatial filtering operations can be carried out at a significantly lower rate compared to previously reported 3D ST FIR cone filter banks implying lower power consumption. Furthermore, the proposed cone and frustum filter banks require a significantly lower computational complexity than previously reported 3D ST FIR cone and frustum filter banks. Importantly, this is achieved without deteriorating the improvement in signal-to-interference-plus-noise ratio. A theoretical analysis of brightness distribution (BD) errors caused by parameter perturbations and mismatches among the transfer functions of receivers employed in synthetic aperture systems is presented. First, the BD errors caused by perturbations in the transfer functions of low noise amplifiers (LNAs) and anti-aliasing filters (AAFs) are considered, and the characteristics of the additive BD error and its effects on synthesized BDs are thoroughly analyzed. Second, the conditions that should be satisfied by the transfer …

The paper presents a video transmission technique for surveillance systems deployed over wireless networks. In order to be integrated into modern wireless and IP based networks, video surveillance systems should possess the ability to deal with variable network quality. The developed technique allows the system to adaptively compensate for unpredictable changes in the network. A statistical method is used to predict parameters related to network quality, and accordingly vary the frame rate of the video stream transmitted.

It is shown that, by employing 2D FIR spatial filters and under-decimation, the computational time of 3D space-time polyphase cone filter banks may be significantly reduced. The resultant input-output 3D transfer function is almost alias-free and, further it is shown that the proposed cone filters can be used to effectively attenuate such stopband signals as radio frequency interference in radio astronomy applications.

The next generation of radio telescopes requires unprecedented levels of sensitivity, resolution, and field of view (FOV) that cannot be achieved with current radio telescopes. Radio astronomers and scientist are working on the development of next generation telescope designs, such as the square kilometre array (SKA)[1]. Depending on the observations, the signals of interest (SOIs) in the next generation of radio telescopes, such as the SKA, may span at least a decade of temporal frequency and are therefore broadband (BB). Moreover, the requirement of unprecedented levels of sensitivity makes radio frequency interference (RFI) a major problem. In achieving the science specifications, two emerging synthetic aperture applications of dense phased arrays are being considered [2],[3] for the so-called mid-frequency range from about 0.3-2 GHz: they are, rectangular dense aperture arrays (DAAs)(as in Fig. 1 (a)) and/or focal plane arrays (FPAs) in the focal plane of paraboloidal dishes (as in Fig. 1 (b)). Previously reported works on the mitigation of RFIs are based on temporal blanking techniques which are mostly suitable for intermittent RFI [2], on spatial filtering techniques which use subspace-projection methods [3]-[5], on cancellation techniques which use a reference signal approach [6], and on post-correlation cancellation and anti-coincidence techniques [7]. Such methods are mostly being investigated in the context of narrowband applications. However, for next generation synthetic aperture systems, such as the SKA, the SOIs are inherently BB and therefore it is an interesting possibility to consider a suitable BB approach capable for …