Department Publications

Five-dimensional (5-D) light field video (LFV) (also known as plenoptic video) is a more powerful form of representing information of dynamic scenes compared to conventional three-dimensional (3-D) video. In this paper, the 5-D spectrum of an object in an LFV is derived for the important practical case of objects moving with constant velocity and at constant depth. In particular, it is shown that the region of support (ROS) of the 5-D spectrum is a skewed 3-D hyperfan in the 5-D frequency domain, with the degree of skew depending on the velocity and depth of the moving object. Based on this analysis, a 5-D depth-velocity digital filter to enhance moving objects in LFVs is proposed, described and implemented. Further, by means of the commercially available Lytro light-field camera, LFVs of real scenes are generated and used to test and confirm the performance of the 5-D depth-velocity filters for enhancing such …

We investigate the problem of automatic action recognition and classification of videos. In this paper, we present a convolutional neural network architecture, which takes both motion and static information as inputs in a single stream. We show that the network is able to treat motion and static information as different feature maps and extract features off them, although stacked together. We trained and tested our network on Youtube dataset. Our network is able to surpass state-of-the-art hand-engineered feature methods. Furthermore, we also studied and compared the effect of providing static information to the network, in the task of action recognition. Our results justify the use of optic flows as the raw information of motion and also show the importance of static information, in the context of action recognition.

A 5-D IIR depth-velocity filter is proposed for enhancing objects moving on linear trajectories, with constant velocity and at constant depth, in light field videos (also known as plenoptic videos). The passband of the proposed filter is a plane in the 5-D frequency domain and is realized by cascading three first-order 5-D IIR filters having 4-D hyperplanar passbands of appropriate orientations. Numerical simulation results are presented to confirm the performance of the proposed filter.

The spectrum of an object moving in 5-D light field videos (also known as plenoptic videos) with constant velocity and at non-constant depth is analyzed. Based on this analysis, a novel 5-D IIR adaptive depth-velocity filter is proposed for enhancing such objects. The time-variant coefficients of the proposed 5-D IIR adaptive depth-velocity filter are derived in closed form. Numerical simulation results confirm the effectiveness of the proposed filter compared to 3-D IIR adaptive velocity filters and 4-D IIR adaptive depth filters.

Five-dimensional (5-D) light field video (LFV) (also known as plenoptic video) is a more powerful form of representing information of dynamic scenes compared to conventional three-dimensional (3-D) video. In this dissertation, the spectra of moving objects in LFVs are analyzed, and it is shown that such moving objects can be enhanced based on their depth and velocity by employing 5-D digital filters, what is defined as depth-velocity filters. In particular, the spectral region of support (ROS) of a Lambertian object moving with constant velocity and at constant depth is shown to be a skewed 3-D hyperfan in the 5-D frequency domain. Furthermore, it is shown that the spectral ROS of a Lambertian object moving at non-constant depth can be approximated as a sequence of ROSs, each of which is a skewed 3-D hyperfan, in the 5-D continuous frequency domain. Based on the spectral analysis, a novel 5-D finite-extent impulse response (FIR) depth-velocity filter and a novel ultra-low complexity 5-D infinite-extent impulse response (IIR) depth-velocity filter are proposed for enhancing objects moving with constant velocity and at constant depth in LFVs. Furthermore, a novel ultra-low complexity 5-D IIR adaptive depth-velocity filter is proposed for enhancing objects moving at non-constant depth in LFVs. Also, an ultra-low complexity 3-D linear-phase IIR velocity filter that can be incorporated to design 5-D IIR depth-velocity filters is proposed. To the best of the author’s knowledge, the proposed 5-D FIR and IIR depth-velocity filters and the proposed 5-D IIR adaptive depth-velocity filter are the first such 5-D filters applied for enhancing moving objects in …

The massive volume of video and image data, compels them to be stored in a distributed file system. To process the data stored in the distributed file system, Google proposed a programming model named MapReduce. Existing methods of processing images held in such a distributed file system, requires whole image or a substantial portion of the image to be streamed every time a filter is applied. In this work an image filtering technique using MapReduce programming model is proposed, which only requires the image to be streamed only once. The proposed technique extends for a cascade of image filters with the constrain of a fixed kernel size. To verify the proposed technique for a single filter a median filter is applied on an image with salt and pepper noise. In addition a corner detection algorithm is implemented with the use of a filter cascade. Comparison of the results of noise filtering and corner detection …

When establishing object correspondence across non-overlapping cameras, the existing methods combine separate likelihoods of appearance and kinematic features in a Bayesian framework, constructing a joint likelihood to compute the probability of re-detection. A drawback of these methods is not having a proper approach to reduce the search space when localizing an object in a subsequent camera once the kinematic and appearance features are extracted in the current camera. In this work we introduce a novel methodology to condition the location of an object on its appearance and time, without assuming independence between appearance and kinematic features, in contrast to existing work. We characterize the linear movement of objects in the unobserved region with an additive Gaussian noise model. Assuming that the cameras are affine, we transform the noise model onto the image plane of …