Department Publications

Activity recognition in videos in a deep-learning setting-or otherwise-uses both static and pre-computed motion components. The method of combining the two components, while keeping the burden on the deep network less, still remains uninvestigated. Moreover, it is not clear what the level of contribution of individual components is, and how to control the contribution. In this paper, we use a combination of convolutional-neural-network-generated static features and motion features in the form of motion tubes. We propose three schemas for combining static and motion components: based on a variance ratio, principal components, and Cholesky decomposition. The Cholesky-decomposition-based method allows the control of contributions. The ratio given by variance analysis of static and motion features matches well with the experimental optimal ratio used in the Cholesky decomposition-based method. The …

A 5-D depth–velocity filter is proposed for enhancing moving objects in noisy light field videos (LFVs) (also known as plenoptic videos). The proposed filter consists of an ultra-low complexity 5-D IIR depth filter and a 5-D FIR velocity filter. The 5-D IIR depth filter is employed to denoise a noisy LFV. The denoised LFV is then utilized to estimate the 3-D apparent velocity of the moving object of interest. The 5-D FIR velocity filter is designed based on the estimated 3-D apparent velocity and is used to enhance the moving object of interest while attenuating other interfering moving objects. Experimental results confirm the effectiveness of the proposed 5-D depth–velocity filter compared to previously reported 5-D depth–velocity filters.

An ultra-low complexity three-dimensional (3-D) linear-phase infinite-extent impulse response (IIR) velocity filter is proposed for attenuating moving artifacts in videos while enhancing objects moving on smooth approximately-linear trajectories. The proposed 3-D linear-phase IIR velocity filter consists of an ultra-low complexity 3-D wide-angle linear-phase IIR cone filter bank between two two-dimensional (2-D) spatial variable-shift filters. The ultra-low complexity 3-D wide-angle linear-phase IIR cone filter bank consists of a one-dimensional (1-D) temporal modified discrete Fourier transform filter bank and 2-D spatial allpass, IIR highpass and allstop filters. A typical 3-D linear-phase IIR velocity filter of order , applied to enhance a heavily corrupted test video signal, requires only 26 real multiplications and 60 real additions to process a sample and provides a scene-complexity-independent signal-to …

Purpose Automatic Optical Inspection (AOI) systems, used in electronics industry have been primarily developed to inspect soldering defects of Surface Mount Devices (SMD) on a Printed Circuit Board (PCB). However, no commercially available AOI system exists that can be integrated to a desktop soldering robotic system, which is capable of identifying soldering defects of Through Hole Technology (THT) solder joints along with the soldering process. In our research, we have implemented an AOI platform that is capable of performing automatic quality assurance of THT solder joints in a much efficient way. In this paper, we have presented a novel approach to identify soldering defects of THT solder joints, based on the location of THT component lead top. This paper presents the methodologies that can be used to precisely identify and localize THT component lead inside a solder joint. Design/methodology/approach We have discussed the importance of lead top localization and presented a detailed description on the methodologies that can be used to precisely segment and localize THT lead top inside the solder joint. Findings It could be observed that the precise localization of THT lead top makes the soldering quality assurance process more accurate. A combination of template matching algorithms and colour model transformation provide the most accurate outcome in localizing the component lead top inside solder joint, according to the analysis carried out in this paper. Research limitations/implications When the component lead top is fully covered by the soldering, the implemented methodologies will not be able to identify the actual …

A novel multidimensional spatio-temporal signal processing algorithm is proposed for depth-velocity filtering in 5-D light field videos along with a tutorial review of light field video spectra and the general architecture of associated 5-D depth-velocity filters. A 5-D FIR filter having a hyperfan-shaped passband which can selectively enhance the spectra of moving objects in light field videos based on their depth and motion trajectory is employed. A 5-D hyperfan-shaped passband is highly desired to achieve enhanced spatio-temporal selectivity over a wide range of lower frequencies. Experimental results are provided by using a light field videos captured using a Lytro light-field camera and mean structural similarity index is used to verify the effectiveness of the proposed 5-D FIR hyperfan depth-velocity filter.

A complex activity is a temporal composition of sub-events, and a sub-event typically consists of several low level micro-actions, such as body movement of different actors. Extracting these micro actions explicitly is beneficial for complex activity recognition due to actor selectivity, higher discriminative power, and motion clutter suppression. Moreover, considering both static and motion features is vital for activity recognition. However, optimally controlling the contribution from static and motion features still remains uninvestigated. In this work, we extract motion features at micro level, preserving the actor identity, to later obtain a high-level motion descriptor using a probabilistic model. Furthermore, we propose two novel schemas for combining static and motion features: Cholesky-transformation based and entropy-based. The former allows to control the contribution ratio precisely, while the latter obtains the optimal …

Voice commands have been used as the basic method of interaction between humans and robots over the years. Voice interaction is natural and require no additional technical knowledge. But while using voice commands humans frequently use uncertain information. In the case of object manipulation on a table, frequently used uncertain terms “Left,“Right”,“Middle”,“Front”... etc. These terms fail to depict an exact location on the table and the interpretation is governed by the robots point of view. Depending solely on vocal cues is not ideal as it requires the users to explain the exact location with more words and phrases making the interaction process cumbersome and less human like. However, using hand gestures to pinpoint the location is as natural as using the voice commands and frequently used when manipulating items on a surface. When compared to voice commands use of hand gestures is a more direct and less cumbersome approach. But when used alone hand gestures can result in errors while extracting the pointed location making the user dissatisfied. This paper proposes a multi-modal interaction method which uses hand gestures combined with voice commands to interpret uncertain information when placing an object on a table. Two fuzzy inference systems have been used to interpret the uncertain terms related to the two axes of the table. The proposed system has been implemented on an assistive robot platform. Experiments have been conducted to analyze the behaviour of the system.