Department Publications

Automatic stroke recognition of badminton video footages plays an important role in the process of analyzing players and building up statistics. Yet recognizing activities from broadcast videos is a challenging task due to person dependant body postures and blurring of the fast moving body parts. We propose a robust and an accurate approach for badminton stroke recognition using dense trajectories and trajectory aligned HOG features which are calculated inside local bounding boxes around players. A four-class SVM classifier is then used to classify badminton strokes to be either smash, forehand, backhand or other. This approach is robust to noisy backgrounds and provides accurate results for low resolution broadcast videos. Our experiments also reveal that this approach needs relatively fewer training samples for accurate recognition of strokes compared to existing approaches.

Vehicle tracking and surveillance is an area which is having a considerable attention in the context of security and safety. The detection and tacking of moving vehicles through multiple cameras is considered as a method of vehicle surveillance. This work addresses a problem of detecting and matching vehicles through multiple cameras. The power of GPUs are used to increase the number of video streams which can be processed using a single computer. In the detection process the Gabor filter is used as a directional filter and the SURF is used by the matcher to uniquely represent the vehicle.

Manual analysis of large volumes of video surveillance footage stemming from the widespread deployment of security cameras is error prone, expensive and time consuming. Despite the commercial availability of software for automated analysis, many products lack third party extensibility, the capability to perform simultaneous event detection and have no provision for anomaly detection in highly dense crowded scenes. We present a plugin based software system for video surveillance applications addressing these shortcomings and achieve realtime performance in typical crowded scenes. Core parameters are computed once per frame and shared among plugins to improve performance by eliminating redundant calculations. A novel multiple pedestrian tracking algorithm is incorporated into the framework to achieve the expected performance. We also propose an improvement to anomaly detection in densely …

Abnormal activity detection plays an important role in many areas such as surveillance, military installations, and sports. Existing abnormal activity detectors mostly rely on motion data obtained over a number of frames to characterize abnormality. However, only motion may not be able to capture all forms of abnormality, in particular, poses that do not amount to motion "outliers". In this paper, we propose two different spatio-temporal descriptors, a silhouette and optic flow based method and a dense trajectory based method which additionally include trajectory shape descriptor, to detect abnormalities. These two descriptors enable us to classify abnormal versus non-abnormal activities using SVM. Comparison with existing methods, using five standard datasets, shows that dense trajectory based method outperforms state-of-the-art results in crowd dataset and silhouette and optic flow based method outperforms …

Multi-view video has gained widespread popularity in the recent years. 3DTV, surveillance, immersive teleconferencing and free view-point television are few notable applications of multi-view video. Excessive storage and transmission bandwidth requirements are the major challenges faced by the industry in facilitating multi-view video applications. This paper presents efficient tools for coding of multi-view video based on the state of the art single view video coding standard H.265/HEVC (High Efficiency Video Coding). Our approach employs the LDI (Layered Depth Image) representation technique which is capable of compactly representing 3D scene content. We propose techniques and algorithms for LDI construction, view synthesis, efficient coding of LDI layers and associated auxiliary information. Subjective assessments indicate that our approach offers more than 50% reduction in bitrate compared to HEVC …

Action recognition in a video plays an important role in computer vision and finds many applications in areas such as surveillance, sports, and elderly monitoring. Existing methods mostly rely on stationary backgrounds. Action recognition in dynamic backgrounds typically requires standard preprocessing steps such as motion compensation, background modeling, moving object detection and object recognition. The errors of the motion compensation step and background modelling increase the mis-detections. Therefore action recognition in dynamic background is challenging. In this paper, we use a combination of pose characterized by a silhouette and optic flows synthesized into a histogram. This enables us to classify the movement of the actor versus movement of the background. We use four background models to extract the silhouette from the frame. We use SVM to recognize actions, according to several …

Typical Automatic Number Plate Recognition (ANPR) system uses high resolution cameras to acquire good quality images of the vehicles passing through. In these images, license plates are localized, characters are segmented, and recognized to determine the identity of the vehicles. However, the steps in this workflow will fail to produce expected results in low resolution images and in a less constrained environment. Thus in this work, several improvements are made to this ANPR workflow by incorporating intelligent heuristics, image processing techniques and domain knowledge to build an ANPR system that is capable of identifying vehicles even in low resolution video frames. Main advantages of our system are that it is able to operate in real-time, does not rely on special hardware, and not constrained by environmental conditions. Low quality surveillance video data acquired from a toll system is used to …

The natural sciences have been transformed with the incorporation of advanced technologies, and the current technological wave of change is also revolutionizing biological science. With the wide use of herbal medicines in traditional medical systems, the demand for medicinal plants has increased enormously. Though the current trend has made the use of medicinal plants more popular, we still require better methods to distinguish between medicinal plants and plants which do not possess medicinal value. Manual methods of plant recognition rely on a plants flower information. However, these manual processes of flower recognition are not straightforward for lay persons unless expert guidance is provided. Motivated by this reasoning, an intelligent flower analyzing system has been developed to recognize medicinal plants. Various tests have been performed on 160 images taken from four types of flowers, to recognize flowers based on their colour and shape features. The experiment was carried out using Support Vector Machine (SVM) and Principal Component Analysis (PCA) methodologies respectively for colour and shape extraction. Acquiring an average of 65% recognition rate implies that the applicability PCA and SVM in the specified domain is a valuable step forward.

Grouping the detected feature points traditionally requires the storage of long corner tracks. The traditional method does not permit to arrive at a decision to cluster the feature points based on a frame by frame basis. This paper presents a method to group the feature points directly into objects using the most recent 20 frames. The detected corner features are validated and clustered based on two approaches. When objects move in isolation, an EM algorithm is used to cluster and every object is detected and tracked. When objects move under partial occlusion, the corner features are clustered based on an agglomerative hierarchical clustering approach. A probabilistic framework has also been applied to determine the object level membership of the candidate corner features. A novel foreground estimation algorithm with an accuracy of 98% based on color information, background subtraction result and detected …

Human activity recognition finds many applications in areas such as surveillance, and sports. Such a system classifies a spatio-temporal feature descriptor of a human figure in a video, based on training examples. However many classifiers face the constraints of the long training time, and the large size of the feature vector. Our method, due to the use of an Support Vector Machine (SVM) classifier, on an existing spatio-temporal feature descriptor resolves these problems in human activity recognition. Comparison of our system with existing classifiers using two standard datasets shows that our system is much superior in terms of the computational time, and either it surpasses or is on par with the existing recognition rates. It performs on par or marginally inferior to existing systems, when the number of training examples are a few due to the imbalance, although consistently better in terms of computation time.

Feature descriptors have enabled feature matching under varying imaging conditions, while mostly being backed by experimental evidence. In addition to imposing some restrictions in imaging conditions needed to ensure matching, extending the existing descriptors is not straightforward due to the lack of sound mathematical bases. In this work, by using a surface bending versus shape histogram based on the principal curvatures, we are able to produce a descriptor which is not sensitive to the errors in dominant orientation assignment. Experimental evaluations show that our descriptor outperforms existing descriptors in the areas of viewpoint, rotation, scale, zoom, lighting and compression changes, with the exception of resilience to blur. Further, we apply this descriptor for accuracy demanding applications such as homography estimation and pose estimation. The experimental results show significant improvements in estimated homography and pose in terms of residual error and Sampson distance respectively.

Developing autonomous mobile robots that can coexist with human in populated environments is still considered a big challenge. To address this problem the authors propose a novel scheme to assist mobile robots by providing localization information externally. This scheme combines the autonomous navigation and target tracking research fields to arrive at a structured assistance system for autonomous mobile robots. In the proposed scheme, the environment is sensed using a laser range finder and a camera based sensor unit. Using the Rao-Blackwellized particle filter technique, the robots that need assistance are continuously tracked. In contrast with conventional laser range finder based tracking systems, the placement of the sensor is changed to a level above average human height and the mobile robots are modified by attaching a cylindrical pole. The experiments showed the validity of the proposed …

Motion Blur due to the relative motion between the camera and object can seriously degrade image quality. We present an FPGA based motion blur detection and correction algorithm which is implemented on top of a configurable soft-processor based architecture. The system consists of two main modules. The blur detection module identifies the blur length and angle, and the restoration module uses regularized inverse filtering to remove the motion blur. The Processing algorithms are implemented as separate cores on the FPGA fabric where the soft processor core is only used for managing system configuration. The system can achieve a frame rate of 15fps for a 720p HD video stream.

In this work we focus on the problem of detecting vehicles that come to a stop, in a video of approximately 15 frames per second. The videos are retrieved by surveillance cameras fixed as shown in Fig. 1 which are located few meters from the ground.

Increasing use of CCTV for city and building surveillance has given rise to an environment where an object (a person) might traverse through the field of vi>vv of many cameras. In this paper we explore the problem of tracking multiple objects in a multi camera environment, which is a highly addressed area in computer vision. Our research involves real time tracking of objects while they are moving in a multi camera environment with non-overlapping field of v/ewjr and detecting them when they re-uppear in the same or another camera in the same system. Previous methods of using offline trained classifiers with huge databases are time consuming and have the drawback of incapability of detecting arbitrarily selected objects. We address this issue by online training with the initial sample given and is based on the TLD (Tracking, Learning, Detection) framework. We extend the idea to formulate our methodology to …

In computer vision crass-correlation is a standard approach for local feature matching in feature tracking applications. For this Normalized cross-correlation (NCC) is implemented on spatial domain, because it does not have a simple and efficient frequency domain expression. Therefore in applications which demand real-time processing, a dedicated hardware implementation of NCC is essential to meet the computational cost in spatial domain. In this paper we present a Field Programmable Gate Array (FPGA) implementation of NCC based on the multi-port memory controller together with Fast Normalize Cross-correlation. Practical experimentation shows that our system can achieve frame rates closer to 100 for a search window size of 100x100 and template size of 15x15, with only using two dual-port memories.

Augmented Reality (AR) is one of the emerging technologies being used for enhancing user experience in many applications. Specific devices for various tasks are been replaced by Smart Phones making them a unique platform to implement all functionalities in one device. In this paper we present a real-time AR framework for mobile devices that takes into consideration key technical challenges such as limited processing power and battery life. The framework is having four separate modules, Marker Detection, Identification, Camera pose calculation and Embedding visual information. A 2D pattern called a marker is used to uniquely identify the objects. The marker in the camera view is detected and tracked so that information contained in the marker is exploited. Tracked four corners of the marker are used to calculate the 6 DOF of camera position, which is further processed to place 3D graphics on the real scene …

Traditional monoSLAM assumes stationary landmarks making it unable to cope up with dynamic environments where moving objects are present in the scene. This paper presents the parallel implementation of monoSLAM with a set of independent EKF trackers where stationary features and moving features are tracked separately. The difficult problem of detecting moving points from a moving camera is addressed by the epipolar constraint computed by using the measurement information already available with the monoSLAM algorithm. While doing so SLAM measurement outlier rejection is also performed. Results are presented to verify and highlight the advantages of our approach over traditional SLAM.

This paper outlines the problem of outdoor autonomous robot localization for agricultural operations. Vision based outdoor localization system is developed entirely of off-the-shelf components that can accurately guide a field robot in small agricultural fields. Visual odometry using downward faced camera is proposed as a high resolution relative localization system. A stereo vision based range measurement system is also developed and field tested as an absolute localization system that can bound the incremental error caused by the visual odometry system. The extended Kalman filter with measurement gating is implemented using both visual odometry and stereo range measurement data. Field tests demonstrated proposed system is capable of localization of two wheel tractor at a very low cost with acceptable accuracy in small agricultural fields.

As the problem of urban traffic congestion spreads, there is a pressing need for the introduction of advanced technology and equipment to improve the state of the art of traffic control. In this context, the techniques used for sensing traffic flow information plays a vital role, ensuring accurate real time controllability. Closed Circuit Television (CCTV) has gained popularity because of its ability to provide diverse information on relatively large regions leading to opportunities for performing substantially more complex tasks than conventional detection schemes. By processing these video images, traffic parameters such as speed, traffic composition, queue length etc. can be extracted. In addition CCTV images can be further processed for other useful information such as detection of vehicle shapes, vehicle types, occurrence of traffic violations and vehicle identification numbers etc. To introduce CCTV based vehicle …

One of the most challenging problems of field robots is self-localization, which involves incremental update of position while in motion. Though wheel based odometry is cheaper to implement its accuracy degrades when wheels slip. In this paper performance of low-cost visual odometry approach is experimented as a feasibility test for field robot localization. We have used a downward-facing camera and tested localization error in view of various parameters such as frame size, frame rate, etc. A FFT-based image registration techniques was utilized to determine the precise translation distances and heading between consecutive frames captured from ground surface. Basic navigation experiments, including a loop-closing test and error propagation were conducted and interesting numerical results have been reported.

In elephant management and conservation, it is vital to have non-invasive methods to track elephants. Image based recognition is a non-invasive mechanism for elephant tracking, albeit the inefficiency in manual method due to difficulties in handling large amount of data from multiple sources. To mitigate the drawbacks in manual method, we have proposed a computer vision based, automated, elephant recognition mechanism, which mainly relies on appearance based recognition algorithms. We have tested feasibility of the system running on a web based interface, which can facilitate researchers and conservationists all around the world to actively participate in elephant conservation.

In order to obtain depth perception in computer vision, it is needed to process pairs of stereo images. This process is computationally challenging to be carried out in real-time, because it requires the search for matches between objects in both images. Such process is significantly simplified if the images are rectified. Stereo image rectification involves a matrix transformation which when done in software will not produce real-time results although it is very demanding. Therefore, the video streaming and matrix transformation are not usually implemented in the same system. Our product is a stereo camera pair which produces a rectified real time image output with a resolution of 320×240 at a frame rate of 15FPS and delivers them via a 100-Ethernet interface. We use a Spartan 3E FPGA for real-time processing within which we implement an image rectification algorithm.

Position estimation using wheel odometric systems tends to give rather poor performances for an outdoor four-wheel skid steered mobile robot. Therefore autonomous control of these vehicles is extremely challenging in outdoor environments. This paper describes an outdoor localization system based on visual odometry for skid steered vehicle using forward faced camera and a downward faced camera. Optical flow field data is statistically analyzed to correctly estimate the position of the robot. Kalman Filtering is used to fuse data from two cameras for optimum performance. Also real-time Instantaneous Center of Rotation (ICR) detection using optical flow field data is proposed to calculate the heading angle. Two consumer grade cameras were used and algorithm was tested using open source image processing libraries. The proposed system yielded an acceptable positioning accuracy on short runs in typical …

Control the data flow between device interfaces, processing blocks and memories in a vision system is complex in hardware implementation. In the research, high-level synthesis tool is used to design, implement and test the vision system within the context of required control, synchronization, and parameterization on a processor based platform. In addition, both HLS tools and HDL were used for the development of the processing cores, and the performance of the two versions were analyzed and compared. The operational structures of benchmarked vision core consist of custom vision coprocessor with efficient memory and bus interfaces. The performance properties such as accuracy, throughput and efficiency are measured and presented. Xilinx XC5VLX110T FPGA, has been used for prototype the hardware platforms. According to results, without any complex optimizations, pipeline length and resource …