Department Publications

The bag of visual words model has seen immense success in addressing the problem of image classification. Algorithms using this model generate the codebook of visual words by vector quantizing the features (such as SIFT) of the images to be classified. However, a codebook so formed tends to get biased by the nature of the dataset. In this paper we propose an alternative method to create the codebook for the dataset of images to be classified. Instead of directly using the dataset itself we first create a visual word dictionary by studying the SIFT features of a universal set of images. The codebook for the images to be classified is derived from this dictionary. To assess the effectiveness of the codebook thus derived, we classify the images using Probabilistic Latent Semantic Analysis in an unsupervised setting and Naive Bayes' classification in a supervised setting. The use of a dictionary achieves results …

Analysis of fertile material such as flowers and fruit is a key factor in the proper identification of plant species. Despite object recognition being a mature research area, the use of it in automated plant identification is still relatively new. This paper describes a novel method of detecting fertile material in plant images using rectangular features. Rectangular features are obtained for the grayscale image, the J value image, the magnitude and angle of the gradient of the image. From these, the features with the best performance are selected based on their ability to detect fertile material (flowers) and non-fertile material (leaves). Based on performance, the rectangular features of the grayscale image and J value image are used. Multiple Support Vector Machine (SVM) classifiers with different kernels are tested and the final result is obtained using classifier voting based on the confidence of each classifier. After applying the …

• Digital image processing helps us enhance images to make them visually pleasing, or accentuate regions or features of an image to better represent the content.

Real-time image processing demands much more processing power than a conventional processor can deliver. As a result hardware acceleration became necessary to augments processors with application-specific coprocessors. Due to the limited resources on FPGA and nature of some sequential algorithms, it is difficult to depend entirely on slice resources. In this research, we implemented a multiprocessor architecture to support real-time image processing on FPGA. Furthermore, we benchmarked and compared our implemented architectures with their counterparts. The operational structure of multiprocessor architecture consists of on-chip processors implemented in a parallel manner with efficient memory and bus architectures. The performance properties such as accuracy, throughput and efficiency are measured and presented. Multiprocessor systems are effective in software level parallelism on FPGA. Our …

In object tracking identifying the best feature which discriminates object and background improves the performance. Most of the existing methods do not consider the suitability of such features for the tracker. Here we enhance the discriminative features which elevate the tracker performance. To accommodate object and background variations over time we dynamically update the best feature using a distance measure. We demonstrate the performance of the resulting systems on the UNIVERSITÄT KARLSRUHE Image Sequences.

We saw that a given image may have a peaked histogram. Such a peaked or non-flat histogram shows that pixels do not equally occupy all the gray levels. This results in a low contrast image. We can remedy this by carrying out the operation called histogram equalization. Histogram equalization is a gray-level transformation that results in an image with a more or less flat histogram. Consider, for now, that continuous intensity values of an image are to be processed. We assume that r∈[0, L− 1]. Lets consider the intensity transformation s= T (r) 0≤ r≤ L− 1(3) that produces an output intensity level s for every pixel in the input image having intensity r. We assume that

Robust feature tracking is a requirement for many computer vision tasks such as indoor robot navigation. However, indoor scenes are characterized by poorly localizable features. As a result, indoor feature tracking without artificial markers is challenging and remains an attractive problem. We propose to solve this problem by constraining the locations of a large number of nondistinctive features by several planar homographies which are strategically computed using distinctive features. We experimentally show the need for multiple homographies and propose an illumination-invariant local-optimization scheme for motion refinement. The use of a large number of nondistinctive features within the constraints imposed by planar homographies allows us to gain robustness. Also, the lesser computation cost in estimating these nondistinctive features helps to maintain the efficiency of the proposed method. Our local …

A single-chip FPGA implementation of a vision core is an efficient way to design fast and compact embedded vision systems from the PCB design level. The scope of the research is to design a novel FPGA-based parallel architecture for embedded vision entirely with on-chip FPGA resources. We designed it by utilizing block-RAMs and IO interfaces on the FPGA. As a result, the system is compact, fast and flexible. We evaluated this architecture for several mid-level neighborhood algorithms using Xilinx Virtex-2 Pro (XC2VP30) FPGA. Our algorithm uses a vision core with a 100 MHz system clock which supports image processing on a low-resolution image of 128×128 pixels up to 200 images per second. The results are accurate. We have compared our results with existing FPGA implementations. The performance of the algorithms could be substantially improved by applying sufficient parallelism.

Let Z be a set of integers. The sampling process used to produce the digital image may be viewed as a partitioning in the xy-plane in to a grid, with coordinates of the center of each grid being a pair of elements from the Cartesian product Z2. In the terminology of set theory the function f (x, y) is said to be a digital image if (x, y) are integers from Z2 and f is a mapping that assigns an intensity value in the set of real numbers, R, to each distinct pair of coordinates (x, y). If the elements of R also are integers, a digital image then becomes a two-dimensional function, whose coordinates and the amplitude values are integers. Let A be a set in Z2, the elements of which are pixel coordinates (x, y). If w=(x, y) is an element of A, then we write w∈ A.

In this paper, we address the problem of extracting objects in an image that conform to prior object shape knowledge. Major challenges include the judgment of scale and rotation parameters as well as tolerating occlusions and noise. We propose the use of a novel log polar domain mapping of the Cartesian domain image to efficiently and effectively overcome these. This method greatly simplifies rotation, scaling and provides an opportunity to incorporate a decision threshold. Whilst, the initial theoretical framework is developed on binary images, placement of this into the gray scale domain is achieved by incorporating a pre-processing binary segmentation step. A post processing Cartesian domain energy optimization is done to counteract discripancies caused at the initial stage. We demonstrate our results using several examples.

• In some vision systems, the knowledge of the environment consist of a grid representation in which each object in the environment was represented by a 2D projection of its volume onto the horizontal plane. Such a representation is usually referred to as an “occupancy map”.

Computer vision based spectroscopy is feasible due to the advent of faster cameras and computers. Here, spectroscopy means detecting elements based on the intensity patterns of laser induced breakdown spectra seen by a camera. This is conventionally done by using specific instruments-spectroscopes-which too may use CCD devices as the sensors. Using a commercial camera reduces the cost of the product, while enabling the testing of new detection algorithms. A simulation setup is helpful in this context. The computer vision based spectroscopy simulation, which we propose here, involves displaying patterns on a computer screen corresponding to a reduced standard database of spectra, observing the screen using a camera, and detecting the elements, preferably on a different computer. We show that, using markers and computing homographies, correct alignment of the scene and image planes can be …

Simultaneous localization and map-building (SLAM) continues to draw considerable attention in the robotics community due to the advantages it can offer in building autonomous robots. It examines the ability of an autonomous robot starting in an unknown environment to incrementally build an environment map and simultaneously localize itself within this map. Recent advances in computer vision have contributed a whole class of solutions for the challenge of SLAM. This paper surveys contemporary progress in SLAM algorithms, especially those using computer vision as main sensing means, i.e., visual SLAM. We categorize and introduce these visual SLAM techniques with four main frameworks: Kalman filter (KF)-based, particle filter (PF)-based, expectation-maximization (EM)-based and set membership-based schemes. Important topics of SLAM involving different frameworks are also presented. This article …

In this work, we propose a framework that can provide performance metrics with regards to the usability of an assistive device designed to provide visual servoing of a reaching task performed by a visually impaired individual. The framework provides a model and methodology from which performance metrics can be synthesized based on robotic manipular kinematics and an adaptation of Fitts' law for reaching task motions in a 3D environment. The purpose of which is to facilitate an understanding of how well the system's design accommodates the user's natural movement style when performing the reaching task. Use of advances in visual tracking techniques employing scale, view-point, and illumination invariant features are incorporated to allow for a further decoupling of the feature space from the joint-level control space.

Detection of straight lines in an image is a fundamental requirement for many applications in computer vision. We formulate the straight line detection task as an energy minimization problem. This formulation helps the detection of lines in a global manner in contrast to the local detection methods used in conventional algorithms. As a result the proposed straight line detection algorithm can handle virtually co-located straight lines, slightly curved lines and edge linking in a unified manner. In addition, due to its the global nature, the algorithm is not deceived by image noise giving rise to spurious line segments. Therefore, the proposed algorithm can robustly detect straight lines. The main component of the algorithm is formulating the energy to be minimized. The contribution to this energy function is less at a pixel which is a good candidate to be a member of an existing line segment depending on the directional …

Monocular vision based robot navigation requires feature tracking for localization. In this paper we present a tracking system using discriminative features as well as less discriminative features. Discriminative features such as SIFT are easily tracked and useful to obtain the initial estimates of the transforms such as affinities and homographies. On the other hand less discriminative features such as Harris corners and manually selected features are not easily tracked in a subsequent frame due to problems in matching. We use SIFT features to obtain the the estimates of the planar homographies representing the motion of the major planar structures in the scene. Planar structure assumption is valid for indoor and architectural scenes. The combination of discriminative and less discriminative feature are tracked using the prediction by these homographies. Then normalized cross correlation matching is used to find the …

An autonomous mobile robot is an intelligent agent which explores an unknown environment with minimal human intervention. Building a relative map which describes the spatial model of the environment is essential for exploration by such a robot. Recent advances for robot navigation motivate mapping algorithms to evolve into simultaneous localization and map-building (SLAM). Initial uncertainty is one of the key factors in SLAM. An update scheme of the feature initialization in monocular vision based SLAM will be briefly introduced, which is within a detailed implementation of feature detection and matching, and 3-D reconstruction by multiple view geometry (MVG) within extended Kalman filter (EKF) framework. Experiments clearly show that the proposed scheme can maximize the optimization capacity of EKF.

This paper describes an optimal method of inserting new frames or recovering missing frames in a video sequence. The method is based on an optimization scheme using graph-cuts that finds the 'optimal' frames to be inserted in between two given frames. The core problem is a typical visual correspondence problem between pixels in two or more frames and having formulated the appropriate energy, graph-cuts can be used for optimization. The two frames are assumed to be 'close' and the motion of the objects is small. The motion is seen as a set of two dimensional disparities, and the graph-cuts based optimization is able to find these. Once the disparities are found, an intermediate frame can be trivially placed at an arbitrary position in between the two original frames. The advantage of using graph-cuts instead of the typical techniques used in calculating optical flow lies in the global nature of the graph-cuts …

In this paper the design and implementation of a camera platform that can mimic the neck and the eye movements of humans is presented. The camera platform is intended for active vision applications. It has seven mechanical degrees of freedom corresponding to the degrees of freedom of the human visual system

This paper presents our approach in building an image guided navigational tool for neurosurgeons. The paper emphasizes critical methods which we have implemented within the main modules of the system, namely physical space model, calibrated frame and model transformation, MRI space model and the GUI. The surgeon is provided with special tools which contain markers which are detected by cameras to compute the real time positions of the tool tip. Special plastic markers are attached to the patient's head for the registration between two sets of coordinate systems. We were able to obtain satisfactory accuracy when equipped with low quality web cameras. Our GUI provides a modeled three-dimensional viewing of the patients head with the capability of rotating around fixed axes.

The advent of low-cost, photo-quality inkjet printers has raised the need for an objective means of determining print quality that is consistent with what the end-user perceives. High-level quality metrics have been specified in the International Association for Standardization/International Electrotechnical Commission (ISO/IEC) guidelines on hardcopy print assessment [1]. These guidelines include metrics for four distinct categories of printed areas: line and character metrics, solid fill metrics, tint solid metrics, and background field metrics. Other metrics that enable the quantification of performance aspects relevant to inkjet printers have also been proposed. These aspects include color registration, color consistency, modulation transfer function (MTF), text quality, sharpness [2], dot quality, and line quality [3, 4]. Multiple efforts have been made to automate the process of image quality assessment both during product …

The structure and camera pose obtained using multiple view geometry based techniques cannot readily be used for robot localization and mapping. This is due to the fact that the structure and pose obtained relate to the actual environment and motion only up to a transform. In this paper, a method to localize the robot using monocular vision is presented. The assumptions are that the initial pose of the robot is known and that five or more landmarks (true, world points) can be identified. If two or more dissimilar views of at least five non coplanar feature points are initially available, subsequent robot locations with respect to the landmarks in view can be established. The exploration of the environment can then take place incorporating new feature points as the robot moves and successive images are acquired. The feature points which are no longer present in the field of view have to be handled along with the occluded …

In this work, we have taken the first step towards the creation of a computerized seeing-eye guide dog. The system we presented extends the development of assistive technology for the visually impaired into a new area: object tracking and visual servoing. The system uses computer vision to provide a kind of surrogate sight for the human user; sensing information from the environment and communicating it through haptic signalling. Our proof-of concept prototype is a low-cost wearable system which uses a colour camera to analyze a scene and recognize a desired object, then generate tactile cues to the wearer to steer his or her hand towards the object. We have proved the system in trials with random users in an unstructured environment.

Object shape and camera motion can be recovered from a sequence of images using a set of feature point correspondences. This is known as the structure from motion problem and the factorization method is the most widely applied solution due to its stability and robustness. This paper describes a method of employing geometrical features available in a scene, in the form of straight lines, in a factorizationbased structure from motion application. The effects of inaccuracies of feature data can be reduced by constraining the reconstructed features corresponding to the points forming straight lines. Our main contribution in this paper is the use of such geometric features to refine the shape recovery using the current advancements in the factorization method. Reconstructed features are mapped to straight lines and the measurement matrix containing image feature data is updated with the adjusted data. This increases …

Focus of attention is often attributed to biological vision system where the entire field of view is first monitored and then the attention is focused to the object of interest. We propose using a similar approach for object recognition in a color image sequence. The intention is to locate an object based on a prior motive, concentrate on the detected object so that the imaging device can be guided toward it. We use the abilities of the intelligent image analysis framework developed in our laboratory to generate an algorithm dynamically to detect the particular type of object based on the user's object description. The proposed method uses color clustering along with segmentation. The segmented image with labeled regions is used to calculate the shape descriptor parameters. These and the color information are matched with the input description. Gaze is then controlled by issuing camera movement commands as …