Department Publications

Capsule Network is a promising concept in deep learning, yet its true potential is not fully realized thus far, providing sub-par performance on several key benchmark datasets with complex data. Drawing intuition from the success achieved by Convolutional Neural Networks (CNNs) by going deeper, we introduce DeepCaps, a deep capsule network architecture which uses a novel 3D convolution based dynamic routing algorithm. With DeepCaps, we surpass the state-of-the-art capsule domain networks results on CIFAR10, SVHN and Fashion MNIST, while achieving a 68% reduction in the number of parameters. Further, we propose a class independent decoder network, which strengthens the use of reconstruction loss as a regularization term. This leads to an interesting property of the decoder, which allows us to identify and control the physical attributes of the images represented by the instantiation parameters.

Many localized languages struggle to reap the benefits of recent advancements in character recognition systems due to the lack of substantial amount of labeled training data. This is due to the difficulty in generating large amounts of labeled data for such languages and inability of deep learning techniques to properly learn from small number of training samples. We solve this problem by introducing a technique of generating new training samples from the existing samples, with realistic augmentations which reflect actual variations that are present in human hand writing, by adding random controlled noise to their corresponding instantiation parameters. Our results with a mere 200 training samples per class surpass existing character recognition results in the EMNIST-letter dataset while achieving the existing results in the three datasets: EMNIST-balanced, EMNIST-digits, and MNIST. We also develop a strategy to …

Capsule networks excel in understanding spatial relationships in 2D data for vision related tasks. Even though they are not designed to capture 1D temporal relationships, with TimeCaps we demonstrate that given the ability, capsule networks excel in understanding temporal relationships. To this end, we generate capsules along the temporal and channel dimensions creating two temporal feature detectors which learn contrasting relationships. TimeCaps surpasses the state-of-the-art results by achieving 96.21% accuracy on identifying 13 Electrocardiogram (ECG) signal beat categories, while achieving on-par results on identifying 30 classes of short audio commands. Further, the instantiation parameters inherently learnt by the capsule networks allow us to completely parameterize 1D signals which opens various possibilities in signal processing.

Occlusion removal is an interesting application of image enhancement, for which, existing work suggests manually-annotated or domain-specific occlusion removal. No work tries to address automatic occlusion detection and removal as a context-aware generic problem. In this paper, we present a novel methodology to identify objects that do not relate to the image context as occlusions and remove them, reconstructing the space occupied coherently. The proposed system detects occlusions by considering the relation between foreground and background object classes represented as vector embeddings, and removes them through inpainting. We test our system on COCO-Stuff dataset and conduct a user study to establish a baseline in context-aware automatic occlusion removal.

Making a single network effectively address diverse contexts---learning the variations within a dataset or multiple datasets---is an intriguing step towards achieving generalized intelligence. Existing approaches of deepening, widening, and assembling networks are not cost effective in general. In view of this, networks which can allocate resources according to the context of the input and regulate flow of information across the network are effective. In this paper, we present Context-Aware Multipath Network (CAMNet), a multi-path neural network with data-dependant routing between parallel tensors. We show that our model performs as a generalized model capturing variations in individual datasets and multiple different datasets, both simultaneously and sequentially. CAMNet surpasses the performance of classification and pixel-labeling tasks in comparison with the equivalent single-path, multi-path, and deeper single-path networks, considering datasets individually, sequentially, and in combination. The data-dependent routing between tensors in CAMNet enables the model to control the flow of information end-to-end, deciding which resources to be common or domain-specific.

Application of depth-selective filtering to modulated-sparse light fields towards reducing the DSP and memory complexities in real-time light field processing is investigated. A modulated-sparse light field is obtained by spatially windowing an original 4-D light-field signal, which is subsequently processed by 4-D depth-selective filters to achieve planar focus and occlusion suppression. Multidimensional spectral properties of modulated-sparse light fields are explored and demonstrative examples with real light fields are provided to show that almost similar depth filtering and occlusion suppression performance can be achieved by processing such modulated-sparse light fields, leading to significant reduction in DSP hardware and memory complexities.

This paper presents a method for localizing an Unmanned Aerial Vehicle (UAV) using geo-referenced aerial images, in case of GPS failure. When flying in urban environments in low altitudes the inaccurate GPS localization has become a major problem. The proposed method fuses measurements from inertial sensors and visual odometer and perform image registration process using the images captured by the onboard down looking camera. A pre-built local orthomosaic map with geotagged aerial images is used for the registration. The problem of unreliable matches caused by the dynamic objects is addressed in our approach. The experiment results show that it is possible to localize the UAV in urban environments with a low margin of error by using our method.

This paper describes a hardware and software codesign approach for object detection and tracking algorithm which is based on modified ViBe background subtraction method and scale adaptable particle filtering algorithm. The modified ViBe performs with a close accuracy to the original work of ViBe but with less complexity and it outperforms the Mixture of Gaussian(MoG) method. Since the modified algorithm has a low off chip memory access requirement, the real time object detection in embedded level with the help of hardware acceleration techniques could be achieved. The implemented particle filtering algorithm is capable of adapting to the scale variation of the tracking objects. After careful investigation of the time consumption profile of each function in our algorithm, the complete algorithm was implemented in IoT edge devices targeted Avnet Ultra 96 development board which is based on Xilinx Zynq …

This paper has addressed the problem of transporting a long object from a given starting location to an end location using leader-follower robot formation in an environment filled with obstacles. Only the leader robot has the authority to make decisions. It possesses the main controller and is capable of navigating through an unknown environment using the data provided by on-board sensors. In the proposed system, a vision-based feedback method is combined with an explicit control method that oversights the virtual trailer between the leader and the follower. The parameters required to maintain a constant distance between the two robots are computed by the leader and transmitted to the follower directly. A fiducial marker is mounted on the front of the follower robot and is captured by a camera attached to the leader robot. Information exchanging methods based only on impedance control can be expensive …

This paper proposes a new method of estimation of the wet biomass of three floating aquatic plants, Spirodela polyrhiza, Lemna minor and Azolla pinnata. The method uses a homogeneity measure known as the J-value to detect the texture of the fronds of the plants from the uniform water surface. The proposed method is called J-value thresholding (JVT) and is highly accurate compared to the alternative green layer extraction (GLE). The average accuracy under normal illumination for the proposed method is 91.59% for Spirodela polyrhiza, 80.11% for Lemna minor and 77.4% for Azolla pinnata. Furthermore, it is robust to different illumination levels with the Pearson correlation coefficient of the estimates of different illumination levels being above 95%. When used to obtain the growth rate of each plant specie through linear regression, the result from the proposed method is highly consistent with the ground truth of …