Department Publications

Traffic light detection is essential for self-driving cars to navigate safely in urban areas. Publicly available traffic light datasets are inadequate for the development of algorithms for detecting distant traffic lights that provide important navigation information. We introduce a novel bench-mark traffic light dataset captured using a synchronized pair of narrow-angle and wide-angle cameras covering urban and semi-urban roads. We provide 1032 images for training and 813 synchronized image pairs for testing. Additionally, we provide synchronized test video pairs for qualitative analysis. The dataset includes images of resolution covering 10 different classes. Furthermore, we propose a post-processing algorithm for combining outputs from the two cameras. Results show that our technique can strike a balance between speed and accuracy, compared to the conventional approach of using a single camera frame …

Precision agriculture is one of the trending research areas in the world. AI technologies have been applied to greenhouse hydroponics agriculture (which comes under precision agriculture) to control the requirements of the plants in greenhouse agriculture. Apart from the environmental conditions such as temperature, humidity, and light intensity, the optimum combination of the fertilizer components, EC, and pH value provided during the plants’ growth phase is vital to obtaining a higher yield during the harvesting period. However, finding the optimal fertilizer composition is challenging due to the fact that the required fertilizer composition is different for different plant types and the experimental cost is higher. Sparse data samples will be received in data-driven experiments because of the large search space (number of permutations) of the fertilizer combinations. This paper presents an experimental design and AI …

In this paper, we address the problem of direction of arrival (DOA) estimation for multiple targets in the presence of sensor failures in a sparse array. Generally, sparse arrays are known with very high-resolution capabilities, where N physical sensors can resolve up to uncorrelated sources. However, among the many configurations introduced in the literature, the arrays that provide the largest hole-free co-array are the most susceptible to sensor failures. We propose here two machine learning (ML) methods to mitigate the effect of sensor failures and maintain the DOA estimation performance and resolution. The first method enhances the conventional spatial smoothing using deep neural network (DNN), while the second one is an end-to-end data-driven method. Numerical results show that both approaches can significantly improve the performance of MRA with two failed sensors. The data-driven method can …

Hearing-impaired listeners have a reduced ability to selectively attend to sounds of interest amid distracting sounds in everyday environments. This ability is not fully regained with modern hearing technology. A better understanding of the brain mechanisms underlying selective attention during speech processing may lead to brain-controlled hearing aids with improved detection and amplification of the attended speech. Prior work has shown that brain responses to speech, measured with magnetoencephalography (MEG) or electroencephalography (EEG), are modulated by selective attention. These responses can be predicted from the speech signal through linear filters called Temporal Response Functions (TRFs). Unfortunately, these sensor-level predictions are often noisy and do not provide much insight into specific brain source locations. Therefore, a novel method called Neuro-Current Response Functions …

A low-complexity multibeamforming scheme for uniform linear arrays is proposed based on warped and approximated discrete Fourier transform (DFT), albeit in the spatial domain. Conventional DFT- and approximated-DFT-based multibeamformers provide a low-complexity alternatives to produce multiple simultaneous beams with fixed beam profiles. The proposed approach aims to produce configurable (i.e., steerable) multibeam profiles using the warped DFT. Design examples are presented for an 8-element uniform linear array verifying, 81% and 18 % reduction in multiplier and adder complexities, respectively, when compared to a conventional narrowband multibeamformer having dedicated steering weights for each beam. Three system configurations are proposed using approximations for warped transform and the DFT. Hardware designs implemented on Xilinx Ultrascale KCU105 FPGA device verify 5.2 …

Direct policy learning (DPL) is a widely used approach in imitation learning for time-efficient and effective convergence when training mobile robots. However, using DPL in real-world applications is not sufficiently explored due to the inherent challenges of mobilizing direct human expertise and the difficulty of measuring comparative performance. Furthermore, autonomous systems are often resource-constrained, thereby limiting the potential application and implementation of highly effective deep learning models. In this work, we present a lightweight DPL-based approach to train mobile robots in navigational tasks. We integrated a safety policy alongside the navigational policy to safeguard the robot and the environment. The approach was evaluated in simulations and real-world settings and compared with recent work in this space. The results of these experiments and the efficient transfer from simulations to real-world settings demonstrate that our approach has improved performance compared to its hardware-intensive counterparts. We show that using the proposed methodology, the training agent achieves closer performance to the expert within the first 15 training iterations in simulation and real-world settings.

Neural networks are known to give better performance with increased depth due to their ability to learn more abstract features. Although the deepening of networks has been well established, there is still room for efficient feature extraction within a layer, which would reduce the need for mere parameter increment. The conventional widening of networks by having more filters in each layer introduces a quadratic increment of parameters. Having multiple parallel convolutional/dense operations in each layer solves this problem, but without any context-dependent allocation of input among these operations: The parallel computations tend to learn similar features making the widening process less effective. Therefore, we propose the use of multi-path neural networks with data-dependent resource allocation from parallel computations within layers, which also lets an input be routed end-to-end through these parallel …

Recent advancements in artificial intelligence have enabled reinforcement learning (RL) agents to exceed human-level performance in various gaming tasks. However, despite the state-of-the-art performance demonstrated by model-free RL algorithms, they suffer from high sample complexity. Hence, it is uncommon to find their applications in robotics, autonomous navigation, and self-driving, as gathering many samples is impractical in real-world hardware systems. Therefore, developing sample-efficient learning algorithms for RL agents is crucial in deploying them in real-world tasks without sacrificing performance. This paper presents an advisor-based learning algorithm, incorporating prior knowledge into the training by modifying the deep deterministic policy gradient algorithm to reduce the sample complexity. Also, we propose an effective method of employing an advisor in data collection to train autonomous navigation agents to maneuver physical platforms, minimizing the risk of collision. We analyze the performance of our methods with the support of simulation and physical experimental setups. Experiments reveal that incorporating an advisor into the training phase significantly reduces the sample complexity without compromising the agent’s performance compared to various benchmark approaches. Also, they show that the advisor’s constant involvement in the data collection process diminishes the agent’s performance, while the limited involvement makes training more effective.

This paper derives the optimum detection rule for communication systems with n-bit phase quantization when data is differentially encoded at the transmitter. The proposed approach avoids the channel estimation problem at the receiver. First, a maximum likelihood detection rule for block-2 detectors utilizing only two consecutive quantized observations at the channel output is obtained. Second, it is shown that the derived maximum likelihood detection rule continues to be optimum for the class of block-L detectors for L ≥ 3 when n = log 2 M, where M is the input alphabet size. Finally, utilizing the structure of the derived optimum detector, a message error probability expression is obtained for Rayleigh fading wireless channels. A simulation study is performed to illustrate the performance of the optimum detectors as well as the performance loss due to the lack of channel knowledge at the receiver. The proposed …

In this paper, we consider the problem of recovering speech from audio recordings of on-board microphones in a noisy drone platform. Enhancement of speech degraded by drone noise is considered to be a difficult task due to the strong noise generated from its motors and propellers causing an extremely low signal-to-drone noise ratio (SdNR). We propose a solution by (i) developing a multichannel Wiener filter (MWF) to remove drone noise from microphone recordings, and (ii) further reduction of residual noise using a Gaussian mixture model (GMM) based dual-stage parametric Wiener filter (WF). The method exploits the known statistics of motor current-specific drone noise. The theory developed is applicable to irregular microphone array embedded in a drone enabling realistic integration to most drones. We evaluate the proposed framework using two different drone acoustics datasets under extreme SdNR levels including− 30 dB. The experimental results confirm promising performance in terms of SdNR improvement, speech quality (PESQ), and intelligibility (STOI) and show a strong potential for speech enhancement applications using noisy drones.

Dynamic spectrum access is recognized as a potential solution to the problem of spectrum scarcity. Consequently, a significant amount of research has been conducted on spectrum allocation. In this study, we present a multi-traffic class queue-based spectrum allocation solution. The proposed solution is designed for a scenario where communication and other systems do not necessarily require agility in the frequency bands they operate in. The spectrum sensing and forecasting processes are carried out over relatively long time periods compared to the actual transmission rates. The study considers two traffic classes: priority and non-priority. Three packet handling strategies are presented and their average queuing delay performances are compared. While all three strategies yield satisfactory results under low/medium traffic conditions, Strategy 3 exhibits the greatest flexibility in terms of delay performance …

The optimal designs of multi-dimensional (M-D) complex-coefficient finite-extent impulse response (FIR) filters with low group delays predominantly consider only the minimization of the frequency response error. This leads to substantial deviation of the group delay from the desired constant group delay of such a filter, especially near the passband edges. In this paper, we propose a minimax design method for M-D complex-coefficient FIR filters with reduced group delay error. We incorporate constraints on phase response error while minimizing the frequency response error. We formulate the proposed minimax design method as a second-order cone programming problem. Design examples confirm that the proposed minimax design method significantly reduces the group delay error compared to previously proposed methods.

Broadband beamformers designed as two-dimensional (2-D) spatially-interpolated finite-extent impulse responses (FIR) filters, using a cascade structure, cannot employ sparse linear arrays despite significantly reducing the computational complexity. This happens due to the fact that the 2-D masking filter requires spatial samples equal to the spatial order at the input. Therefore, the output of the 2-D spatially-interpolated prototype filter should be computed for more than one spatial index. In order to address this limitation, we propose a 2-D spatially-interpolated FIR filter using a parallel structure. With the proposed structure, both 2-D spatially-interpolated prototype filter and the 2-D masking filter need to compute the output only for one spatial index, therefore allowing to employ sparse linear arrays. In order to support broadband beamforming, we design the 2-D FIR filter to have a parallelogram passband …

The use of approximate transforms in conjunction with approximate adder hardware is explored towards implementing low-complexity multi-beam antenna arrays. The use of approximate adders is motivated by the efficient look up table (LUT) usage on field programmable gate arrays (FPGA) resulting in better area and time complexities on hardware. 8-beam and 16-beam hardware architectures for multi-beamforming are designed using previously reported 8-point and 16-point spatial approximate-DFT (ADFT) algorithms, respectively, albeit with approximate adders. Designs have been implemented on Xilinx Kintex UltraScale KCU105 FPGA device verifying 31.5% reduction in (FPGA) LUT count and 58.08% improvement in critical path delay with a maximum beam error of 0.6 degrees and maximum peak side lobe level deviation 0.21 dB for the 8-beam case, when compared to ADFT with accurate adders with the …

Since its introduction, diffractive deep neural networks (D2NN) have been an emerging technology with many useful applications, such as 3D object recognition, saliency segmentation, and quantitative phase microscopy. However, fabricating D2NNs operating in the visible range has not been performed due to the complexities of fabricating nano-scale elements. Recent advancements such as Implosion Fabrication have made it possible to fabricate such networks using a discrete number of phase weights. We propose a quantization-aware training approach for D2NNs through modeling the quantization process in a differentiable manner using a sigmoid-based quantization function, facilitating the fabrication process. We also propose an efficient training schedule to guide the optimization process to converge to a better minima despite the limited number of quantization levels. Our method is simulated and …

Deep neural network models can learn clinically relevant features from millions of histopathology images. However generating high-quality annotations to train such models for each hospital, each cancer type, and each diagnostic task is prohibitively laborious. On the other hand, terabytes of training data–while lacking reliable annotations–are readily available in the public domain in some cases. In this work, we explore how these large datasets can be consciously utilized to pre-train deep networks to encode informative representations. We then fine-tune our pre-trained models on a fraction of annotated training data to perform specific downstream tasks. We show that our approach can reach the current state-of-the-art (SOTA) for patch-level classification with only 1-10% randomly selected annotations compared to other SOTA approaches. Moreover, we propose an uncertainty-aware loss function, to quantify the model confidence during inference. Quantified uncertainty helps experts select the best instances to label for further training. Our uncertainty-aware labeling reaches the SOTA with significantly fewer annotations compared to random labeling. Last, we demonstrate how our pretrained encoders can surpass current SOTA for wholeslide image classification with weak supervision. Our work lays the foundation for data and task-agnostic pretrained deep networks with quantified uncertainty.

Class Incremental Learning (CIL) is inspired by the human ability to learn new classes without forgetting previous ones. CIL becomes more challenging in real-world scenarios when the samples in each incremental step are imbalanced. This creates another branch of problem, called Generalized Class Incremental Learning (GCIL) where each incremental step is structured more realistically. Grow When Required (GWR) network, a type of Self-Organizing Map (SOM), dynamically creates and removes nodes and edges for adaptive learning. GWR performs incremental learning from feature vectors extracted by a Convolutional Neural Network (CNN), which acts as a feature extractor. The inherent ability of GWR to form distinct clusters, each corresponding to a class in the feature vector space, regardless of the order of samples or class imbalances, is well suited to achieving GCIL. To enhance GWR's classification …

Real-time and online action localization in videos poses a critical and formidable challenge. Achieving accurate action localization necessitates the integration of both temporal and spatial information. However, existing approaches rely on computationally intensive 3D convolutional neural network (CNN) architectures or redundant two-stream architectures with optical flow, rendering them unsuitable for real-time, online applications. To address this, we propose a novel approach that leverages fast and efficient key-point-based bounding box prediction for spatial action localization. Additionally, we introduce a tube-linking algorithm that ensures the temporal continuity of action tubes even in the presence of occlusions. By combining temporal and spatial information into a cascaded input for a single network, we eliminate the need for a two-stream architecture, enabling the network to effectively learn from both types …

Law enforcement officials heavily depend on Forensic Video Analytic (FVA) Software in their evidence extraction process. However present-day FVA software are complex, time consuming, equipment dependent and expensive. Developing countries struggle to gain access to this gateway to a secure haven. The term forensic pertains the application of scientific methods to the investigation of crime through post-processing, whereas surveillance is the close monitoring of real-time feeds. The principle objective of this Final Year Project was to develop an efficient and effective FVA Software, addressing the shortcomings through a stringent and systematic review of scholarly research papers, online databases and legal documentation. The scope spans multiple object detection, multiple object tracking, anomaly detection, activity recognition, tampering detection, general and specific image enhancement and video synopsis. Methods employed include many machine learning techniques, GPU acceleration and efficient, integrated architecture development both for real-time and postprocessing. For this CNN, GMM, multithreading and OpenCV C++ coding were used. The implications of the proposed methodology would rapidly speed up the FVA process especially through the novel video synopsis research arena. This project has resulted in three research outcomes Moving Object Based Collision Free Video Synopsis, Forensic and Surveillance Analytic Tool Architecture and Tampering Detection Inter-Frame Forgery. The results include forensic and surveillance panel outcomes with emphasis on video synopsis and Sri Lankan context. Principal …

Video synopsis, summarizing a video to generate a shorter video by exploiting the spatial and temporal redundancies, is important for surveillance and archiving. Existing trajectory-based video synopsis algorithms will not able to work in real time, because of the complexity due to the number of object tubes that need to be included in the complex energy minimization algorithm. We propose a real-time algorithm by using a method that incrementally stitches each frame of the synopsis by extracting object frames from the user specified number of tubes in the buffer in contrast to global energy-minimization based systems. This also gives flexibility to the user to set the threshold of maximum number of objects in the synopsis video according his or her tracking ability and creates collision-free summarized videos which are visually pleasing. Experiments with six common test videos, indoors and outdoors with many …

Since the dawn of its development the computer has been a staple of science fiction and thus mostly due to it humanity has constantly dreaded a day when intelligence stemming from computers commonly known as Artificial Intelligence (AI) would allegedly take over the world. The recent advent of advanced AI chatbot technology has again sparked that fear due to its ability to compose essays similar to that of a human being. This chapter discusses the concept of intelligence in biology and binary digital computing. Based upon the weaknesses of either, especially the hidden vulnerabilities of AI predicted by Gödel’s First Incompleteness Theorem and role of humans in programming AI, the question of whether it is likely to happen in the near future is discussed.

Traffic management is a serious problem in many cities around the world. Even the suburban areas are now experiencing regular traffic congestion. Inappropriate traffic control wastes fuel, time, and the productivity of nations. Though traffic signals are used to improve traffic flow, they often cause problems due to inappropriate or obsolete timing that does not tally with the actual traffic intensity at the intersection. Traffic intensity determination based on statistical methods only gives the average intensity expected at any given time. However, to control traffic accurately, it is required to know the real-time traffic intensity. In this research, image processing and machine learning have been used to estimate actual traffic intensity in real time. General-purpose electronic hardware has been used for in-situ image processing based on the edge-detection method. A deep neural network (DNN) was trained to infer traffic intensity in each image in real time. The trained DNN estimated traffic intensity accurately in 90% of the real-time images during road tests. The electronic system was implemented on a Raspberry Pi single-board computer; hence, it is cost-effective for large-scale deployment.

The escalating cost of VLSI packaging, driven by the proliferation of input and output pins and the growing design complexity, necessitates cost-effective solutions. This research paper explores a method to mitigate these costs by leveraging SerDes (Serializer/Deserializer) technology. We aim to reduce the number of I/O pins within internal chips that lack external signal requirements by employing open-source SerDes. We propose an integrated approach to incorporate SerDes into VLSI designs and validate its effectiveness through existing design cases.