Department Publications

Computational optical imaging (COI) systems leverage optical coding elements (CEs) in their setups to encode a high-dimensional scene in a single or in multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches rely on random patterns or analytical designs to set distribution of the CE. However, the available data and algorithm capabilities of deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical and computational decoders. Specifically, by modeling the COI measurements through a fully differentiable image-formation model that considers the physics-based propagation of light and its interaction with the CEs, the parameters that define the CE and the …

Sleep plays a crucial role in the well-being of human lives. Traditional sleep studies using Polysomnography are associated with discomfort and often lower sleep quality caused by the acquisition setup. Previous works have focused on developing less obtrusive methods to conduct high-quality sleep studies, and ear- EEG is among popular alternatives. However, the performance of sleep staging based on ear-EEG is still inferior to scalp- EEG based sleep staging. In order to address the performance gap between scalp-EEG and ear- EEG based sleep staging, we propose a cross-modal knowledge distillation strategy 1 † https://github.com/Mithunjha/EarEEG_KnowledgeDistillation, which is a domain adaptation approach. We employ model architectures from the transformer and convolutional neural network families to demonstrate the model-agnostic nature of the method. Our experiments and analysis validate the …

This paper presents an effective method of identifying elephant rumbles in infrasonic seismic signals. The design and implementation of electronic circuitry to amplify, filter, and digitize the seismic signals captured through geophones are presented. A collection of seismic infrasonic elephant rumbles was collected at a free-ranging area of an elephant orphanage in Sri Lanka. The seismic rumbles were converted to spectrograms, and several methods were used for spectral feature extraction. Using LasyPredict, the features extracted using different methods were fed into their corresponding machine-learning algorithms to train them for automatic seismic rumble identification. It was found that the Mel frequency cepstral coefficient (MFCC) together with the Ridge classifier machine learning algorithm produced the best performance in identifying seismic elephant rumbles. A novel method for denoising the spectrum that leads to enhanced accuracy in identifying seismic rumbles is also presented.

Precision agriculture is one of the trending research areas in the world. AI technologies have been applied to greenhouse hydroponics agriculture (which comes under precision agriculture) to control the requirements of the plants in greenhouse agriculture. Apart from the environmental conditions such as temperature, humidity, and light intensity, the optimum combination of the fertilizer components, EC, and pH value provided during the plants’ growth phase is vital to obtaining a higher yield during the harvesting period. However, finding the optimal fertilizer composition is challenging due to the fact that the required fertilizer composition is different for different plant types and the experimental cost is higher. Sparse data samples will be received in data-driven experiments because of the large search space (number of permutations) of the fertilizer combinations. This paper presents an experimental design and AI …

In this paper, we address the problem of direction of arrival (DOA) estimation for multiple targets in the presence of sensor failures in a sparse array. Generally, sparse arrays are known with very high-resolution capabilities, where N physical sensors can resolve up to uncorrelated sources. However, among the many configurations introduced in the literature, the arrays that provide the largest hole-free co-array are the most susceptible to sensor failures. We propose here two machine learning (ML) methods to mitigate the effect of sensor failures and maintain the DOA estimation performance and resolution. The first method enhances the conventional spatial smoothing using deep neural network (DNN), while the second one is an end-to-end data-driven method. Numerical results show that both approaches can significantly improve the performance of MRA with two failed sensors. The data-driven method can …

Neural networks are known to give better performance with increased depth due to their ability to learn more abstract features. Although the deepening of networks has been well established, there is still room for efficient feature extraction within a layer, which would reduce the need for mere parameter increment. The conventional widening of networks by having more filters in each layer introduces a quadratic increment of parameters. Having multiple parallel convolutional/dense operations in each layer solves this problem, but without any context-dependent allocation of input among these operations: The parallel computations tend to learn similar features making the widening process less effective. Therefore, we propose the use of multi-path neural networks with data-dependent resource allocation from parallel computations within layers, which also lets an input be routed end-to-end through these parallel …

Since its introduction, diffractive deep neural networks (D2NN) have been an emerging technology with many useful applications, such as 3D object recognition, saliency segmentation, and quantitative phase microscopy. However, fabricating D2NNs operating in the visible range has not been performed due to the complexities of fabricating nano-scale elements. Recent advancements such as Implosion Fabrication have made it possible to fabricate such networks using a discrete number of phase weights. We propose a quantization-aware training approach for D2NNs through modeling the quantization process in a differentiable manner using a sigmoid-based quantization function, facilitating the fabrication process. We also propose an efficient training schedule to guide the optimization process to converge to a better minima despite the limited number of quantization levels. Our method is simulated and …

Deep neural network models can learn clinically relevant features from millions of histopathology images. However generating high-quality annotations to train such models for each hospital, each cancer type, and each diagnostic task is prohibitively laborious. On the other hand, terabytes of training data–while lacking reliable annotations–are readily available in the public domain in some cases. In this work, we explore how these large datasets can be consciously utilized to pre-train deep networks to encode informative representations. We then fine-tune our pre-trained models on a fraction of annotated training data to perform specific downstream tasks. We show that our approach can reach the current state-of-the-art (SOTA) for patch-level classification with only 1-10% randomly selected annotations compared to other SOTA approaches. Moreover, we propose an uncertainty-aware loss function, to quantify the model confidence during inference. Quantified uncertainty helps experts select the best instances to label for further training. Our uncertainty-aware labeling reaches the SOTA with significantly fewer annotations compared to random labeling. Last, we demonstrate how our pretrained encoders can surpass current SOTA for wholeslide image classification with weak supervision. Our work lays the foundation for data and task-agnostic pretrained deep networks with quantified uncertainty.

Class Incremental Learning (CIL) is inspired by the human ability to learn new classes without forgetting previous ones. CIL becomes more challenging in real-world scenarios when the samples in each incremental step are imbalanced. This creates another branch of problem, called Generalized Class Incremental Learning (GCIL) where each incremental step is structured more realistically. Grow When Required (GWR) network, a type of Self-Organizing Map (SOM), dynamically creates and removes nodes and edges for adaptive learning. GWR performs incremental learning from feature vectors extracted by a Convolutional Neural Network (CNN), which acts as a feature extractor. The inherent ability of GWR to form distinct clusters, each corresponding to a class in the feature vector space, regardless of the order of samples or class imbalances, is well suited to achieving GCIL. To enhance GWR's classification …

Real-time and online action localization in videos poses a critical and formidable challenge. Achieving accurate action localization necessitates the integration of both temporal and spatial information. However, existing approaches rely on computationally intensive 3D convolutional neural network (CNN) architectures or redundant two-stream architectures with optical flow, rendering them unsuitable for real-time, online applications. To address this, we propose a novel approach that leverages fast and efficient key-point-based bounding box prediction for spatial action localization. Additionally, we introduce a tube-linking algorithm that ensures the temporal continuity of action tubes even in the presence of occlusions. By combining temporal and spatial information into a cascaded input for a single network, we eliminate the need for a two-stream architecture, enabling the network to effectively learn from both types …

Since the dawn of its development the computer has been a staple of science fiction and thus mostly due to it humanity has constantly dreaded a day when intelligence stemming from computers commonly known as Artificial Intelligence (AI) would allegedly take over the world. The recent advent of advanced AI chatbot technology has again sparked that fear due to its ability to compose essays similar to that of a human being. This chapter discusses the concept of intelligence in biology and binary digital computing. Based upon the weaknesses of either, especially the hidden vulnerabilities of AI predicted by Gödel’s First Incompleteness Theorem and role of humans in programming AI, the question of whether it is likely to happen in the near future is discussed.

Traffic management is a serious problem in many cities around the world. Even the suburban areas are now experiencing regular traffic congestion. Inappropriate traffic control wastes fuel, time, and the productivity of nations. Though traffic signals are used to improve traffic flow, they often cause problems due to inappropriate or obsolete timing that does not tally with the actual traffic intensity at the intersection. Traffic intensity determination based on statistical methods only gives the average intensity expected at any given time. However, to control traffic accurately, it is required to know the real-time traffic intensity. In this research, image processing and machine learning have been used to estimate actual traffic intensity in real time. General-purpose electronic hardware has been used for in-situ image processing based on the edge-detection method. A deep neural network (DNN) was trained to infer traffic intensity in each image in real time. The trained DNN estimated traffic intensity accurately in 90% of the real-time images during road tests. The electronic system was implemented on a Raspberry Pi single-board computer; hence, it is cost-effective for large-scale deployment.

In this paper, we propose a method for duplicating digital circuits using neural networks (NNs). Our approach involves training a common neural network model on the inputs and outputs of existing digital circuits, creating a digital twin. This digital twin can be used to predict the outputs of a digital I'C, only using its inputs and this has been verified using counter circuits, and shift register circuits. Moreover, we have successfully validated the functionality duplication of digital logic in a robotic arm equipped with servo motors, further validating the utility of our approach.

We propose a novel attention-based deep learning framework for reconstructing images from compressively sensed measurements. Our method leverages knowledge of the sampling basis and structured sampling patterns, highlighting the importance of learning adaptive features from measurements.

Compressive sensing (CS) reconstructs images from sub-Nyquist measurements by solving a sparsity-regularized inverse problem. Traditional CS solvers use iterative optimizers with hand crafted sparsifiers, while early data-driven methods directly learn an inverse mapping from the low-dimensional measurement space to the original image space. The latter outperforms the former, but is restrictive to a pre-defined measurement domain. More recent, deep unrolling methods combine traditional proximal gradient methods and data-driven approaches to iteratively refine an image approximation. To achieve higher accuracy, it has also been suggested to learn both the sampling matrix, and the choice of measurement vectors adaptively. Contrary to the current trend, in this work we hypothesize that a general inverse mapping from a random set of compressed measurements to the image domain exists for a given measurement basis, and can be learned. Such a model is single-shot, non-restrictive and does not parametrize the sampling process. To this end, we propose MOSAIC, a novel compressive sensing framework to reconstruct images given any random selection of measurements, sampled using a fixed basis. Motivated by the uneven distribution of information across measurements, MOSAIC incorporates an embedding technique to efficiently apply attention mechanisms on an encoded sequence of measurements, while dispensing the need to use unrolled deep networks. A range of experiments validate our proposed architecture as a promising alternative for existing CS reconstruction methods, by achieving the state-of-the-art for metrics of …

With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. However, the speed of current QPM systems is limited by electronic hardware. To improve throughput further, here we propose differentiable optical-electronic quantitative phase microscopy (∂μ) that acquires images in a compressed form such that more information can be transferred beyond the electronic hardware bottleneck. The proposed microscopy uses optical feature extractors as image compressors. The resultant intensity distribution is then decompressed into QPM images by a reconstruction network. By optimizing optical-electronic parameters in an end-to-end manner, our method can improve the QPI throughput from Hours to Seconds (more than an order of magnitude).