Department Publications

Even though human experience unfolds continuously in time, it is not strictly linear; instead, it entails cascading processes building hierarchical cognitive structures. For instance, during speech perception, humans transform a continuously varying acoustic signal into phonemes, words, and meaning, and these levels all have distinct but interdependent temporal structures. Time-lagged regression using temporal response functions (TRFs ) has recently emerged as a promising tool for disentangling electrophysiological brain responses related to such complex models of perception. Here, we introduce the Eelbrain Python toolkit, which makes this kind of analysis easy and accessible. We demonstrate its use, using continuous speech as a sample paradigm, with a freely available EEG dataset of audiobook listening. A companion GitHub repository provides the complete source code for the analysis, from raw data to group-level statistics. More generally, we advocate a hypothesis-driven approach in which the experimenter specifies a hierarchy of time-continuous representations that are hypothesized to have contributed to brain responses, and uses those as predictor variables for the electrophysiological signal. This is analogous to a multiple regression problem, but with the addition of a time dimension. TRF analysis decomposes the brain signal into distinct responses associated with the different predictor variables by estimating a multivariate TRF (mTRF), quantifying the influence of each predictor on brain responses as a function of time(-lags). This allows asking two questions about the predictor variables: (1) Is there a significant neural …

Metaverse is the latest technological hype in the modern world due to its potential for revolutionizing the digital visual perspective. With the COVID-19 pandemic, most industries have moved towards digitization, and the metaverse is identified as one of the most promising platforms for such a transition, as it provides a three-dimensional (3D) immersive experience for the users. Currently, most digital service providers and organizations are actively working on metaverse- based applications. In addition, there has been a rapid increase on research work involving metaverse realization. Launching a large scale metaverse in the real world is a challenging task. However, fifth-generation (5G) and beyond 5G (B5G) technologies are envisioned to improve the feasibility of pragmatic deployments. Although, there are several conceptual designs available, actual adaptations of the concepts are still limited. This survey …

Computational optical imaging (COI) systems leverage optical coding elements (CEs) in their setups to encode a high-dimensional scene in a single or in multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches rely on random patterns or analytical designs to set distribution of the CE. However, the available data and algorithm capabilities of deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical and computational decoders. Specifically, by modeling the COI measurements through a fully differentiable image-formation model that considers the physics-based propagation of light and its interaction with the CEs, the parameters that define the CE and the …

Neural speech tracking has advanced our understanding of how our brains rapidly map an acoustic speech signal onto linguistic representations and ultimately meaning. It remains unclear, however, how speech intelligibility is related to the corresponding neural responses. Many studies addressing this question vary the level of intelligibility by manipulating the acoustic waveform, but this makes it difficult to cleanly disentangle the effects of intelligibility from underlying acoustical confounds. Here, using magnetoencephalography recordings, we study neural measures of speech intelligibility by manipulating intelligibility while keeping the acoustics strictly unchanged. Acoustically identical degraded speech stimuli (three-band noise-vocoded, ~20 s duration) are presented twice, but the second presentation is preceded by the original (nondegraded) version of the speech. This intermediate priming, which generates a …

Cortical ischaemic strokes result in cognitive deficits depending on the area of the affected brain. However, we have demonstrated that difficulties with attention and processing speed can occur even with small subcortical infarcts. Symptoms appear independent of lesion location, suggesting they arise from generalized disruption of cognitive networks. Longitudinal studies evaluating directional measures of functional connectivity in this population are lacking. We evaluated six patients with minor stroke exhibiting cognitive impairment 6–8 weeks post-infarct and four age-similar controls. Resting-state magnetoencephalography data were collected. Clinical and imaging evaluations of both groups were repeated 6- and 12 months later. Network Localized Granger Causality was used to determine differences in directional connectivity between groups and across visits, which were correlated with clinical …

This paper explores the use cases and performance of generative artificial intelligence (AI) tools in the context of Electrical Engineering education. A summary of commonly used generative AI tools in Engineering education is provided along with a review of their use cases from recent literature. The performances of the generative AI tools GPT-3.5, GPT-4 and Google Bard are examined using custom-made question sets in two representative courses: Digital Logic Design and Digital Signal Processing. The paper highlights that GPT4 achieves over 50% accuracy in both courses and generative AI tools are more effective at solving standalone concept questions than analytical or numerical questions.

In disaster-stricken environments, it’s vital to assess the damage quickly, analyse the stability of the environment, and allocate resources to the most vulnerable areas where victims might be present. These missions are difficult and dangerous to be conducted directly by humans. Using the complementary capabilities of both the ground and aerial robots, we investigate a collaborative approach of aerial and ground robots to address this problem. With an increased field of view, faster speed, and compact size, the aerial robot explores the area and creates a 3D feature-based map graph of the environment while providing a live video stream to the ground control station. Once the aerial robot finishes the exploration run, the ground control station processes the map and sends it to the ground robot. The ground robot, with its higher operation time, static stability, payload delivery and tele-conference capabilities, can then …

Auditory cortical responses to speech obtained by magnetoencephalography (MEG) show robust speech tracking to the speaker's fundamental frequency in the high-gamma band (70–200 Hz), but little is currently known about whether such responses depend on the focus of selective attention. In this study 22 human subjects listened to concurrent, fixed-rate, speech from male and female speakers, and were asked to selectively attend to one speaker at a time, while their neural responses were recorded with MEG. The male speaker's pitch range coincided with the lower range of the high-gamma band, whereas the female speaker's higher pitch range had much less overlap, and only at the upper end of the high-gamma band. Neural responses were analyzed using the temporal response function (TRF) framework. As expected, the responses demonstrate robust speech tracking of the fundamental frequency in the high-gamma band, but only to the male's speech, with a peak latency of ~40 ms. Critically, the response magnitude depends on selective attention: the response to the male speech is significantly greater when male speech is attended than when it is not attended, under acoustically identical conditions. This is a clear demonstration that even very early cortical auditory responses are influenced by top-down, cognitive, neural processing mechanisms.

Image generation from a single image using generative adversarial networks is quite interesting due to the realism of generated images. However, recent approaches need improvement for such realistic and diverse image generation, when the global context of the image is important such as in face, animal, and architectural image generation. This is mainly due to the use of fewer convolutional layers for capturing the patch statistics and, thereby, not being able to capture global statistics well. The challenge, then, is to preserve the global structure, while retaining the diversity and quality of image generation. We solve this problem by using attention blocks at selected scales and feeding a random Gaussian blurred image to the discriminator for training. We use adversarial feedback to make the quality of the generation better. Our results are visually better than the state-of-the-art, particularly, in generating images that …

Sleep plays a crucial role in the well-being of human lives. Traditional sleep studies using Polysomnography are associated with discomfort and often lower sleep quality caused by the acquisition setup. Previous works have focused on developing less obtrusive methods to conduct high-quality sleep studies, and ear- EEG is among popular alternatives. However, the performance of sleep staging based on ear-EEG is still inferior to scalp- EEG based sleep staging. In order to address the performance gap between scalp-EEG and ear- EEG based sleep staging, we propose a cross-modal knowledge distillation strategy 1 † https://github.com/Mithunjha/EarEEG_KnowledgeDistillation, which is a domain adaptation approach. We employ model architectures from the transformer and convolutional neural network families to demonstrate the model-agnostic nature of the method. Our experiments and analysis validate the …

Light field saliency detection–important due to utility in many vision tasks–still lacks speed and can improve in accuracy. Due to the formulation of the saliency detection problem in light fields as a segmentation task or a memorizing task, existing approaches consume unnecessarily large amounts of computational resources for training, and have longer execution times for testing. We solve this by aggressively reducing the large light field images to a much smaller three-channel feature map appropriate for saliency detection using an RGB image saliency detector with attention mechanisms. We achieve this by introducing a novel convolutional neural network based features extraction and encoding module. Our saliency detector takes 0.4 s to process a light field of size 9× 9× 512× 375 in a CPU and is significantly faster than state-of-the-art light field saliency detectors, with better or comparable accuracy. Furthermore …

Prioritization of data streams in cellular vehicle-to-everything (C-V2X) may lead to unfavorable packet delays in low-priority streams. This paper studies allocating multiple candidate single-subframe resources (CSRs) per vehicle as a solution. It proposes a methodology to determine the number of CSRs for each vehicle based on the total number of neighboring vehicles, and to assign the multiple data streams among them for simultaneous transmission. The numerical results highlight the achievable delay gains of the proposed approach, and its negligible impact on packet collisions.

This paper proposes a novel approach that utilizes differential encoding to overcome the channel estimation problem in communication systems with low-resolution quantization receivers. For differentially encoded data, we derive the maximum likelihood detection rule for the canonical block-2 detectors, employing just two consecutive quantized observations at the channel output and without any receiver-side channel state information. We establish the optimality of this maximum likelihood detection rule within the class of block- detectors, where , under the condition that , with and denoting the number of quantization bits and input alphabet size, respectively. The derived detector has a simple and easily implementable structure, comparing the quantization region indices of consecutive observations to determine the transmitted message index. By leveraging the structure of the derived optimum …

This paper presents an effective method of identifying elephant rumbles in infrasonic seismic signals. The design and implementation of electronic circuitry to amplify, filter, and digitize the seismic signals captured through geophones are presented. A collection of seismic infrasonic elephant rumbles was collected at a free-ranging area of an elephant orphanage in Sri Lanka. The seismic rumbles were converted to spectrograms, and several methods were used for spectral feature extraction. Using LasyPredict, the features extracted using different methods were fed into their corresponding machine-learning algorithms to train them for automatic seismic rumble identification. It was found that the Mel frequency cepstral coefficient (MFCC) together with the Ridge classifier machine learning algorithm produced the best performance in identifying seismic elephant rumbles. A novel method for denoising the spectrum that leads to enhanced accuracy in identifying seismic rumbles is also presented.

Traffic light detection is essential for self-driving cars to navigate safely in urban areas. Publicly available traffic light datasets are inadequate for the development of algorithms for detecting distant traffic lights that provide important navigation information. We introduce a novel bench-mark traffic light dataset captured using a synchronized pair of narrow-angle and wide-angle cameras covering urban and semi-urban roads. We provide 1032 images for training and 813 synchronized image pairs for testing. Additionally, we provide synchronized test video pairs for qualitative analysis. The dataset includes images of resolution covering 10 different classes. Furthermore, we propose a post-processing algorithm for combining outputs from the two cameras. Results show that our technique can strike a balance between speed and accuracy, compared to the conventional approach of using a single camera frame …

Precision agriculture is one of the trending research areas in the world. AI technologies have been applied to greenhouse hydroponics agriculture (which comes under precision agriculture) to control the requirements of the plants in greenhouse agriculture. Apart from the environmental conditions such as temperature, humidity, and light intensity, the optimum combination of the fertilizer components, EC, and pH value provided during the plants’ growth phase is vital to obtaining a higher yield during the harvesting period. However, finding the optimal fertilizer composition is challenging due to the fact that the required fertilizer composition is different for different plant types and the experimental cost is higher. Sparse data samples will be received in data-driven experiments because of the large search space (number of permutations) of the fertilizer combinations. This paper presents an experimental design and AI …

In this paper, we address the problem of direction of arrival (DOA) estimation for multiple targets in the presence of sensor failures in a sparse array. Generally, sparse arrays are known with very high-resolution capabilities, where N physical sensors can resolve up to uncorrelated sources. However, among the many configurations introduced in the literature, the arrays that provide the largest hole-free co-array are the most susceptible to sensor failures. We propose here two machine learning (ML) methods to mitigate the effect of sensor failures and maintain the DOA estimation performance and resolution. The first method enhances the conventional spatial smoothing using deep neural network (DNN), while the second one is an end-to-end data-driven method. Numerical results show that both approaches can significantly improve the performance of MRA with two failed sensors. The data-driven method can …

Hearing-impaired listeners have a reduced ability to selectively attend to sounds of interest amid distracting sounds in everyday environments. This ability is not fully regained with modern hearing technology. A better understanding of the brain mechanisms underlying selective attention during speech processing may lead to brain-controlled hearing aids with improved detection and amplification of the attended speech. Prior work has shown that brain responses to speech, measured with magnetoencephalography (MEG) or electroencephalography (EEG), are modulated by selective attention. These responses can be predicted from the speech signal through linear filters called Temporal Response Functions (TRFs). Unfortunately, these sensor-level predictions are often noisy and do not provide much insight into specific brain source locations. Therefore, a novel method called Neuro-Current Response Functions …

A low-complexity multibeamforming scheme for uniform linear arrays is proposed based on warped and approximated discrete Fourier transform (DFT), albeit in the spatial domain. Conventional DFT- and approximated-DFT-based multibeamformers provide a low-complexity alternatives to produce multiple simultaneous beams with fixed beam profiles. The proposed approach aims to produce configurable (i.e., steerable) multibeam profiles using the warped DFT. Design examples are presented for an 8-element uniform linear array verifying, 81% and 18 % reduction in multiplier and adder complexities, respectively, when compared to a conventional narrowband multibeamformer having dedicated steering weights for each beam. Three system configurations are proposed using approximations for warped transform and the DFT. Hardware designs implemented on Xilinx Ultrascale KCU105 FPGA device verify 5.2 …

Direct policy learning (DPL) is a widely used approach in imitation learning for time-efficient and effective convergence when training mobile robots. However, using DPL in real-world applications is not sufficiently explored due to the inherent challenges of mobilizing direct human expertise and the difficulty of measuring comparative performance. Furthermore, autonomous systems are often resource-constrained, thereby limiting the potential application and implementation of highly effective deep learning models. In this work, we present a lightweight DPL-based approach to train mobile robots in navigational tasks. We integrated a safety policy alongside the navigational policy to safeguard the robot and the environment. The approach was evaluated in simulations and real-world settings and compared with recent work in this space. The results of these experiments and the efficient transfer from simulations to real-world settings demonstrate that our approach has improved performance compared to its hardware-intensive counterparts. We show that using the proposed methodology, the training agent achieves closer performance to the expert within the first 15 training iterations in simulation and real-world settings.

Neural networks are known to give better performance with increased depth due to their ability to learn more abstract features. Although the deepening of networks has been well established, there is still room for efficient feature extraction within a layer, which would reduce the need for mere parameter increment. The conventional widening of networks by having more filters in each layer introduces a quadratic increment of parameters. Having multiple parallel convolutional/dense operations in each layer solves this problem, but without any context-dependent allocation of input among these operations: The parallel computations tend to learn similar features making the widening process less effective. Therefore, we propose the use of multi-path neural networks with data-dependent resource allocation from parallel computations within layers, which also lets an input be routed end-to-end through these parallel …

Recent advancements in artificial intelligence have enabled reinforcement learning (RL) agents to exceed human-level performance in various gaming tasks. However, despite the state-of-the-art performance demonstrated by model-free RL algorithms, they suffer from high sample complexity. Hence, it is uncommon to find their applications in robotics, autonomous navigation, and self-driving, as gathering many samples is impractical in real-world hardware systems. Therefore, developing sample-efficient learning algorithms for RL agents is crucial in deploying them in real-world tasks without sacrificing performance. This paper presents an advisor-based learning algorithm, incorporating prior knowledge into the training by modifying the deep deterministic policy gradient algorithm to reduce the sample complexity. Also, we propose an effective method of employing an advisor in data collection to train autonomous navigation agents to maneuver physical platforms, minimizing the risk of collision. We analyze the performance of our methods with the support of simulation and physical experimental setups. Experiments reveal that incorporating an advisor into the training phase significantly reduces the sample complexity without compromising the agent’s performance compared to various benchmark approaches. Also, they show that the advisor’s constant involvement in the data collection process diminishes the agent’s performance, while the limited involvement makes training more effective.

This paper derives the optimum detection rule for communication systems with n-bit phase quantization when data is differentially encoded at the transmitter. The proposed approach avoids the channel estimation problem at the receiver. First, a maximum likelihood detection rule for block-2 detectors utilizing only two consecutive quantized observations at the channel output is obtained. Second, it is shown that the derived maximum likelihood detection rule continues to be optimum for the class of block-L detectors for L ≥ 3 when n = log 2 M, where M is the input alphabet size. Finally, utilizing the structure of the derived optimum detector, a message error probability expression is obtained for Rayleigh fading wireless channels. A simulation study is performed to illustrate the performance of the optimum detectors as well as the performance loss due to the lack of channel knowledge at the receiver. The proposed …

In this paper, we consider the problem of recovering speech from audio recordings of on-board microphones in a noisy drone platform. Enhancement of speech degraded by drone noise is considered to be a difficult task due to the strong noise generated from its motors and propellers causing an extremely low signal-to-drone noise ratio (SdNR). We propose a solution by (i) developing a multichannel Wiener filter (MWF) to remove drone noise from microphone recordings, and (ii) further reduction of residual noise using a Gaussian mixture model (GMM) based dual-stage parametric Wiener filter (WF). The method exploits the known statistics of motor current-specific drone noise. The theory developed is applicable to irregular microphone array embedded in a drone enabling realistic integration to most drones. We evaluate the proposed framework using two different drone acoustics datasets under extreme SdNR levels including− 30 dB. The experimental results confirm promising performance in terms of SdNR improvement, speech quality (PESQ), and intelligibility (STOI) and show a strong potential for speech enhancement applications using noisy drones.