Department Publications

With the recent emergence of virtual reality, augmented reality, and mixed-reality technologies, considerable attention has been received to integrate spatial audio recording capabilities into these devices. Extracting higher order spherical harmonics from the recordings is not always possible due to complex scattering effects from the arbitrary-shaped devices, which motivates the need for modeling the acoustic properties beyond free-field convention. In this paper, we propose modeling the wearable device-related transfer function (WDRTF) that incorporates scattering, and diffraction from the device and wearer's body to an incident soundfield. We adopt the spherical wave model to translate the measurements into microphone alignment, by exploiting both magnitude and phase corrections. We use a discrete set of measurements by multiple circular arrays for efficiently estimating the WDRTF coefficients. We illustrate the WDRTF modeling procedure from a head-worn microphone array and obtain accurate reconstruction performance over frequency and 3D directions.

The Relative Transfer Matrix (ReTM), recently introduced as a generalization of the relative transfer function for multiple sources and multiple microphones, shows promising performance when applied to speech denoising and speaker separation in a noisy reverberant room. This work utilizes the ReTM to propose a novel framework for drone noise suppression. Difficulties in noise cancellation for drone audition applications arise due to its self-generated noise that causes an adverse noisy environment of low Signal-to-Drone Noise Ratio (SDNR) levels. In this paper, we divide the drone on-board microphones into two multichannel groups to approximately estimate the drone noise from one group to the other with known drone noise ReTM for denoising. We demonstrate the ReTM spatial mapping ability in both indoor and outdoor experiments using hovering drone on-board microphone recordings with low average magnitude spectrum error. Finally, we validate the method in a real-life environment for source signal enhancement over different SDNR conditions and offer both improved speech intelligibility and signal-to-distortion ratio.

The auditory brainstem response (ABR) is a measure of subcortical activity in response to auditory stimuli. The wave V peak of the ABR depends on stimulus intensity level, and has been widely used for clinical hearing assessment. Conventional methods to estimate the ABR average electroencephalography (EEG) responses to short unnatural stimuli such as clicks. Recent work has moved towards more ecologically relevant continuous speech stimuli using linear deconvolution models called Temporal Response Functions (TRFs). Investigating whether the TRF waveform changes with stimulus intensity is a crucial step towards the use of natural speech stimuli for hearing assessments involving subcortical responses. Here, we develop methods to estimate level-dependent subcortical TRFs using EEG data collected from 21 participants listening to continuous speech presented at 4 different intensity levels. We find that level-dependent changes can be detected in the wave V peak of the subcortical TRF for almost all participants, and are consistent with level-dependent changes in click-ABR wave V. We also investigate the most suitable peripheral auditory model to generate predictors for level-dependent subcortical TRFs and find that simple gammatone filterbanks perform the best. Additionally, around 6 minutes of data may be sufficient for detecting level-dependent effects and wave V peaks above the noise floor for speech segments with higher intensity. Finally, we show a proof-of-concept that level dependent subcortical TRFs can be detected even for the inherent intensity fluctuations in natural continuous speech. Visual abstract Significance …

Denoising of four-dimensional (4-D) light fields (LFs) is a fundamental preprocessing step before employing LFs for downstream tasks. Most of LF denoising methods have higher computational complexities despite excellent performance. Denoising of LFs using 4-D linear filters is a low-complexity approach that can achieve near real-time or real-time processing. We propose an ultra low-complexity LF denoising methods using 4-D infinite-extent impulse response (IIR) hyperfan filters in this paper. Our approach exploits the sparsity of an LF in the 4-D frequency domain, and employ a set of one-dimensional and two-dimensional first-order IIR filters to approximate the 4-D hyperfan-shaped passband. On average, the proposed 4-D IIR hyperfan filter needs only 2.56 multiplications and 21.06 additions to process a LF sample leading to more than 92% and 64% savings in the number of multiplications and additions to …

A spatially-interpolated (SI) beamformer is proposed in this paper to enhance narrowband plane waves received by a sparse linear antenna array. Similar to a conventional SI filter, the proposed beamformer employs two filters: an SI prototype filter and a masking filter. However, the proposed SI beamformer employs a parallel structure compared to a cascade structure of conventional SI filters leading to a sparse antenna array. Both the prototype filter and the masking filter are designed as optimal filters in the minimax sense. Furthermore, we employ a bell-shaped passband for the ideal magnitude response for the design compared to a conventional ideal frequency response that employs 1 and 0 as the passband and the stopband gains with a finite transition band. Simulation results confirm that the proposed SI beamformer provides approximately 37% reduction in computational complexity and a similar reduction …

We propose a minimax design method for multi-dimensional (M-D) interpolated finite-support impulse response (IFIR) filters. As a result of the cascade structure of an MD IFIR filter, the optimization problem of the minimax design is nonconvex. In order to solve this nonconvex optimization problem, we employ a convex-concave procedure, where each iteration solves a convex constrained optimization problem. We present experimental results to confirm the effectiveness and the superior performance of the proposed method compared to previously proposed minimax designs of M-D IFIR filters. Furthermore, we present an implementation of M-D IFIR filter on the Xilinx Kintex UltraScale KCU105 field-programmable gate array, where a quantization error of −60.59 dB is achieved with the coefficient word length of 24 bits.

Chromatic derivatives (CDs) and associated chromatic approximations (CAs) provide a numerically robust and powerful framework for digital processing of continuous-time/space signals, using not only the discrete signal amplitudes, but also higher order derivatives. Acquisition of CDs has so far been limited to software implementations. This paper presents field-programmable gate array (FPGA)-based digital hardware architectures suitable for the real-time acquisition of CDs, thus enabling CDs to be used for practical signal processing algorithm development in embedded systems. With 24 bits of fixed-point precision, the proposed hardware architectures provide approximate 116 dB accuracy in the frequency responses of FIR filters capturing the CDs, and 76.85 dB and 75.09 dB accuracy in signal reconstruction with synthetic and real audio signals using CDs up to order 28 degrees. The designs have been …

With applications in quantitative metabolomics and label-free digital pathology, Quantitative Phase Microscopy (QPM) measures refractive index maps of thin transparent specimens like live cells or tissue sections. In QPM, refractive index maps are usually reconstructed from interference measurements of the object’s light field with respect to a reference field. To this end, many previous works focused on designing stable full-field interferometers from the bottom up. In this work, we present an alternative strategy to design a QPM system top-down, starting from the desired measurement outcomes, with no explicit knowledge about interferometry. We call our inverse design strategy Differentiable Microcopy. To this end, our Differentiable Microcopy approach designed a range of Fourier-filter-based QPM systems that do not require computational post-reconstruction. Our designs are superior to existing similar designs in …

Recently, various optical neural architectures have been designed to perform imaging and all-optical image processing. These designs consist of several optical masks with optimized transmission coefficients for the task. Designing sparse optical masks for them is important as it can promote different aspects of the design such as ease of fabrication and power efficiency of the design. To this end, inspired by the sparse filter designs, we propose training optical neural architectures with a regularization promoting sparsity in the masks. As preliminary results, we demonstrate a D2NN design for QPI achieving a sparsity of 33% with a performance degradation of only 12%.

Wi-Fi has become a cost-effective sensing mechanism due to its widespread availability. Channel State Information (CSI) from the OFDM-based physical layer of Wi-Fi contains rich information for sensing static and dynamic channel conditions, and hence for sensing activities within the channel. Numerous applications which combine CSI-based sensing with machine learning (ML) available for both indoor and outdoor environments. We extend the use of CSI- based Wi-Fi sensing to a unique setting: a confined environment. An industrial beverage cooler is specifically considered. We address the challenges of sensing device placement in such confined environments, where line-of-sight obstruction and near-field effects play a key role in CSI stability. We compensate for these effects by introducing novel combinations of data preprocessing techniques and input feature definitions. We demonstrate an accuracy of 89 …

Even though human experience unfolds continuously in time, it is not strictly linear; instead, it entails cascading processes building hierarchical cognitive structures. For instance, during speech perception, humans transform a continuously varying acoustic signal into phonemes, words, and meaning, and these levels all have distinct but interdependent temporal structures. Time-lagged regression using temporal response functions (TRFs ) has recently emerged as a promising tool for disentangling electrophysiological brain responses related to such complex models of perception. Here, we introduce the Eelbrain Python toolkit, which makes this kind of analysis easy and accessible. We demonstrate its use, using continuous speech as a sample paradigm, with a freely available EEG dataset of audiobook listening. A companion GitHub repository provides the complete source code for the analysis, from raw data to group-level statistics. More generally, we advocate a hypothesis-driven approach in which the experimenter specifies a hierarchy of time-continuous representations that are hypothesized to have contributed to brain responses, and uses those as predictor variables for the electrophysiological signal. This is analogous to a multiple regression problem, but with the addition of a time dimension. TRF analysis decomposes the brain signal into distinct responses associated with the different predictor variables by estimating a multivariate TRF (mTRF), quantifying the influence of each predictor on brain responses as a function of time(-lags). This allows asking two questions about the predictor variables: (1) Is there a significant neural …

Metaverse is the latest technological hype in the modern world due to its potential for revolutionizing the digital visual perspective. With the COVID-19 pandemic, most industries have moved towards digitization, and the metaverse is identified as one of the most promising platforms for such a transition, as it provides a three-dimensional (3D) immersive experience for the users. Currently, most digital service providers and organizations are actively working on metaverse- based applications. In addition, there has been a rapid increase on research work involving metaverse realization. Launching a large scale metaverse in the real world is a challenging task. However, fifth-generation (5G) and beyond 5G (B5G) technologies are envisioned to improve the feasibility of pragmatic deployments. Although, there are several conceptual designs available, actual adaptations of the concepts are still limited. This survey …

Computational optical imaging (COI) systems leverage optical coding elements (CEs) in their setups to encode a high-dimensional scene in a single or in multiple snapshots and decode it by using computational algorithms. The performance of COI systems highly depends on the design of its main components: the CE pattern and the computational method used to perform a given task. Conventional approaches rely on random patterns or analytical designs to set distribution of the CE. However, the available data and algorithm capabilities of deep neural networks (DNNs) have opened a new horizon in CE data-driven designs that jointly consider the optical and computational decoders. Specifically, by modeling the COI measurements through a fully differentiable image-formation model that considers the physics-based propagation of light and its interaction with the CEs, the parameters that define the CE and the …

Neural speech tracking has advanced our understanding of how our brains rapidly map an acoustic speech signal onto linguistic representations and ultimately meaning. It remains unclear, however, how speech intelligibility is related to the corresponding neural responses. Many studies addressing this question vary the level of intelligibility by manipulating the acoustic waveform, but this makes it difficult to cleanly disentangle the effects of intelligibility from underlying acoustical confounds. Here, using magnetoencephalography recordings, we study neural measures of speech intelligibility by manipulating intelligibility while keeping the acoustics strictly unchanged. Acoustically identical degraded speech stimuli (three-band noise-vocoded, ~20 s duration) are presented twice, but the second presentation is preceded by the original (nondegraded) version of the speech. This intermediate priming, which generates a …

Cortical ischaemic strokes result in cognitive deficits depending on the area of the affected brain. However, we have demonstrated that difficulties with attention and processing speed can occur even with small subcortical infarcts. Symptoms appear independent of lesion location, suggesting they arise from generalized disruption of cognitive networks. Longitudinal studies evaluating directional measures of functional connectivity in this population are lacking. We evaluated six patients with minor stroke exhibiting cognitive impairment 6–8 weeks post-infarct and four age-similar controls. Resting-state magnetoencephalography data were collected. Clinical and imaging evaluations of both groups were repeated 6- and 12 months later. Network Localized Granger Causality was used to determine differences in directional connectivity between groups and across visits, which were correlated with clinical …

This paper explores the use cases and performance of generative artificial intelligence (AI) tools in the context of Electrical Engineering education. A summary of commonly used generative AI tools in Engineering education is provided along with a review of their use cases from recent literature. The performances of the generative AI tools GPT-3.5, GPT-4 and Google Bard are examined using custom-made question sets in two representative courses: Digital Logic Design and Digital Signal Processing. The paper highlights that GPT4 achieves over 50% accuracy in both courses and generative AI tools are more effective at solving standalone concept questions than analytical or numerical questions.

In disaster-stricken environments, it’s vital to assess the damage quickly, analyse the stability of the environment, and allocate resources to the most vulnerable areas where victims might be present. These missions are difficult and dangerous to be conducted directly by humans. Using the complementary capabilities of both the ground and aerial robots, we investigate a collaborative approach of aerial and ground robots to address this problem. With an increased field of view, faster speed, and compact size, the aerial robot explores the area and creates a 3D feature-based map graph of the environment while providing a live video stream to the ground control station. Once the aerial robot finishes the exploration run, the ground control station processes the map and sends it to the ground robot. The ground robot, with its higher operation time, static stability, payload delivery and tele-conference capabilities, can then …

Auditory cortical responses to speech obtained by magnetoencephalography (MEG) show robust speech tracking to the speaker's fundamental frequency in the high-gamma band (70–200 Hz), but little is currently known about whether such responses depend on the focus of selective attention. In this study 22 human subjects listened to concurrent, fixed-rate, speech from male and female speakers, and were asked to selectively attend to one speaker at a time, while their neural responses were recorded with MEG. The male speaker's pitch range coincided with the lower range of the high-gamma band, whereas the female speaker's higher pitch range had much less overlap, and only at the upper end of the high-gamma band. Neural responses were analyzed using the temporal response function (TRF) framework. As expected, the responses demonstrate robust speech tracking of the fundamental frequency in the high-gamma band, but only to the male's speech, with a peak latency of ~40 ms. Critically, the response magnitude depends on selective attention: the response to the male speech is significantly greater when male speech is attended than when it is not attended, under acoustically identical conditions. This is a clear demonstration that even very early cortical auditory responses are influenced by top-down, cognitive, neural processing mechanisms.

Image generation from a single image using generative adversarial networks is quite interesting due to the realism of generated images. However, recent approaches need improvement for such realistic and diverse image generation, when the global context of the image is important such as in face, animal, and architectural image generation. This is mainly due to the use of fewer convolutional layers for capturing the patch statistics and, thereby, not being able to capture global statistics well. The challenge, then, is to preserve the global structure, while retaining the diversity and quality of image generation. We solve this problem by using attention blocks at selected scales and feeding a random Gaussian blurred image to the discriminator for training. We use adversarial feedback to make the quality of the generation better. Our results are visually better than the state-of-the-art, particularly, in generating images that …

Sleep plays a crucial role in the well-being of human lives. Traditional sleep studies using Polysomnography are associated with discomfort and often lower sleep quality caused by the acquisition setup. Previous works have focused on developing less obtrusive methods to conduct high-quality sleep studies, and ear- EEG is among popular alternatives. However, the performance of sleep staging based on ear-EEG is still inferior to scalp- EEG based sleep staging. In order to address the performance gap between scalp-EEG and ear- EEG based sleep staging, we propose a cross-modal knowledge distillation strategy 1 † https://github.com/Mithunjha/EarEEG_KnowledgeDistillation, which is a domain adaptation approach. We employ model architectures from the transformer and convolutional neural network families to demonstrate the model-agnostic nature of the method. Our experiments and analysis validate the …

Light field saliency detection–important due to utility in many vision tasks–still lacks speed and can improve in accuracy. Due to the formulation of the saliency detection problem in light fields as a segmentation task or a memorizing task, existing approaches consume unnecessarily large amounts of computational resources for training, and have longer execution times for testing. We solve this by aggressively reducing the large light field images to a much smaller three-channel feature map appropriate for saliency detection using an RGB image saliency detector with attention mechanisms. We achieve this by introducing a novel convolutional neural network based features extraction and encoding module. Our saliency detector takes 0.4 s to process a light field of size 9× 9× 512× 375 in a CPU and is significantly faster than state-of-the-art light field saliency detectors, with better or comparable accuracy. Furthermore …

Prioritization of data streams in cellular vehicle-to-everything (C-V2X) may lead to unfavorable packet delays in low-priority streams. This paper studies allocating multiple candidate single-subframe resources (CSRs) per vehicle as a solution. It proposes a methodology to determine the number of CSRs for each vehicle based on the total number of neighboring vehicles, and to assign the multiple data streams among them for simultaneous transmission. The numerical results highlight the achievable delay gains of the proposed approach, and its negligible impact on packet collisions.

This paper proposes a novel approach that utilizes differential encoding to overcome the channel estimation problem in communication systems with low-resolution quantization receivers. For differentially encoded data, we derive the maximum likelihood detection rule for the canonical block-2 detectors, employing just two consecutive quantized observations at the channel output and without any receiver-side channel state information. We establish the optimality of this maximum likelihood detection rule within the class of block- detectors, where , under the condition that , with and denoting the number of quantization bits and input alphabet size, respectively. The derived detector has a simple and easily implementable structure, comparing the quantization region indices of consecutive observations to determine the transmitted message index. By leveraging the structure of the derived optimum …

This paper presents an effective method of identifying elephant rumbles in infrasonic seismic signals. The design and implementation of electronic circuitry to amplify, filter, and digitize the seismic signals captured through geophones are presented. A collection of seismic infrasonic elephant rumbles was collected at a free-ranging area of an elephant orphanage in Sri Lanka. The seismic rumbles were converted to spectrograms, and several methods were used for spectral feature extraction. Using LasyPredict, the features extracted using different methods were fed into their corresponding machine-learning algorithms to train them for automatic seismic rumble identification. It was found that the Mel frequency cepstral coefficient (MFCC) together with the Ridge classifier machine learning algorithm produced the best performance in identifying seismic elephant rumbles. A novel method for denoising the spectrum that leads to enhanced accuracy in identifying seismic rumbles is also presented.