Department Publications

We present Seg-TTO, a novel framework for zero-shot, open-vocabulary semantic segmentation (OVSS), designed to excel in specialized domain tasks. While current open-vocabulary approaches show impressive performance on standard segmentation benchmarks under zero-shot settings, they fall short of supervised counterparts on highly domain-specific datasets. We focus on segmentation-specific test-time optimization to address this gap. Segmentation requires an understanding of multiple concepts within a single image while retaining the locality and spatial structure of representations. We propose a novel self-supervised objective adhering to these requirements and use it to align the model parameters with input images at test time. In the textual modality, we learn multiple embeddings for each category to capture diverse concepts within an image, while in the visual modality, we calculate pixel-level losses followed by embedding aggregation operations specific to preserving spatial structure. Our resulting framework termed Seg-TTO is a plug-and-play module. We integrate Seg-TTO with three state-of-the-art OVSS approaches and evaluate across 22 challenging OVSS tasks covering a range of specialized domains. Our Seg-TTO demonstrates clear performance improvements (up to 27% mIoU increase on some datasets) establishing new state-of-the-art. Our code and models will be released publicly.

Test-time prompt tuning (TPT) has emerged as a promising technique for adapting large vision-language models (VLMs) to unseen tasks without relying on labeled data. However, the lack of dispersion between textual features can hurt calibration performance, which raises concerns about VLMs' reliability, trustworthiness, and safety. Current TPT approaches primarily focus on improving prompt calibration by either maximizing average textual feature dispersion or enforcing orthogonality constraints to encourage angular separation. However, these methods may not always have optimal angular separation between class-wise textual features, which implies overlooking the critical role of angular diversity. To address this, we propose A-TPT, a novel TPT framework that introduces angular diversity to encourage uniformity in the distribution of normalized textual features induced by corresponding learnable prompts. This uniformity is achieved by maximizing the minimum pairwise angular distance between features on the unit hypersphere. We show that our approach consistently surpasses state-of-the-art TPT methods in reducing the aggregate average calibration error while maintaining comparable accuracy through extensive experiments with various backbones on different datasets. Notably, our approach exhibits superior zero-shot calibration performance on natural distribution shifts and generalizes well to medical datasets. We provide extensive analyses, including theoretical aspects, to establish the grounding of A-TPT. These results highlight the potency of promoting angular diversity to achieve well-dispersed textual features, significantly …

Splatting-based 3D reconstruction methods have gained popularity with the advent of 3D Gaussian Splatting, efficiently synthesizing high-quality novel views. These methods commonly resort to using exponential family functions, such as the Gaussian function, as reconstruction kernels due to their anisotropic nature, ease of projection, and differentiability in rasterization. However, the field remains restricted to variations within the exponential family, leaving generalized reconstruction kernels largely underexplored, partly due to the lack of easy integrability in 3D to 2D projections. In this light, we show that a class of decaying anisotropic radial basis functions (DARBFs), which are non-negative functions of the Mahalanobis distance, supports splatting by approximating the Gaussian function's closed-form integration advantage. With this fresh perspective, we demonstrate varying performances across selected DARB …

Unmanned aerial vehicles are increasingly deployed across diverse domains including security surveillance, precision agriculture, infrastructure inspection, and logistics. However, their rapid proliferation raises critical concerns regarding unauthorized intrusions, smuggling, and adversarial exploitation. Robust drone detection and classification systems are therefore essential. In this paper, we propose a multi-modal framework that fuses frequency-modulated continuous-wave radar and acoustic sensing using a transformer encoder-based architecture. Unlike single-modality approaches, our method exploits cross-modal complementarity to improve robustness against environmental noise, false alarms, and stealthy drone operations. Notably, our approach directly processes raw acoustic waveforms without domain transformations such as spectrograms, reducing preprocessing overhead and improving real-time …

This paper addresses the urgent question of how emerging telecommunication networks, particularly beyond 5G and 6G networks, can be designed and governed to support climate action while limiting their own environmental footprint. We provide a comprehensive and cross-disciplinary analysis of the role of telecommunications in climate change adaptation, mitigation, and sectoral decarbonization, going beyond existing studies that typically focus on isolated technologies or single application domains. We review international climate and telecom policy frameworks, evaluate emerging 5G/6G capabilities, and analyze regulatory pathways for green networks to assess how next-generation connectivity, AI-based energy optimization, remote monitoring systems, and renewable integration can reduce emissions and enhance network resilience. Our findings organize these developments into a novel three-pillar …

Cathodic protection plays a crucial role in preventing the corrosion of cast and ductile iron pipes. However, less attention has been given to the long-term chemical reactions affecting crack growth and failures of pipes with cathodic protection. The present study, therefore, attempted to fill this gap by developing a model where cathodic protection is positioned as the moderating variable in the relationship between physical condition and pipe break frequency. Data from 426 pipes installed between 1955 and 1974 were obtained from a large-scale municipality in Ontario, Canada. The results showed a direct effect of physical condition on the pipe break frequency ( , ), demonstrating that as the physical condition worsens, the rate of pipe breakages increases. The findings further support the long-term moderating role of cathodic protection on physical condition and pipe breakage linkage ( , …

Integrated sensing and communication (ISAC) is expected to be a key enabler for future wireless networks, improving spectral and hardware efficiency by jointly performing radar sensing and wireless communication within a unified framework. This paper proposes BeamNet , an unsupervised deep learning framework for transmit beamforming in dual-function radar-communication systems operating over general fading with imperfect channel state information (CSI). BeamNet maps noisy estimates of the communication and sensing channels to a transmit beamforming vector and is trained end-to-end by maximizing a weighted sum of the communication rate (CR) and sensing rate (SR), thereby learning the CR–SR Pareto frontier without beamforming labels or embedded optimization solvers. Using Rayleigh fading with perfect CSI, we first show that BeamNet reproduces the analytical Pareto-optimal beamforming solutions. We then use BeamNet to characterize, for Nakagami- m and Rician fading, the CR–SR trade-off across a range of fading parameters, and to assess robustness under distribution mismatch between training and test channels. Finally, under imperfect CSI, we demonstrate that BeamNet yields CR–SR trade-offs that are consistently sandwiched between the perfect-CSI and mismatched analytical baselines, outperforming the closed-form beamformer applied to imperfect CSI and recovering part of the performance loss caused by channel estimation errors. These results indicate that unsupervised learning offers a flexible and robust approach to ISAC beamforming in fading environments with imperfect channel knowledge.

Separation of simultaneously active multiple speakers is a difficult task in environments with strong reverberation and many background noise sources. This paper uses the relative transfer matrix (ReTM), a generalization of the relative transfer function of a room, to propose a simple yet effective approach for separating concurrent speakers using noisy multichannel microphone recordings. The method is built upon the covariance subtraction to enable practical applicability to mixtures of independent sound sources. The proposed method (i) allows multiple speech and background noise sources, (ii) includes reverberation, (iii) does not need the knowledge of the locations of speech and noise sources nor microphone locations and their relative geometry, and (iv) uses a relatively small segment of recordings for training. We illustrate the speaker separation performance through a simulation study, particularly under …

Isolating a desired speech signal in noisy multi-talker conversational scenarios is a key requirement for augmented reality (AR) wearable microphone array systems. In this work, a binaural target speaker extraction (TSE) framework, termed BiTSE, is proposed. It leverages both spatial and temporal cues, specifically the direction-of-arrival (DoA) of the target speaker and corresponding voice activity information, to guide the extraction process. Built upon a binaural signal denoising architecture, our model integrates three key enhancements: (i) a DoA-aware attention mechanism using cyclic positional embeddings, (ii) a timestamp-based masking strategy that utilizes speaker activity to suppress non-target segments, and (iii) a novel two-stage loss optimization strategy that first trains the model for robust denoising and then fine-tunes it to improve perceptual quality. Evaluations on the SPeech Enhancement for Augmented Reality (SPEAR) challenge dataset demonstrate that the proposed BiTSE consistently improves upon conventional approaches, leading to enhanced signal fidelity and perceptual quality.

Intelligent control of RF transceivers adapting to dynamic operational conditions is essential in the modern and future communication systems. We propose a multi-agent neurosymbolic AI system, where AI agents are assigned for circuit components. Agents have an internal model and a corresponding control algorithm as its constituents. Modeling of the IF amplifier shows promising results, where the same approach can be extended to all the components, thus creating a fully intelligent RF system.

Whole Slide Image (WSI) analysis serves as the fundamental method for the clinical diagnosis of cancer in digital histopathology. Recent advances in Multimodal Large Language Models (MLLMs) have enabled state-of-the-art (SOTA) automated histopathology report generation. However, MLLMs reason purely through distributions encoded in their neural parameters, limiting the amount of knowledge and reasoning logic they can contain given the model scale. Their black-box reasoning poses another challenge to the human interpretation of intermediate reasoning. We address these issues by utilizing knowledge graphs to inject external structured knowledge for MLLMs to reason over, while making this reasoning component visible. Knowledge graphs are capable of partially externalizing the containment of medical knowledge, an area that remains underexplored in pathology report generation. We introduce PathKGen, a knowledge graph-assisted MLLM for histopathology report generation. PathKGen has two main branches: (1) a visual path that learns visual token inputs to the MLLM using pathology foundation models, and (2) a knowledge path that learns knowledge token inputs by extracting key morphological features of WSIs through querying a pathology-specific knowledge graph. A comprehensive evaluation on WSI-Bench dataset shows that PathKGen surpasses previous SOTA methods while revealing diagnostic reasoning as visually intuitive graphs. Due to its modular design, the proposed knowledge path is complementary to vision-aligned MLLMs and can be extended to other applications by curating domain-specific knowledge …

Low-rank adaptation (LoRA) attaches small low-rank matrices to a frozen backbone, and public repositories host many LoRAs per backbone, each for a different concept or task. Combining several LoRAs in one generation forces a trade-off. Merging their weights into one update needs a single forward pass, but LoRAs trained for different concepts interfere and quality drops. Running each LoRA in its own forward pass (composite) preserves quality, but inference time grows linearly with the number of active LoRAs. We study a property of LoRA pools overlooked by prior multi-LoRA work. In real LoRA pools, the stacked low-rank factors need far fewer directions than their summed ranks. Using singular value decomposition, we find the minimal subspace that keeps 99% of the factor energy. On three diffusion pools, only 46% to 57% of the summed LoRA ranks is needed, and the subspace grows sub-linearly with pool size. A 48-LoRA sub-pool of LoRA-Hub on FLAN-T5 tasks shows comparable overlap (62%), while the diversity-curated LoRA-Retriever benchmark sits near the no-overlap baseline (96%), so pool composition rather than modality predicts where compression is large. We build on this overlap with LoRASpace, a training-free method that computes the subspace once per pool and reuses it in two inference modes, a static mode at merge cost and a dynamic mode for input-dependent routing. On the diffusion pools, static LoRASpace matches composite quality at merge cost, and dynamic LoRASpace is more than 6x faster than composite with six active LoRAs on a 17-LoRA pool, mainly because it runs one backbone pass instead of six …

Feed-forward Gaussian models have made novel view synthesis almost instantaneous. Feed-forward methods predict Gaussians directly, either in two stages or in a single merged pass. Using two separately trained models is attractive because each can specialize in its own objective: one for accurate pose estimation and the other for high-quality reconstruction, rather than forcing a single jointly trained network to compromise between the two. A natural instantiation pairs a frozen pose estimator, VGGT, with a frozen reconstruction backbone, TokenGS. Prior work, such as VGGT-X and 3R-GS, also adopts a two-stage design, but attributes the resulting quality degradation to noisy pose estimates and introduces learned refinement. We show that this is not the primary source of the problem. Through per-camera analysis across 107 scenes from two datasets, DL3DV and MipNeRF360, using COLMAP poses as the reference, we find that the rotation error after alignment is small (median 1.6 degrees), whereas the translation scale is offset by a large, dataset-consistent factor. Controlled ablations over 140 DL3DV scenes show that the primary obstacles are two integration failures rather than pose error: a coordinate-frame inconsistency and a translation-scale mismatch, both of which can be corrected deterministically. With these fixes, a frozen VGGT driving a frozen TokenGS achieves 22.47 PSNR, within 0.51 dB of the COLMAP reference, with no additional training, when poses are estimated from all scene views. In the sparse setting, where reconstruction uses only four context images under the standard pose-free protocol, the same pipeline …

4D understanding and reasoning is a fundamental capability for embodied AI agents operating in dynamic physical environments. However, existing vision encoders are largely limited to static 2D images or 3D point clouds without temporal modeling, or to 2D videos that lack accurate geometric depth reasoning. Consequently, current approaches fail to jointly capture spatial structure and motion evolution in dynamic scenes. We present CL4D, the first foundational 4D vision encoder that directly operates on dynamic point clouds, trained with a contrastive learning objective to align spatio-temporal geometric representations with natural language descriptions. By learning a shared embedding space between text and 4D scene dynamics, CL4D enables zero-shot motion-to-text and text-to-motion retrieval in dynamic environments and serves as a foundational 4D vision encoder for downstream 4D vision–language tasks …

Semi-supervised Domain Generalization (SSDG) addresses the challenge of generalizing to unseen target domains with limited labeled data. Existing SSDG methods highlight the importance of achieving high pseudo-labeling (PL) accuracy and preventing model overfitting as the main challenges in SSDG. In this light, we show that the SSDG literature’s excessive focus on PL accuracy, without consideration for maximum data utilization during training, limits potential performance improvements. We propose a novel approach to the SSDG problem by aligning the intermediate features of our model with the semantically rich and generalized feature space of a Vision Language Model (VLM) in a way that promotes domain-invariance. The above approach is enhanced with effective image-level augmentation and output-level regularization strategies to improve data utilization and minimize overfitting. Extensive …

Identification of less-articulated objects using single-channel images, such as thermal images, is important in many applications, such as surveillance. However, in this domain, existing methods show poor performance due to high similarity among objects of the same category in the absence of color information (overlooking shape information) and de-emphasized texture information. Furthermore, variability in viewpoint adds more complexity as the features vary from side to side. We address these issues by constructing viewpoint-conditioned feature vectors and area-specific feature comparisons in separate feature spaces. These interventions enable leveraging the advancements of existing RGB-pre-trained ViT feature extractors while effectively adapting them to address the challenges specific to the thermal domain. We test our system with RGBNT100 (IR) vehicle dataset and a thermal maritime dataset acquired …

Vision-Language Navigation (VLN) enables robots to navigate through environments using natural language instructions, making human-robot interaction intuitive. Traditional VLN models often rely on navigation graphs, 360-degree views, and perfect localization which pose significant challenges when adapting these models to real-world settings. This work addresses these limitations by performing a simulation-to-real domain shift of a VLN approach that operates in continuous environments without requiring navigation graphs or panoramic views. The proposed system integrates vision-language models that align visual inputs and linguistic instructions within a shared embedding space, facilitating natural language-driven navigation. We employ a Cross-Modal Attention (CMA) based architecture trained on an existing dataset in a simulated environment and fine-tune it using real-world data collected from a …

Spatial transcriptomics is an emerging field that enables the identification of functional regions based on the spatial distribution of gene expression. Integrating this functional information present in transcriptomic data with structural data from histopathology images is an active research area with applications in identifying tumor substructures associated with cancer drug resistance. Current histopathology-spatial-transcriptomic region segmentation methods suffer due to either making spatial transcriptomics prominent by using histopathology features just to assist processing spatial transcriptomics data or using vanilla contrastive learning that make histopathology images prominent due to only promoting common features losing functional information. In both extremes, the model gets either lost in the noise of spatial transcriptomics or overly smoothed, losing essential information. Thus, we propose our novel …

Semi-supervised domain generalization (SSDG) leverages a small fraction of labeled data alongside unlabeled data to enhance model generalization. Most of the existing SSDG methods rely on pseudo-labeling (PL) for unlabeled data, often assuming access to domain labels--a privilege not always available. However, domain shifts introduce domain noise, leading to inconsistent PLs that degrade model performance. Methods derived from semi-supervised learning baselines suffer particularly from lower PL accuracy, reducing the effectiveness of unlabeled data. We address this in a more challenging setting---domain-label agnostic SSDG---, where domain labels for unlabeled data are not available during training. First, we propose a feature modulation strategy that enhances class-discriminative features while suppressing domain-specific information. This modulation shifts features toward Similar Average Representations---a modified version of class prototypes---that are robust across domains, encouraging the classifier to distinguish between closely related classes and feature extractor to form tightly clustered, domain-invariant representations. Second, to mitigate domain noise and improve pseudo-label accuracy, we introduce a loss-scaling function that dynamically lowers the fixed confidence threshold for pseudo-labels, optimizing the use of unlabeled data. With these key innovations, our approach achieves significant improvements on four major domain generalization benchmarks--even without domain labels. We will make the code available.

Maximum likelihood (ML) detection in massive multiple-input multiple-output (MIMO) systems is an NP-hard combinatorial problem whose complexity grows exponentially with the number of transmit antennas and the modulation order. The Quantum Approximate Optimization Algorithm (QAOA) has recently emerged as a promising near-term quantum heuristic for such problems; however, existing qubit-based formulations require qubits per -QAM symbol, inflating both circuit width and depth. In this paper, we propose a qudit-based QAOA framework that encodes each -QAM symbol directly into a pair of -dimensional quantum states, where . We derive a Hermitian cost Hamiltonian via a real-imaginary decomposition of the constellation, and introduce a generalized cyclic-shift mixer that ensures ergodic exploration of the search space without binary encoding. The proposed formulation reduces the number of quantum carriers from to and the Hamiltonian interaction complexity from to , while preserving the geometric structure of the constellation. Bit-error-rate (BER) simulations for MIMO systems employing 4-QAM and 16-QAM over Rayleigh fading channels, with circuit depth , demonstrate that the proposed detector closely approximates the performance of the optimal classical maximum-likelihood (ML) detector, while consistently outperforming the conventional minimum mean square error (MMSE) detector.

This paper characterizes the communication–sensing rate tradeoff of an active reconfigurable intelligent surface (RIS)-assisted integrated sensing and communication system operating under residual environmental clutter. The BS beamformer and active-RIS coefficients are jointly designed while accounting for amplifier noise on the forward and return sensing paths, a global RIS power budget, and perelement amplification limits. Residual clutter introduces a beamformer-dependent term in the sensing denominator, leading to a generally indefinite, nonconvex sensing-QoS constraint in the communication-centric (C-C) design and a fractional quadratic beamformer objective in the sensing-centric (S-C) design. We develop alternating optimization procedures for both formulations based on established MM, Kronecker-vectorization, and Dinkelbach techniques. Their QoS-threshold sweeps are refined and filtered to …

Low-resolution analog-to-digital converters (ADCs) are a promising way to cut power consumption in future 6G receivers. However, they break many assumptions behind traditional communication system design. Instead of optimizing each block separately, this work uses an end-to-end autoencoder (AE) to jointly learn and optimize the transmitter and receiver for systems with phase quantization. Two key obstacles arise: (1) phase quantization is non-differentiable, which blocks gradientbased training back to the transmitter, and (2) standard crossentropy loss leads to poor block error rate (BLER) performance. To overcome these issues, we introduce a differentiable approximation of phase quantization and a new loss function that adds a distance-based regularization term to improve symbol separability. Simulations show that the proposed AE achieves near-optimal BLER for uncoded -ary phase shift keying …

This paper investigates sum-rate maximization in a multi-user downlink system assisted by a movable RIS (MRIS) under dynamic blockage conditions. Unlike conventional fixed RIS (FRIS) deployments, the RIS is modeled as a rigid panel whose anchor position can be continuously adjusted within a predefined region, introducing an additional spatial degree of freedom. In practical environments, temporary blockages arise from moving objects or environmental changes, significantly degrading communication links. To capture this effect, a geometric blockage model is incorporated into the RIS-to-user channels, where propagation paths are selectively obstructed based on line-of-sight conditions. The base station-to-RIS channel is modeled using a field-response-based multipath formulation, while RIS-to-user links follow distance-dependent path loss with phase propagation. For a fixed position, a common RIS …

Low-resolution digital-to-analog converters (DACs) and analog-to-digital converters (ADCs) are important enablers of energy-efficient wideband uplinks, but they introduce nonlinear distortions whose impact depends on the operating point and, in multi-user systems, on the time- and frequency-resource split. This paper studies quantization-aware uplink resource allocation for multi-user orthogonal multiple access (OMA), focusing on frequency division multiple access (FDMA) and time division multiple access (TDMA) with finite-resolution quantization at both the user transmitters and the base station receiver. Using Bussgang-style decompositions for power-preserving DAC quantization and mid-point ADC quantization, we derive a closed-form, conservative lower bound on the per-user achievable rate in terms of the allocation variables, channel gains, noise level, and quantization distortion factors. Under this …

Noncoherent phase shift keying receivers with coarse phase quantization require one additional bit of phase resolution than their coherent counterparts to achieve full diversity. In this letter, we propose a differential encoding scheme that applies a fixed, resolution-dependent phase shift to the transmitted symbols to remove this identified one-bit performance penalty. The impact of phase quantization on the error performance is characterized under the proposed differential encoding strategy. We show that the proposed differential encoding enables an -bit phase-quantized noncoherent receiver to match the error performance of a conventional -bit phase-quantized noncoherent receiver. Ultimately, our results demonstrate that the one-bit performance penalty previously observed in phase-quantized noncoherent receivers is not a fundamental limitation due to the lack of receiver channel state information, but …