Department Publications

Since its introduction, diffractive deep neural networks (D2NN) have been an emerging technology with many useful applications, such as 3D object recognition, saliency segmentation, and quantitative phase microscopy. However, fabricating D2NNs operating in the visible range has not been performed due to the complexities of fabricating nano-scale elements. Recent advancements such as Implosion Fabrication have made it possible to fabricate such networks using a discrete number of phase weights. We propose a quantization-aware training approach for D2NNs through modeling the quantization process in a differentiable manner using a sigmoid-based quantization function, facilitating the fabrication process. We also propose an efficient training schedule to guide the optimization process to converge to a better minima despite the limited number of quantization levels. Our method is simulated and …

We propose a novel attention-based deep learning framework for reconstructing images from compressively sensed measurements. Our method leverages knowledge of the sampling basis and structured sampling patterns, highlighting the importance of learning adaptive features from measurements.

Compressive sensing (CS) reconstructs images from sub-Nyquist measurements by solving a sparsity-regularized inverse problem. Traditional CS solvers use iterative optimizers with hand crafted sparsifiers, while early data-driven methods directly learn an inverse mapping from the low-dimensional measurement space to the original image space. The latter outperforms the former, but is restrictive to a pre-defined measurement domain. More recent, deep unrolling methods combine traditional proximal gradient methods and data-driven approaches to iteratively refine an image approximation. To achieve higher accuracy, it has also been suggested to learn both the sampling matrix, and the choice of measurement vectors adaptively. Contrary to the current trend, in this work we hypothesize that a general inverse mapping from a random set of compressed measurements to the image domain exists for a given measurement basis, and can be learned. Such a model is single-shot, non-restrictive and does not parametrize the sampling process. To this end, we propose MOSAIC, a novel compressive sensing framework to reconstruct images given any random selection of measurements, sampled using a fixed basis. Motivated by the uneven distribution of information across measurements, MOSAIC incorporates an embedding technique to efficiently apply attention mechanisms on an encoded sequence of measurements, while dispensing the need to use unrolled deep networks. A range of experiments validate our proposed architecture as a promising alternative for existing CS reconstruction methods, by achieving the state-of-the-art for metrics of …

With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. However, the speed of current QPM systems is limited by electronic hardware. To improve throughput further, here we propose differentiable optical-electronic quantitative phase microscopy (∂μ) that acquires images in a compressed form such that more information can be transferred beyond the electronic hardware bottleneck. The proposed microscopy uses optical feature extractors as image compressors. The resultant intensity distribution is then decompressed into QPM images by a reconstruction network. By optimizing optical-electronic parameters in an end-to-end manner, our method can improve the QPI throughput from Hours to Seconds (more than an order of magnitude).