Research

Research Vision

Imaging allows us to perceive details of objects or scenes that are normally invisible to the human eye due to their size or distance. Computational imaging takes this a step further by embedding computational techniques directly into the imaging workflow, pushing beyond traditional imaging limitations, enabling ultra-high resolution, multi-dimensional, hyperspectral, ultra-fast, and high dynamic range imaging. This fusion of optics and computation fundamentally expands our ability to observe and understand the world.

At its core, computational imaging operates through two key components: an optical system that encodes light radiation from the target, and computational algorithms that decode this information to represent the target. While powerful, this approach faces two critical challenges that currently limit its practical applications: physical sensitivity and computational load. The first challenge stems from the system’s sensitivity to physical variations. Computational imaging requires precise harmony between the physical system and its mathematical model. However, real-world factors like optical aberrations, mechanical misalignments, sensor noise, and unpredictable physical variations can disrupt this harmony. This system-computation mismatch can significantly degrade imaging performance or fail the imaging. The second challenge involves imaging speed. Unlike conventional imaging, which produces images almost instantly, computational imaging usually relies on multiple measurements or complex algorithms that often require multiple iterations to reconstruct a high performance image. These processes result in significantly longer imaging times, making the technology impractical for applications requiring real-time process. These fundamental challenges — physical sensitivity and computational load — currently represent the primary bottlenecks in computational imaging, constraining both its performance capabilities and broader applications.

My research addresses these challenges through an innovative approach that reimagines computational imaging from the ground up. Instead of treating the physical system and numerical model as separate entities, I propose creating a seamless bridge between them through a unified imaging pipeline that inherently resolves system-computation mismatches.
This research strategy encompasses four interconnected facets:

Fundamental Theory: Establishing the theoretical foundations that govern the underlying physical phenomena;
Numerical System Models: Developing precise digital representations that bridge theoretical formulations with actual optical measurements;
Computational Inverse Solving: Implementing sophisticated algorithms to extract object states and system parameters from optical measurements by inverting the forward model;
Interdisciplinary Applications: Exploring practical implementations of this unified pipeline across various fields.

These elements work together in a dynamic sequence, where advances in one area inform and improve the others. This interconnected approach converges into a new paradigm: Differentiable Imaging.

Differentiable Imaging employs a two-pronged approach to handle physical uncertainties: it constructs numerical models that explicitly account for deterministic uncertainties (such as known optical aberrations and system misalignments), while developing optimization strategies to handle stochastic uncertainties (such as sensor noise and environmental variations) through numerical methods or neural networks.

This comprehensive approach to uncertainty management offers two key advantages:

Flexibility: The automatic differentiation principles underlying this approach provide unprecedented freedom in solving inverse imaging problems, enabling flexible design of both imaging systems and computational algorithms. This allows for co-design of optical systems and computational algorithms in an all-in-loop manner.
Deep Learning Integration: By sharing common ground with neural network backpropagation, differentiable imaging seamlessly integrates with deep learning techniques while maintaining explicit physical constraints. This combination makes differentiable imaging particularly powerful for addressing both core challenges of computational imaging.

By bridging the gap between physical systems and computational models while leveraging modern machine learning techniques, we can create imaging systems that are both more robust to real-world variations and capable of faster image reconstruction.

In essence, differentiable imaging represents an advanced machine learning framework specifically designed for imaging systems—one that embeds physical models into the learning architecture, enabling end-to-end optimization while maintaining physical interpretability and theoretical guarantees that conventional deep learning approaches cannot provide.

Past Research

Building upon computational imaging’s foundation in encoding light radiation fields, my past research focuses on scalar wave field imaging and its applications. To address key challenges in wave field imaging, I have developed several complementary research strategies: Digital/imaging processing techniques that develop algorithmic solutions to identify and retrieve key system uncertainties; Physics-informed methods that incorporate domain-specific physics priors and construct physics-aware neural networks; Differentiable imaging approaches that effectively manage physical uncertainties. These strategic investigations have resulted in several significant contributions:

Uncertainty-aware computing advances algorithmic solutions to identify and compensate for key system uncertainties, particularly misalignment. Notable achievements include misalignment correction in Fourier Ptychographic Microscopy and Align-free multi-plane phase retrieval. These works demonstrate that precise optical systems are critical for achieving optimal imaging performance in computational imaging, while system uncertainties may be compensated through data redundancy.
Physics-aware computing develops computational decoding algorithms that leverage physical priors to enhance imaging performance. Key contributions include: Joint space-time framework, which utilizes spatial-temporal physical priors for image reconstruction, enabling precise 3D particle tracking and flow velocity measurements crucial for fluid science, aerosol science, and medical applications. Additionally, our Model-based neural network for 3D imaging systems addresses the challenge of limited experimental training data through physics-informed deep learning architectures, providing innovative solutions to data scarcity in optical imaging applications.
Differentiable imaging pioneers novel methodologies to manage system-computation mismatches and improve computational efficiency. Significant advances include: single-shot complex field imaging that successfully models unpredictable sample-sensor distance variations; pixel-super resolution lensless imaging that addresses errors in multiple measurements; high-density imaging that resolves multi-scattering light-object interactions; and uncertainty-aware Fourier Ptychography that tackles various system-level challenges including light source variations, optical element aberrations, and sensor-induced data quality issues.

Beyond my primary research focus, my work has extended into multiple domains of optics and imaging. I have contributed to fundamental theory development, advanced light field photography, and innovated holographic display techniques. My exploration of diverse imaging modalities has included significant work in phase imaging and Ptychography. I have also engaged extensively with industry and government through impactful collaborations. Notable among these are partnerships with Samsung on projects including ‘‘Coded Aperture Photography” and ‘‘Wearable Display: Head mounted devices.” Additionally, I have made substantial contributions to various specialized areas, including ray-tracing engines for self-calibrated metrology, end-to-end lens design, and scattering imaging. This breadth of research and diverse experience has provided me with comprehensive expertise in computational imaging.

My research framework, employs a problem-driven strategy that connects fundamental science with practical implementation. This approach systematically tackles computational imaging challenges through rigorous methodology development, targeting key applications in system design, sensing/metrology, biomedical imaging, and fluid dynamics. Through my research journey, three crucial insights have emerged:

The foundation of effective computational imaging lies in precise optical systems and accurate modeling;
Computational algorithms must be deeply rooted in physical principles to achieve optimal performance;
Managing the interplay between physical systems and computational methods is essential for both imaging quality and true co-design implementation.

While my work in differentiable imaging has demonstrated significant promise in addressing these challenges, several critical frontiers remain unexplored. My future research will focus on these unexplored territories to fully realize the potential of differentiable imaging.

Selected opensource code