Research

Research Vision

Imaging lets us see details of objects or scenes that the eye cannot see, because they are too small or too far away. Computational imaging goes further by building computation into the imaging process itself. This lets it pass the limits of traditional imaging in resolution, spectral range, speed and dynamic range, and so widens what we can observe.

A computational imaging system has two parts. An optical system encodes the light from the target, and an algorithm decodes the measurement to reconstruct the target. The approach is powerful, but two problems still limit its use in practice, variance sensitivity and computational complexity.

The first problem is sensitivity to physical variation. Computational imaging works only when the physical system matches its mathematical model. In the real world, aberrations, misalignment, sensor noise and other changes break this match. When the system and the model do not match, image quality can drop sharply, or the reconstruction can fail completely.

The second problem is the cost of solving the inverse problem. A conventional camera forms an image almost at once. Computational imaging usually needs several measurements or a long iterative algorithm to reconstruct a good image. This takes much longer, so it is hard to use in real-time applications.

These two problems, variance sensitivity and computational complexity, are the main bottlenecks of computational imaging today. They limit both how well it performs and where it can be used.

My research tackles both problems by rethinking how computational imaging is built. Instead of treating the physical system and the numerical model as separate things, I connect them in one imaging pipeline, so that the pipeline itself can correct the mismatch between them.

This work has four connected parts:

Fundamental Theory: the physics behind the imaging process
Numerical System Models: accurate digital models that link the theory to real optical measurements
Computational Inverse Solving: algorithms that invert the forward model to recover the object and the system parameters from the measurements
Interdisciplinary Applications: using this pipeline in other fields

Progress in one part feeds the others. Together they form a new approach, which we call Differentiable Imaging.

Differentiable Imaging handles physical uncertainty in two ways. Deterministic uncertainties, such as known aberrations and misalignment, are built into the numerical model. Random uncertainties, such as sensor noise and changes in the environment, are handled by optimization, using numerical methods or neural networks.

This brings two main benefits:

Flexibility: Because it is built on automatic differentiation, the approach gives a lot of freedom in solving inverse imaging problems. The optical system and the algorithm can be designed together, across the whole physical pipeline.
Deep Learning Integration: Differentiable imaging uses the same backpropagation as neural networks, so it works naturally with deep learning while keeping explicit physical constraints. This makes it well suited to both problems above.

By connecting physical systems with computational models and using modern machine learning, we can build imaging systems that cope better with real-world variation and reconstruct images faster.

In short, differentiable imaging is a machine learning framework built for imaging systems. It embeds physical models in the learning process, so that physics and computation are optimized together, and the results keep their physical meaning and theoretical guarantees, which conventional deep learning cannot give.

Research Areas

My research in light field imaging has followed three connected lines, each aimed at the problems above.

Uncertainty-Aware Computing

This line develops algorithms that find and correct the main system uncertainties, especially optical misalignment. Examples are misalignment correction in Fourier ptychographic microscopy and alignment-free multi-plane phase retrieval. This work shows that redundancy in the data can make up for system uncertainty without losing image quality.

Physics-Aware Computing

This line adds physical priors to the reconstruction, to go beyond what purely data-driven methods can do. Main contributions:

Joint Space-Time Framework: uses physical priors in space and time for reconstruction. It gives accurate 3D particle tracking and flow velocity measurement for fluid science, aerosol science and medical applications.
Model-Based Neural Networks: physics-informed deep learning for 3D imaging systems, which deals with the lack of experimental training data in optical imaging.

Differentiable Imaging

This line develops methods that handle the mismatch between the system and the computation, and make the computation more efficient. Main results:

Single-Shot Complex Field Imaging: models unknown changes in the sample-to-sensor distance
Pixel Super-Resolution Lensless Imaging: corrects errors across multiple measurements
High-Density Imaging: handles multiple scattering of light between objects
Uncertainty-Aware Fourier Ptychography: deals with changes in the light source, aberrations in the optics, and data problems caused by the sensor

Broader Contributions

My work also covers fundamental theory, light field photography, holographic displays, phase imaging and ptychography. Industry projects include coded aperture photography and wearable displays with Samsung. I have also worked on ray-tracing-based metrology, differentiable lens design and scattering imaging.

Key Insights

Three lessons run through this work:

Good computational imaging starts with precise optics and accurate models
Algorithms work best when they are built on the physics
Image quality and real co-design both depend on how well the physical system and the computation work together

Selected Open-Source Code