Designing Phase Masks for Under-Display Cameras

Diffractive blur and low light levels are two fundamental challenges in producing high-quality photographs in under-display cameras (UDCs). In this paper, we incorporate phase masks on display panels to tackle both challenges. Our design inserts two phase masks, specifically two microlens arrays, in front of and behind a display panel. The first phase mask concentrates light on the locations where the display is transparent so that more light passes through the display, and the second phase mask reverts the effect of the first phase mask. We further optimize the folding height of each microlens to improve the quality of PSFs and suppress chromatic aberration. We evaluate our design using a physically-accurate simulator based on Fourier optics. The proposed design can double the light throughput while improving the invertibility of the PSFs. Lastly, we discuss the effect of our design on the display quality and show that implementation with polarization-dependent phase masks can leave the display quality uncompromised.

Motivation

Diffractive blur in a UDC is produced by the small openings on the display pixels that have sizes comparable to the wavelength of incident light. Smaller opening results in a more severe diffraction blur. Our key intuition is to optically expand the size of display openings, i.e. let a larger portion of light pass through display openings, by inserting phase masks.

Proposed design: Double-sided microlens array

Our design is to place two microlens arrays (MLA) with equal focal lengths on both sides of the display such that the display panel lies in the focal plane of both MLAs, as shown in Figure (a) on the right. The first microlens array modulates light incident on the display so that, after propagating for some distance, most of the intensity of the wavefront is concentrated at the display openings. The second microlens array diverges the light back to a parallel beam. Compared to UDCs with a pure display, the proposed setup allows a larger portion of light to reach to camera's main lens and therefore improves the conditioning of incident wavefront and SNR. However, as shown in Figure (b), the microlens array in front of the display also modulates light emitting from the display pixels and negatively affects the display quality.

Alleviate the effect on display quality

Polarization-dependent implementation. To keep the display quality untouched, we implement MLAs as polarization-dependent optics and place a pair of orthogonal linear polarizers on both sides of the display panel. In this way, any light emitting from the display is free of modulation of the phase mask. A detailed working principle is described in the paper. Since polarization-dependent optical elements are only available in thin formats, we fold the microlens arrays into thin phase masks, as shown on the right.

Optimizing the height map of phase mask. A thin phase plate with a fixed height tends to produce chromatic aberration. We therefore optimize a different height for each microlens. This optimization improves the overall PSF quality across RGB channels and eliminates chromatic aberration.

Qualitative results

We show qualitative results from UDCs under TOLED displays without and with our proposed phase masks. All TOLED displays have a pixel density of around 600 Dot Per Inch. Our phase masks have a thickness of around 5 microns. Note that we are able to largely suppress the ringing artifacts simply by adding phase masks to TOLED displays.

Quantitative results

We compare TOLED without and with two sets of proposed phase masks that have thicknesses of around 1 μm and 5 μm. For each thickness, we compare three choices of wrapping heights — a fixed height determined by λ_0 = 530 nm, different heights determined by wavelengths uniformly sampled from 400 nm to 700 nm, and optimized heights. Figure 2 shows that the proposed setups outperform TOLED at all light levels. At 1 μm, the optimized height map largely outperforms the fixed one; while at 5 μm, different designs perform similarly. Because thinner phase masks have more phase wrappings and are more sensitive to the selection of d0. At 5μm, the phase mask is quite similar to a thick lens and the system performance is more consistent across different choices of d_0.

Comparison with other displays

We compare our design with two display layouts commonly used in smartphone screens, TOLED and POLED, and two display layouts designed specifically for UDCs. Table 1 summarizes the design, LTR, and imaging performance. We list averaged PSNR(↑) and SSIM(↑) across different scenes and light levels. Figure 3 shows qualitative results. Ours falls into the category of requiring no change of the display openings and outperforms the other two common displays, TOLED and POLED. While Yang and Sankaranarayanan and ZTE Axon have higher imaging quality, the modifications of the display have nontrivial negative effects on the display quality.

Table 1. Comparing ours with other OLED displays.

For an in-depth description of the technology behind this work, please refer to our paper.

Anqi Yang, Eunhee Kang, Hyong-Euk Lee, and Aswin C. Sankaranarayanan, "Designing Phase Masks for Under-Display Cameras", In IEEE/CVF International Conference on Computer Vision (ICCV), 2023

Code and data are available at the following GitHub repository.