DC2: Dual-Camera Defocus Control by Learning to Refocus

Hadi Alzayer, Abdullah Abuolaim, Leung Chun Chan, Yang Yang, Ying Chen Lou, Jia-Bin Huang, and Abhishek Kar
IEEE/CFV Conference on Computer Vision and Pattern Recognition (CVPR), 2023

📝 Paper: https://arxiv.org/abs/2304.03285
🌐 Website: https://defocus-control.github.io/

📄 Abstract: Smartphone cameras today are increasingly approaching the versatility and quality of professional cameras through a combination of hardware and software advancements. However, fixed aperture remains a key limitation, preventing users from controlling the depth of field (DoF) of captured images. At the same time, many smartphones now have multiple cameras with different fixed apertures - specifically, an ultra-wide camera with wider field of view and deeper DoF and a higher resolution primary camera with shallower DoF. In this work, we propose DC2, a system for defocus control for synthetically varying camera aperture, focus distance and arbitrary defocus effects by fusing information from such a dual-camera system. Our key insight is to leverage real-world smartphone camera dataset by using image refocus as a proxy task for learning to control defocus. Quantitative and qualitative evaluations on real-world data demonstrate our system's efficacy where we outperform state-of-the-art on defocus deblurring, bokeh rendering, and image refocus. Finally, we demonstrate creative post-capture defocus control enabled by our method, including tilt-shift and content-based defocus effects.

🎵 Music: Happy and Joyful Children www.youtube.com/watch?v=lv1YPeP1O8k
Released by: Oleg Mazur @fm_freemusic

Jia-Bin Huang

A research approach for refocusing photos after they are taken using dual smartphone cameras (wide and ultra-wide). Instead of requiring special hardware or time-consuming focus stacking, a learning-based model is trained using a proxy task: pairs of images captured at different focal planes teach the model to predict refocused outputs. After training, users can adjust the focal plane and depth of field by specifying a target focus map.

Out-of-focus Photos No More