74 research outputs found
Efficient Unified Demosaicing for Bayer and Non-Bayer Patterned Image Sensors
As the physical size of recent CMOS image sensors (CIS) gets smaller, the
latest mobile cameras are adopting unique non-Bayer color filter array (CFA)
patterns (e.g., Quad, Nona, QxQ), which consist of homogeneous color units with
adjacent pixels. These non-Bayer sensors are superior to conventional Bayer CFA
thanks to their changeable pixel-bin sizes for different light conditions but
may introduce visual artifacts during demosaicing due to their inherent pixel
pattern structures and sensor hardware characteristics. Previous demosaicing
methods have primarily focused on Bayer CFA, necessitating distinct
reconstruction methods for non-Bayer patterned CIS with various CFA modes under
different lighting conditions. In this work, we propose an efficient unified
demosaicing method that can be applied to both conventional Bayer RAW and
various non-Bayer CFAs' RAW data in different operation modes. Our Knowledge
Learning-based demosaicing model for Adaptive Patterns, namely KLAP, utilizes
CFA-adaptive filters for only 1% key filters in the network for each CFA, but
still manages to effectively demosaic all the CFAs, yielding comparable
performance to the large-scale models. Furthermore, by employing meta-learning
during inference (KLAP-M), our model is able to eliminate unknown
sensor-generic artifacts in real RAW data, effectively bridging the gap between
synthetic images and real sensor RAW. Our KLAP and KLAP-M methods achieved
state-of-the-art demosaicing performance in both synthetic and real RAW data of
Bayer and non-Bayer CFAs
- …