ChromaFusionNet (CFNet): Natural Fusion of Fine-grained Color Editing

Yi Dong1*, Yuxi Wang12, Ruoxi Fan2, Wenqi Ouyan3, Zhiqi Shen2*, Peiran Ren3*, Xuansong Xie3
1Alibaba-NTU Singapore Joint Research Institute, Nanyang Technological University, Singapore 2School of Computer Science and Engineering, Nanyang Technological University, Singapore 3Institute for Intelligent Computing, Alibaba Group *Corresponding Author
Paper Supplementary Materials ArXiv Code
Visual 1

CFNet can effectively alleviate the spatial color inconsistency in direct composition of multiple color edits without changing the overall look and the color of the non-boundary regions.

BibTeX

@inproceedings{CFNet2024,
      author = {Yi Dong and Yuxi Wang and Ruoxi Fan and Wenqi Ouyang and Zhiqi Shen and Peiran Ren and Xuansong Xie},
      title = {ChromaFusionNet (CFNet): Natural Fusion of Fine-grained Color Editing},
      booktitle = {AAAI},
      year = {2024}
      }

CFNet: Pioneering Color Fusion beyond Existing Method Limitations

Architecture

A comparative visual analysis of color fusion techniques across three related domains: image harmonization, color transfer, and image inpainting. The figure juxtaposes results from leading methods—Harmonizer, S2CRNet, two variants of Color Matcher (MKL and HM-MVGD-HM), CoordFill, and ZITS—against our CFNet. Note the discrepancies in boundary blending, color consistency, and texture preservation, with CFNet exemplifying superior performance in all aspects.

Seamless Integration of CFNet into Post-production: Eradicating Manual Edge Refinement and Saving Significant Time

Architecture

CFNet can be easily integrated into current pipelines. In conventional approaches for region-specific color enhancement, refining a coarse mask requires intricate manual adjustments, employing over 20 controls for color and geometric attributes, followed by meticulous visual inspections. CFNet radically transforms this workflow by automating edge refinement for these tasks. With CFNet, upon receiving a coarse object mask, potential regions prone to spatial color inconsistencies are swiftly detected using renowned edge detection techniques, such as Canny. Subsequently, CFNet processes the mask and its direct composition, delivering a region-specific color edit that's both consistent and seamless.

Architecture

Architecture

Tackling spatial inconsistencies from varied color edits, CFNet treats them as an image color inpainting problem. The process commences with an RGB to CIELab conversion, followed by mask application to produce an incomplete ab channel. The encoder, built on Vision Transformer (ViT), processes the input into global receptive fields, while the pixel-shuffling based decoder yields the coarse inpainting outcome. Lastly, the Refinement Module, rooted in the Residual-in-Residual Dense Block (RRDB), further refines the output, capitalizing on a deep and feature-rich design, ensuring boundary smoothing and enhanced color coherence.

Application in Fine-grained Color Enhancement

Architecture

While region-specific color adjustments boost visual appeal and expressiveness, they can introduce boundary issues. CFNet effectively blends these areas, removing inconsistencies and improving visual quality.

Additional Visual Results

In this section, we present a series of visual examples that underscore the efficacy of CFNet in color fusion tasks. Through these diverse examples, we highlight CFNet's ability to adeptly address the shortcomings inherent in existing methods. We further extend the analysis to encompass comparisons with image inpainting techniques. This extended evaluation provides a more thorough assessment of CFNet's performance and reveals its wide-ranging applicability in the context of color fusion tasks.