At the time that I developed the adaptive homogeneity-directed (AHD) demosaicking algorithm, demosaicking was the next exciting problem in the imaging world. With a timely publication and the help of Dave Coffin (author of DCRaw) and Paul J. Lee (contributor to DCRaw), AHD succeeded as one of the most widely adopted demosaicking methods. Nearly ten years later, the sensor resolution has exceeded the resolution that the optics can deliver. Once a hot research topic, demosaicking now receives far less attention today.
“So, is demosaicking dead?”
Somehow people take me as a spokesperson for demosaicking, and I am asked this question often. Very often. My answer has been “No, demosaicking is not dead.” In fact, there’s many treasures yet to be uncovered.
First of all, it is true that demosaicking *research* has limited impact on camera design today. A poor handling on demosaicking will certainly degrade image quality, so demosaicking certainly qualifies as an “important” or at least “relevant” problem. But the newest demosaicking algorithm will not yield significantly better results than the AHD in most scenarios. In other words, the existing methods are “good enough” for practical purposes.
But, what many overlook is the fact that other problems in camera pipeline are intimately connected with the demosaicking. For example, most camera manufacturers consider either pre-demosaicking or post-demosaicking image enhancement steps. This is the result of the fact that various imaging algorithms (such as denoising, deblurring, white balance) are developed separately from demosaicking. Because of this, you should be suspicious of image processing or computer vision papers that attribute any unintended outcomes to demosaicking. (this happens quite frequently)
Let’s be practical and admit that sensors based on color filter array (CFA) will be around for a long time. The reality is that image enhancement and computer vision techniques can squeeze many extra mileages by coupling their methods to demosaicking. Paying attention to the CFA does pay dividends. Here are some examples.
- Joint demosaicking and denoising outperforms demosaicking and denoising applied separately. In fact, you can couple your favorite denoising algorithm with our joint demosaicking and denoising framework.
- Our universal demosaicking algorithm can interpolate any CFA pattern.
- Single-shot high dynamic range imaging using conventional camera hardware is now a possibility using the properties of CFA.
- Binning artifacts can be explained using the properties of demosaicking.
- Color cross-talk artifacts can be explained using the properties of demosaicking.
- A new CFA pattern requires only 10 add operations per *full* pixel reconstruction, with unmatched image quality. This is at least 10x speedup in computational complexity from any comparable demosaicking methods.
- Most imaging experts are ignorant of the fact that demosaicking influences white balance and white balance influences demosaicking, and why this is the case.
In conclusion, demosaicking is not dead. Demosaicking is not high on the priority list of the manufacturers (nor of the ISSL). But demosaicking is the key to enabling new capabilities in cameras that manufacturers care about. By parsing through our publication list, you will realize that research at ISSL thrives on our *understanding* of demosaicking, even when the development of new demosaicking method is not part of the ISSL’s goal.