The PixInsight Debayer / Demosaicing Process Explained
Last week was the 10th anniversary of the death of Bryce Bayer, the inventor of the Bayer Matrix.
For many of us who use One Shot Color CCD, CMOS or DSLR cameras to image, we have to deal with a process called Debayering. This is a very important process that needs to be included in your processing workflow for the sub exposures you’ve taken using these kinds of cameras. If you don’t include this process with one shot color data, you will not get the color image you’re expecting to see. It was Bryce’s work that makes it possible for us to process image data from these types of cameras.
In the mid 1970s, Mr Bayer, then a Kodak scientist developed what is known as the Bayer Matrix and patented it in 1976. The Bayer Matrix is basically a set of photosensors or pixels as we call them now whose main characteristic is that these matrix combinations of red, green and blue color filters placed over pixels would see light just like the human eye does. The most common matrix pattern for this is the Red-Green-Green-Blue pattern or RGGB pattern. Since the human retina in the eye is more sensitive to green light, this matrix became the standard matrix for color photosensors, although there are other possible combinations that can be used for different reasons.
The raw output of these cameras is formatted so that each pixel is filtered to record only one color. The problem is that the data from each color filtered pixel on its own, can’t specify the values of that color. What you need to do is use an algorithm that will interpolate a set of the surrounding similar red, green or blue pixel values to estimate what the value of a particular color filtered pixel (red, green or blue) should be and outputs this into raw data from the sensor data. This is what happens for those of us who use PixInsight to process our raw data; in particular, we execute the PixInsight Debayer process. PixInsight has 3 algorithm choices available in this process to use which are also called demosaicing methods, although I should note there are other demosaicing methods that could be used in other image processing programs.
The information below is taken directly from the PixInsight documentation of the Debayer process describing the 3 demosaicing methods that can be used. You should have some understanding of these methods to use the best method for your particular situation when using PixInsight to process one shot color data.
VNG, or The Variable Number of Gradients Method
Superpixel Method
Bilinear Method
The key to using the Debayer process is that you should execute this process after you’ve calibrated and cosmetically corrected your images. The reason is that the Image calibration process (calibrating the light sub exposures with bias, darks, flats and flat dark calibration frames) is a pixel by pixel process. Each pixel is compared to and manipulated with each of the calibration frames. If we used the Debayer process before we did any calibration, the interpolation done by the Debayering algorithm would ruin our ability to do the image calibration process.
After debayering your sub exposures, you continue processing as normal to register / align your data and then use image integration to stack them. After that, it’s on to getting that first look at what you’ve captured and then on to all the fun you love and enjoy with post processing.
Stay safe and Clear skies….
Astro Image Credit:
IC 5332 - Telescope Live 1-Click Observation Data, Processed using PixInsight by Reggie Jones - https://app.telescope.live/click-grab/all?target=ic%205332