Bayer Drizzle in DeepSkyStacker

2011, June 8

In May 2011, Hasselblad announced a new digital camera, H4D-200MS. Its 50 MP sensor can generate high resolution images in which each pixel has real R/G/B information, rather than interpolated R/G/B values as in most other digital cameras. This is achieved by taking four exposures and shift the sensor by 1 pixel between each. This allows to fill the holes in the filter array on the sensor and provide real R/G/B values to all the pixels in an image combined from the four shifted exposures.

It turns out, this idea is not new at all. The same technique has been used by high-end medium format digital backs, and even by astronomers. Astronomers invented a technique called "drizzle" to fill the holes between the pixels to create high-resolution images. This technique was originally developed for the Hubble Space Telescope and was very soon adopted by amateur astrophotographers. Furthermore, nowadays, amateur astrophotographers often use DSLRs to capture images. DSLR images suffer from degrade of resolution because of the GRGB Bayer filter array---each pixel only has true information of one of the R/G/B colors. The information of the remaining two colors is interpolated based on information collected by adjacent pixels. This problem is overcome by a technique called "Bayer drizzle," which is exactly what Hasselblad does on H4D-200MS.

Bayer drizzle is implemented by a free software called DeepSkyStacker (hereafter DSS). As far as I can tell, to date, DSS is the only software that provides this function. According to DSS's webpage, this function is suggested by Dave Coffin, the author of the very famous DCRaw program. Thanks to DCRaw and DSS, astrophotographers have been enjoying the very nice Bayer drizzle function for quite a long while.

I encourage the readers to look at DSS's webpage to see how this technique works. In short, we take many exposures of the same celestial objects, and all the exposures were offset from each other by several pixels. (The procedure of taking offset pictures is called "dither" by astronomers.) Since all the exposures have slightly different positions on the sky, a pixel in the final stacked image could receive real R information in some of the exposures, real G information in some other exposures, and so on. DSS can figure this out and only uses the real information to reconstruct a color image, without any interpolation. This is the idea of Bayer drizzle.

Below I will show the power of Bayer drizzle. In DSS, two other methods are provided for reconstructing color images from a Bayer array: bilinear interpolation, and Adaptive Homogeneity-Directed (AHD) interpolation. Here I compare the results based on these three methods.

The images were taken with a Takahashi TOA-150 refractor (D=150mm) with a 67 field flattener (f=1100mm). The camera is Canon EOS 5D2 (pixel size = 6.4 um). I took 24 5-minute exposures. All the exposures were dark subtracted and flat-fielded (both in DSS) before being stacked.

Here are 2x enlargements of a small region in the images. Move your mouse over the images to compare them.

mouse off: Bayer drizzle mouse on: bilinear interpolation	mouse off: Bayer drizzle mouse on: AHD interpolation	mouse off: bilinear interpolation mouse on: AHD interpolation

It can be seen that, in terms of image sharpness, Bayer drizzle is the best, bilinear interpolation is the worst, and AHD interpolation is in between. More quantitatively, the histogram below shows the measured stellar sizes in the three images.

Many thousands of stars were automatically detected and analyzed from the three images by this software. The distributions of the sizes (FWHM) of the stars are shown above. The solid histogram is for the image made with Bayer drizzle, the dashed histogram for AHD interpolation, and the dotted histogram for bilinear interpolation. In the worst cases, bilinear interpolation, most of the stars have sizes of roughly 2.95 pixels, corresponding to 3.4 arcsec. The stars in the Bayer drizzled image have sizes of 2.4 pixels, corresponding to 2.8 arcsec.

The improvement in image resolution brought by Bayer drizzle is approximately 20% in this case, which is not small. How much Bayer drizzle can improve the resolution depends on how well the pixels sample the stars. If the pixels are very fine and the stars are very coarse (caused by either bad optics or bad seeing), then Bayer drizzle does not help. (In such cases, we say the pixels "over-sample" the stars.) On the other hand, if the pixels are coarse and the stars are sharp on the focal plane (requiring good optics and good seeing), then Bayer drizzle will help. (In such cases, we say the pixels "under-sample" the stars.)

However, one should note that Bayer drizzle has its own draw back. The images constructed with Bayer drizzle will always appear noisier than interpolated images. This is the nature of interpolation---it smears out the images, so resolution is lost and noise is wiped out. However, one should be reminded that the Bayer drizzled image just "appears" to be noisier. The true amounts of noise in the Bayer drizzled image and the interpolated images are indeed the same. One clear evidence of this is the nearly (if not entirely) identical limiting magnitudes in these images. These images are all equally deep! One can always apply some noise reduction to the Bayer drizzled image and carefully preserve the sharpness. A Photoshop trick I use is to adopt the L layer of the Bayer drizzled image and the ab layers from an interpolated image to create a new Lab image. This gives the sharpness of the Bayer drizzled image and much less color noise. (Then I apply more noise reduction if I feel there is a need.) The images below compare the detailed look and the global feeling of the noise after such ab layer swapping.

mouse off: original Bayer drizzled image mouse on: L from Bayer drizzled image and ab from bilinearly interpolated image

mouse off: original Bayer drizzled image mouse on: L from Bayer drizzled image and ab from bilinearly interpolated image (image size reduced by 2x)

Another draw back of Bayer drizzle is the need of many exposures. In principle, if one can accurately control the amount of dither offsets of the images, then only four images are required to fill the holes in the Bayer GRGB filter array, just like the case for Hasselblad H4D-200MS. In reality, probably only the Hubble Space Telescope can do this. Since we cannot control the offsets too well, we can only take many exposures and hope their more or less random offsets can eventually fill the holes. My experience shows that it is possible to create a Bayer drizzled image with just four dithered exposures, but the result won't be too good. I think a minimally acceptable Bayer drizzle requires 10 exposures, and a good one requires more than 20 exposures. To collect more than 20 exposures on the same celestial object will either take lot of time (usually fine for me, since I love long integrations), or will require a camera with very low read noise (so exposures can be kept short while the read noise is still lower than sky noise). Of course, it also requires an imaging system that can dither around between the exposures. So there is a price to pay.

In my opinion, if one cannot afford the above price of Bayer drizzle, AHD interpolation is a good compromise. It provides images with reasonably good sharpness, and the "apparent" noise in AHD interpolated images is low. But once again, everything has its price. If you want to use the image for some photometric analyses, you have to avoid AHD interpolation. Recently I created a homework project for high-school kids to produce HR diagrams of open clusters. When I did the homework by myself, I found that the photometry in AHD interpolated images is not accurate. On the other hand, photometry in Bayer drizzled and bilinearly interpolated images seems to be fine and consistent with each other. So, be careful. AHD interpolation is only for pretty pictures, not for scientific analyses.

Finally, if you are doing astrophotography, I strongly suggest to give DSS a try. It's free, and it's a fine software.