DSLR vs. Cooled Monochromatic CCD

The following is the test setup.

Telescope: Takahashi TOA-150 (D=150mm, f=1100mm) with 67 field flattener
Mount: Takahashi EM-400
Camera 1: Apogee F16M (KAF-16803 CCD, cooled to -20 deg C) with Astrodon LRGB filter
Camera 2: Canon EOS 5D II (IR-blocking filter removed, no active cooling, ambient temperature around 10 deg C)
Exposure: All single exposures are 5-minute long. The 5D2 exposures were made at ISO 800 and ISO 1600.

The test images are cropped subsets of the exposures from the two pictures:
http://www3.asiaa.sinica.edu.tw/~whwang/gallery/picutres/M42_2014.htm
http://www3.asiaa.sinica.edu.tw/~whwang/gallery/picutres/M42_2012.html

The DSLR and CCD images were taken under very different seeing conditions. So any differences related to seeing (such as image sharpness and limiting magnitude) should be ignored. Other than seeing, the sky transparency is similar, perhaps slightly better for the CCD image. The sky for the DSLR image may be slightly darker, but not by a larger margin. All the images are linearly stretched to have similar color, brightness, and contrast. There is no saturation adjustment. The CCD images were first registered to the DSLR image. Both are then down-sampled by two times (bilinear interpolation), to slightly hide the problem of poor seeing in the CCD image. Therefore, the effective pixel size is 12.8 um, two times larger than 5D2's pixel size, and roughly 40% larger than 16803's pixel size. The images shown on this page were all up-sampled by three times (nearest neighbor) for better viewing.

Test A: DSLR vs. CCD LRGB, same exposure times

This is to see how the two compare, if both are given the same amounts of time. The total exposure time is 145 minutes for each. For the CCD, L:R:G:B = 85:20:20:20 (roughly 4:1:1:1). Many CCD people like to bin the pixels when they take RGB exposures. This has the advantage of reducing readout noise by sacrificing the RGB resolution a little. Here I did not bin the RGB pixels, because each exposure is long enough for the sky photon noise to overwhelm the readout noise. On the other hand, for both the DSLR and CCD images, I created a separated set of images on which I applied the same amount of noise reduction (NR) in Photoshop to the color space (not the luminance space). Since NR is essentially to average out pixel values and to decrease the image resolution, this has a similar effect of binning the RGB pixels in the LRGB mode. Below are the results. Move your mouse on and off the images to compare, and pay attention to the noise in the images.

DSLR vs. LRGB CCD, same exposure times (145 minutes)
Without NR		With NR
mouse on: 16803 mouse off: 5D2

Again, one should not look at the stars, as both the star sharpness and limiting magnitudes are strongly affected by the seeing. Here, the key is the strength of the noise in the image background, relative to the background and the nebulas.

It is obvious that the CCD LRGB image has a smoother background. In the non-NR case, because of the shorter RGB exposures, the CCD image appears to have stronger color noise. However, this is easily reduced by some simple NR in the RGB channels, somewhat similar to the effect of binning pixels during the CCD readout. The DSLR image does not reach the same smoothness after the NR.

The conclusion is clear. Given the same total exposure time, the mono-CCD LRGB imaging provides a better result, thanks to the large photon throughput of the L filter.

Test B: DSLR vs. CCD LRGB, uneven exposure times

To see how much better the LRGB mode is, I cut down the total exposure time of the CCD images to 70 minutes, approximately half the DSLR exposure time. The ratio of L:R:G:B is 40:10:10:10, again 4:1:1:1. Below are the results.

DSLR (145 minutes) vs. LRGB CCD (70 minutes)
Without NR		With NR
mouse on: 16803 mouse off: 5D2

Here, in both the NR and non-NR cases, I feel that the color noise is stronger in the CCD LRGB images, and the luminance noise is stronger in the DSLR images. Overall, I would say the CCD LRGB image is still better, but the difference is not as dramatic as that in the above equal-exposure case.

These results show that DSLR needs at least two times more total exposure (perhaps up to three times) to catch up the image quality of CCD LRGB imaging. This is the fundamental limit of the Bayer filter array on DSLRs.

Test C: DSLR vs. CCD RGB, same exposure times

Now it comes the most interesting one. I am curious how well a cooled mono-CCD performs against an uncooled DSLR if it only does RGB imaging, not LRGB. In both cases, the total exposure times are 80 minutes. For the CCD, R:G:B = 25:30:25, close to 1:1:1. In the DSLR case, at any given time, only 50% of the pixels are collecting G photons, and only 25% (each) of the pixels are collecting R and B photons. So the effective RGB ratio for the DSLR is R:G:B = 1:2:1, different from the CCD case. This mismatch is a bit unfortunate. Bearing this drawback in mind, below is the result.

DSLR vs. RGB CCD, same exposure times (80 minutes)
Without NR		With NR
mouse on: 16803 mouse off: 5D2

Before applying NR, it is really hard to say which one is better. After applying NR to smooth out small-scale color noise, the color noise in the CCD image dramatically decreases, but not quite so in the DSLR image. After some quick blinking between the two NR images with mouse on and off, I feel the CCD RGB image is a bit better. This can be easily attribute the the difference in the throughputs of the color filters and the difference in the quantum efficiencies of the sensors.

One thing to note here is that the DSLR images were taken with an old DSLR. Later DSLR models, such as Canon 5D3, 6D or Nikon D800, D610, all have substantially lower thermal noise, lower readout noise, and higher quantum efficiencies, comparing to 5D2. If 5D2 is replaced with a newer DSLR here, I believe that the DSLR image will be better than the CCD RGB one.

What this tells us is that, once the advantage of the high-throughput L filter is removed from the equation, a cooled mono-CCD is not better than a previous-generation DSLR. Latest DSLRs can easily outperform a cooled CCD that does not have an L filter. This confirms what I have been telling people for a long time: CCD astronomical images are better than DSLR ones because of the L filter, not because of the sensor (and cooling). L filter is really the key. Overall, recent DSLRs all have high quantum efficiencies (around 50% in G, before the IR-blocking filter is removed), low readout noises (about 2 to 3 electrons, at ISOs higher than 800, some even at ISO 400), and low thermal noises. Their thermal noise is not as low as CCDs (when cooled), but is already lower than sky noise in many cases and therefore does not affect the final image quality (assuming good dark subtraction).

Of course, cooled monochromatic CCDs (when equipped with an L filter) still outperforms DSLRs. This is clearly shown by the first test above. I just think that we should attribute this to the right reason. The reason is the L filter, not the sensor. Once we realize this, we should immediately stop advertising the use of cooled color CCDs. As shown in the above Test C, without an L filter, an RGB CCD is not much better than a modern DSLR, but is much more expansive for the same amount of pixels (or sensor area, however you view it).

Test D: CCD RGB vs. CCD LRGB, similar exposure times

This is just to confirm that LRGB is indeed a more efficient imaging technique.

LRGB CCD (70 minutes) vs. RGB CCD (80 minutes)
Without NR		With NR
mouse on: 16803 LRGB mouse off: 16803 RGB

The LRGB one has a slightly shorter total exposure time, and yet its image quality is remarkably better.

Effect of Noise Reduction

One thing that puzzles me is the effect of NR. To be more specific, the NR I made in Photoshop was a series of smart blur and fading, in the RGB channels all together. A luminance channel was created from a desaturated GRB image before the NR, and this channel is preserved. This is equivalent to just conducting NR in the ab channels in an Lab color mode, or just conducting NR in the RGB channels in the LRGB mode. All images here went through identical NR, so they are all equally affected. However, in all cases, the color noise in the CCD LRGB or RGB images dramatically decreases, but not quite so in the DSLR images.

The NR can improve the CCD images quite much, but not the DSLR images. The direct reason behind this must be that the color noise in the CCD images have small characteristic spatial scales (just a couple of pixels) so they can be smoothed out. On the other hand, the spatial scale of the color noise (or at least some components in the color noise) in the DSLR images must be larger, so a small radius blur does not remove the noise. I suspect that a more careful tuning of the NR parameters can smooth out the large scale color noise in the DSLR images, bringing their quality a step closer to that of the CCD images. Here I just tried to keep everything simple and equal, so I only apply identical NRs to the CCD and DSLR images.

Why do the DSLR images have large-scale color noise? I don't have a good answer. An naive answer would be the Bayer array, but I am not convinced by this. The DSLR images here were stacked with Bayer drizzle in DeepSkyStacker, which does not interpolate the Bayer array at all. So the issue must lie in a deeper layer in the image processing or in the camera.

Final Remarks

OK, I know people are ready to complain about my tests. The above comparisons are not the best we can do. There are still many different combinations of exposure times and image processing parameters I haven't tried. We will also need more quantitative ways to judge the image quality, although I doubt we can find any. The very different seeing conditions are the most disappointing part of this test, as this prevents us from judging the image quality using limiting magnitudes. Ideally, we even want to conduct the tests using two identical telescopes, one shooting with a CCD and the other with a DSLR, at the same time. I will try to improve all these when there is some opportunity.

No matter you agree with the procedure or conclusion or not, I hope the tests here can shed some light on the issue of imaging efficiency and performances of CCD and DSLRs.