ABSTRACT Digital still cameras use a single charge-coupled device (CCD) sensor array and a color filter array (CFA) to sample a full-color image. Thus, the measured image is an interleaving of the subsampled red, green, and blue images. The red and blue images are sampled at a lower rate, so if standard interpolation techniques are used, the reconstructed red and blue images will be missing some highfrequency information and could contain distortions from aliasing. This paper proposes a method of CFA interpolation that combines information from the green image with the subsampled red and blue images to attack these problems. The green high-frequency information is added to the interpolated red and blue images to increase the sharpness of the output and is also used to estimate the aliasing in the interpolated red and blue images, providing a means of reducing the appearance of the aliasing distortions

John W Glotzbach

Ronald W Schafer

CiteSeerX

A METHOD OF COLOR FILTER ARRAY INTERPOLATION WITH ALIAS 
CANCELLATION PROPERTIES 
John W. Glotzbach and Ronald W. Schafer * Klaus Illgner t 
Center for Signal and Image Processing 
Georgia Institute of Technology 
Atlanta, GA 30332-0250 
glotz,rws@ece.gatech.edu 
ABSTRACT 
Digital still cameras use a single charge-coupled device 
(CCD) sensor array and a color filter array (CFA) to sam- 
ple a full-color image. Thus, the measured image is an 
interleaving of the subsampled red, green, and blue images. 
The red and blue images are sampled at a lower rate, so 
if standard interpolation techniques are used, the recon- 
structed red and blue images will be missing some high- 
frequency information and could contain distortions from 
aliasing. This paper proposes a method of CFA interpola- 
tion that combines information from the green image with 
the subsampled red and blue images to attack these prob- 
lems. The green high-frequency information is added to the 
interpolated red and blue images to increase the sharpness 
of the output and is also used to estimate the aliasing in 
the interpolated red and blue images, providing a means of 
reducing the appearance of the aliasing distortions. 
1. INTRODUCTION 
A digital color image requires measurement of light energy 
across the wavelength spectrum at each pixel location. Typ- 
ically, three measurements are made at each pixel location, 
denoted red (R), green (G), and blue (B). Low-cost digital 
still cameras use a single charge-coupled device (CCD) sen- 
sor array to measure a color image. Because the CCD is not 
spectrally selective, a color filter is manufactured onto each 
sensor of the CCD sensor array, allowing only one part of the 
wavelength spectrum to be measured by each CCD sensor 
cell. The arrangement of these color filters is known as the 
color filter array (CFA); the most commonly used CFA is 
the Bayer CFA, whose arrangement is shown in Fig. 1. The 
output of the CCD consists of three downsampled signals 
representing the red, green, and blue images. To generate 
a full-color image, these downsampled signals need to be 
interpolated to the resolution of the sensor array. 
Using standard interpolation techniques, a filter is de- 
signed to pass the low-frequency information and reject the 
high-frequency spectral images. In the context of this prob- 
lem, a rectangular lowpass filter is used to interpolate the 
red and blue images and a diamond-shaped lowpass filter 
*This work was supported in part by Texas Instruments. 
tThis work was conducted while the author was with Texas 
Instruments. 
Siemens AG 
IC CT 2 
81730 Munich, Germany 
klaus.illgner@mchp .siemens. de 
G R G R  
B G B G  
G R G R  
B G B G  
O R O R  G O G O  0 0 0 0  
- 0 0 0 0  O G O G + B O B O  
- O R O R + G O G O  0 0 0 0  
0 0 0 0  O G O G  B O B 0  
Fig. 1. Arrangement of color filters in Bayer CFA. 
is used to interpolate the green image. These filters are 
specified in Fig. 2. For the CFA interpolation problem, this 
approach has two drawbacks: 
1. The red and blue images could be missing high-fre- 
quency components that are present in the green im- 
age. 
2. The red and blue images are more prone to alias than 
the green image, causing color moire patterns in the 
reconstructed image. 
A broad range of approaches has been proposed in the 
literature. Adams proposed a method of directional inter- 
polation, where a preferred interpolation direction is chosen 
for each pixel, based on image content [l]. Other methods 
using the directional approach can be found in [2], [3] and 
[4]. These methods provide good results; however, the al- 
gorithms are more complex because they require analysis of 
the image at each pixel. Neural networks have been used for 
CFA interpolation in [ 5 ] .  CFA interpolation with B-splines 
has been analyzed in [6]. Taubman proposed a method 
of generating a linear, minimized mean square estimator 
(LMMSE) filter to provide the optimal (in the mean square 
sense) reconstructed image, reporting good reconstruction 
results [7]. 
In this paper, a method based on standard signal pro- 
cessing theory is discussed that allows the reconstruction 
of high-frequency content in the red and blue images. This 
method adds high-frequency information from the green im- 
age to the red and blue images and also uses the green 
high-frequency information to estimate the aliasing in the 
red and blue images, thus providing a means of reducing 
0-7803-6725-1/01/$10.00 02001 IEEE 141 
I -  n 
1 
I I L  
(a) Ideal red/blue interpolation filter. 
v .- 
(b) Ideal green interpolation filter. 
Fig. 2. Ideal interpolation filters for Bayer CFA interpola- 
tion. 
the amount of aliasing in the reconstructed image. Section 2 
outlines the theory for this method. Section 3 derives an ef- 
ficient implementation of the algorithm. Section 4 provides 
results showing the advantages of the proposed method and 
Section 5 discusses future research. 
2. THEORY OF PROPOSED METHOD 
Referring to Fig. 1 and accounting for the phase differences 
in the sampling grids, the three sampled image signals are 
represented by their Z-Transforms in ( l ) ,  (2), and (3). The 
X ( z 1 ,  z2) terms contain the desired low frequency infor- 
mation and the X(-zl,zz), X(z1, - z z ) ,  and X(-zl, - 2 2 )  
terms are the sampiing aliases, which could distort the out- 
put if the input signal contains high frequencies. 
With the Bayer CFA, the red and blue images are sam- 
pled at a lower rate than the green image, but by using 
information from the green image, the red and blue images 
can be reconstructed with the same frequency content as 
the green image. In a typical image, the three images are 
very well correlated. A simplified color image model has 
been presented as (4)[1], where the color channels are sepa- 
rated only by an additive constant; thus, the high-frequency 
information in the separate images is identical. Therefore, 
a method to reconstruct the red and blue images is to use 
low-frequency information from the red and blue images 
and high-frequency information from the green image. The 
green image is interpolated using the diamond-shaped, low- 
pass filter, H d s .  
R=G+kR B = G + k B  (4) 
A horizontal highpass filter, Hh,  and a vertical high- 
pass filter, H,,, are designed to pass the horizontal high 
frequencies and vertical high frequencies, respectively, from 
the green image. The output of these filters can be added 
to the output of the red and blue interpolation filters, in- 
creasing the sharpness in the reconstructed red and blue 
images. The ideal response of these filters is shown in Fig. 
3. 
The aliasing in the red and blue images is due to com- 
ponents of the input image outside the rectangular Nyquist 
region. The sampling process replicates the input high fre- 
quencies in the low frequency region of, the spectrum. By 
modulating the output of the horizontal highpass filter back 
to horizontal low frequencies, the horizontal aliasing in the 
red/hlue images caused by the sampling process can he es- 
timated. For the red image, this modulated signal needs to 
he added to the output of the red interpolation filter but 
subtracted from the output of the blue interpolation filter 
since R(- z l ,  a)  has a negative sign in (1) and B ( - z l ,  z2) is 
positive in (3). An estimate of the aliasing caused by high 
vertical frequencies can he produced by modulating the out- 
put of the vertical highpass filter to  low vertical frequencies. 
Here, the modulated signal is added to the blue output but 
subtracted from the red output. Using this process, the re- 
constructed red image and the reconstructed blue image can 
be expressed as ( 5 )  and ( 6 ) ,  respectively. A block diagram 
outlining this approach is presented in Fig. 4. 
B ( Z l , Z 2 )  = 
B(z1,zz) = 
3. EFFICIENT IMPLEMENTATION 
An efficient implementation for the block diagrams in Fig. 4 
can he derived easily. From Fig. 4, the filtered red signal is 
I42 
1 
4: 
(a) Ideal horizontal highpass filter. 
(b) Ideal vertical highpass filter. 
Fig. 3. Ideal green highpass filters. 
combined with horizontal information from the green image 
and vertical information from the green image. Because Gh 
is modulated and added to itself, the signal containing the 
horizontal information is non-zero only on the even columns 
(left-most column indexed as zero). Similarly, G, is mod- 
ulated and subtracted from itself. Thus, this signal is non- 
zero only on the odd rows (top-most row indexed as zero). 
The sum of these two signals will be used as a correction 
signal for the interpolated red image. Thus, referring to 
Fig. 1, the correction signal at  a G ,  pixel (even row, even 
column) will be 2 .  Gh. The correction signal for an R pixel 
(even row, odd column) is zero. The correction signal for 
a B pixel (odd row, even column) will be 2 . Gh + 2 G,. 
Finally, the correction signal for a Gb pixel (odd row, odd 
column) will be 2. G,. Correction values at  G, (green pixels 
in a row containing red pixels), R, B, and Gb (green pixels 
in a row containing blue pixels) pixels will be computed by 
filtering the green samples with the coefficient sets 2"h, 0, 
2.(Hh +H,),  and 2 .  H,,, respectively. Thus, this algorithm 
can be implemented with a periodic, spatially-varying filter, 
where the coefficients for the filter change for the different 
types of pixels 
(a) Block diagram for interpolating the red im- 
age. 
I 
(b) Block diagram for interpolating the blue im- 
age. 
Fig. 4. Block diagram for proposed CFA interpolation 
method. 
4. RESULTS 
In order to  test the proposed algorithm, 15x15 filters were 
designed to approximate the ideal filters: H,., Hb, H d s ,  Hh, 
and H,. A sample image, Lighthouse (Kodak PhotoCD 
Color Image Database), was resampled according to the 
Bayer CFA and reconstructed using the designed filters. In 
this image, the fence provides an increasingly high spatial 
frequency that causes aliasing in the undersampled red im- 
age. The results from a small region of the image are pre- 
sented in Fig. 5. To illuminate the effects of the algorithm 
without full-color printing, only the red image is presented. 
Fig. 5(a) is the original image and Fig. 5(b) is the result 
using only a filter designed to approximate the ideal filter, 
H,, to interpolate the image. Fig. 5(c) shows the effect of 
using the method from Fig. 4(a). This result shows that 
not only has the information from the green image been 
able to restore the sharpness in the red image but it can 
also effectively reduce the appearance of the aliasing. 
143 
(a) Original image. (b) .Using an ap- 
proximation of the 
ideal interpolation lation method. 
filter, H,. 
(c) Using the pro- 
posed alias cancel- 
Fig. 5. Interpolation results on example image. 
5.  CONCLUSIONS 6. R E F E R E N C E S  
This article outlined a new technique for performing CFA 
interpolation. In the algorithm, information from the green 
image was used to add high-frequency information to the 
lower-sampled red and blue images. The green high-fre- 
quency information was also used to estimate the aliasing 
present in the insufficiently sampled red and blue images. 
With an example image, the method was able to signifi- 
cantly improve the interpolation result over standard inter- 
polation techniques. 
One other advantage of this algorithm is that its im- 
plementation involves only the use of linear filters. Thus, 
it can be programmed efficiently on any DSP architecture, 
making it a suitable algorithm for today’s cameras. 
More research needs to be performed to determine the 
best method to combine the green and red/blue informa- 
tion. Currently, the green information is added without 
scaling to the red and blue images. However, in some cases, 
it may be necessary to attenuate or amplify the green in- 
formation before adding it to the red and blue images. 
Also, more work needs to be done to define methods 
of designing the filters for this problem. Metrics of perfor- 
mance need to be defined so that optimization techniques 
can be defined. One such metric could be the amount of 
alias rejection the filters provide. The design technique 
should probably consider the filters as a set and provide 
the best set of filters rather than individually designing each 
filter to meet some criterion. 
[l] J. E. Adams, “Design of practical color filter array in- 
in Proc. 
[2] T. Kuno, H. Sugiura, and N. Matoba, “New interpo- 
lation method using discriminated color correlation for 
digital still cameras,” IEEE Trans. Cons. Elec., vol. 45, 
no. 1, pp. 259-267, Feb. 1999. 
[3] R. Kimmel, “Demosaicing: Image reconstruction from 
color CCD samples,” IEEE Trans. Image Proc., vol. 8, 
no. 9, pp. 1221-1229, Sept. 1999. 
[4] E. Chang, C. Shiufun, and D. Pan, “Color filter ar- 
ray recovery using a threshold-based variable number 
of gradients,” in Proc. SPIE, 1999, vol. 3650, pp. 36- 
43. 
terpolation algorithms for digital cameras,” 
SPIE, 1997, vol. 3028, pp. 117-125. 
(51 J. Go and C. Lee, “Interpolation using neural network 
for digital still cameras,” in 2000 Digest of Tech. Papers. 
Int. Conj. Cons. Elec., 2000, pp. 176-177. 
[6] B. Lee, J. Kim, and C. Lee, “High quality image in- 
terpolation for color filter arrays,” in Proc. Int. Conj. 
Systems, Man, and Cybernetics, 2000, vol. 2, pp. 1547- 
1550. 
“Generalized Wiener reconstruction of 
images from colour sensor data using a scale invariant 
prior,” in Proc. Int. Conj. Image Proc., 2000, pp. 801- 
804. 
[7] D. Taubman, 
144 


A method of color filter array interpolation with alias cancellation properties

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=1A9EC1CAF413CB1BABFC97A8BCEA9219?doi=10.1.1.1042.9781&rep=rep1&type=pdf

A method of color filter array interpolation with alias cancellation properties

Abstract

Similar works

Full text

Available Versions

CiteSeerX