High Accuracy Bicubic Interpolation Using Image Local Features

袁 帥, 阿部 正英, 田口 亮*, 川又 政征

Shuai YUAN, Masahide ABE, Akira TAGUCHI* and Masayuki KAWAMATA

東北大学大学院工学研究科電子工学専攻武蔵工業大学電気電子情報工学科*

Department of Electronic Engineering, Tohoku University

Department of Electrical and Electronic Engineering, Musashi Institute of Technology*

E-mail: yuan@mk.ecei.tohoku.ac.jp

Abstract

In this paper, we propose a novel bicubic methodfor digital image interpolation. Since the conven-tional bicubic method does not consider image lo-cal features, the interpolated images obtained by theconventional bicubic method often have a blurringproblem. In this paper, the proposed bicubic methodadopts both the local asymmetry features and the lo-cal gradient features of an image in the interpolationprocessing. Experimental results show that the pro-posed method can obtain high accuracy interpolatedimages.

1. Introduction

Image interpolation is a prime technique in imageprocessing. It is used in many important applicationssuch as digital high-definition television, big screendisplay, copy and print machine, medical imaging,end-user equipment and so on.

The bicubic interpolation [1], which is linear andeasy to be computed parallel, is used widely in manyinterpolation applications. However, the conven-tional bicubic interpolation has a blurring problemin the interpolated images, because it ignores thefeatures of the image pixel data, such as the fre-quency features, the edge features, the features un-der multi-resolution and so on. For solving the blur-ring problem in the interpolated images, resolutionenhancement (RE) interpolation methods were pro-posed. Some of the RE interpolation algorithms usemulti-resolution pyramids [2] and frequency features[3] of an image to calculate the interpolated image.Another popular type of the RE interpolation al-gorithms [4], [5] needs a sequence of low-resolutionimages for producting an interpolated image. Sincemost of the interpolations based on multi-resolutionpyramids or multi-image datas are not linear and dif-ficult to be computed parallel, they can not replace

the bicubic interpolation in many applications.For improving the interpolation accuracy of the

bicubic interpolation, some methods [6], [7], whichuse image local features, are proposed. In [6], thelocal asymmetry features of image pixels are used tomodify the local up-sampling distance in the bicubicinterpolation. In [7], the local gradient features of im-age edges are used to optimize the local interplationweights in the bicubic interpolation. Since both theasymmetry feature and the gradient feature can helpto reduce the blurring problem of the bicubic inter-polation, in this paper we combine these two kindsof image features to propose a novel high accuacybicubic algorithm.

This paper is organized as follows. The conven-tional bicubic interpolation is reviewed in Section 2.The bicubic interpolation using the local asymme-try feature is described in Section 3.1. The bicubicinterpolation using the local gradient feature is de-scribed in Section 3.2. Then, in Section 3.3 we givethe proposed algorithm, which adopts both the lo-cal asymmetry and the local gradient features. Theexperimental results and the comparison are shownin Section 4. Finally, the conclusion is included inSection 5.

2. Conventional Bicubic Interpolation

The conventional bicubic interpolation needs anup-sampling distance S to estimate the unknown pix-els for the interpolation processing. At the position(i′, j′), which is shown in Fig. 1, the bicubic interpo-lation calculates the interpolated pixel as

fi′,j′ =[

W−1(Sy) W0(Sy) W1(Sy) W2(Sy)]

·

⎡⎢⎢⎣

fi−1,j−1 fi,j−1 fi+1,j−1 fi+2,j−1

fi−1,j fi,j fi+1,j fi+2,j

fi−1,j+1 fi,j+1 fi+1,j+1 fi+2,j+1

fi−1,j+2 fi,j+2 fi+1,j+2 fi+2,j+2

⎤⎥⎥⎦

·

⎡⎢⎢⎣

W−1(Sx)W0(Sx)W1(Sx)W2(Sx)

⎤⎥⎥⎦ , (1)

where Sy = j′ − j, Sx = i′ − i and fi,j meansthe pixel value at the position (i, j). The weightsW−1(S),W0(S),W1(S),W2(S) in conventional bicu-bic interpolation are given as

W−1(S) =−S3 + 2S2 − S

2,

W0(S) =3S3 − 5S2 + 2

2,

W1(S) =−3S3 + 4S2 + S

2,

W2(S) =S3 − S2

2.

3. Proposed High Accuacy Bicubic

Method

Since the conventional bicubic interpolation has ablurring problem, one modified method was proposedin [6] using the image local asymmetry features, andanother modified method was proposed in [7] usingthe image local gradient features. In this section,we review these two modified bicubic interpolationsfirstly, and then give the proposed bicubic interpola-tion, which combines these two kinds of image fea-tures.

3.1 Local Asymmetry Feature Based

Bicubic MethodThe local asymmetry feature based bicubic method

uses the modified interpolation value fAi′,j′ to replace

the conventional interpolation value fi′,j′ . In [6] themodified value fA

i′,j′ is calculated using Eq. (1) byreplacing Sy, Sx with S′

y, S′x, which is given as

fAi′,j′ =

[W−1(S′

y) W0(S′y) W1(S′

y) W2(S′y)

]

·

⎡⎢⎢⎣

fi−1,j−1 fi,j−1 fi+1,j−1 fi+2,j−1

fi−1,j fi,j fi+1,j fi+2,j

fi−1,j+1 fi,j+1 fi+1,j+1 fi+2,j+1

fi−1,j+2 fi,j+2 fi+1,j+2 fi+2,j+2

⎤⎥⎥⎦

·

⎡⎢⎢⎣

W−1(S′x)

W0(S′x)

W1(S′x)

W2(S′x)

⎤⎥⎥⎦ , (2)

where

S′y =

S′y1 + (S′

y2 − S′y1)S

′x1

1 − (S′y2 − S′

y1)(S′x2 − S′

x1), (3)

S′x =

S′x1 + (S′

x2 − S′x1)S

′y1

1 − (S′y2 − S′

y1)(S′x2 − S′

x1). (4)

図 1: Interpolation position (i′, j′) and up-sampling distances Sx, Sy.

In equations (3) and (4) S′x1, S′

x2, S′y1, S′

y2 are cal-culated by

S′x1 = Sx − kAx1Sx(1 − Sx),

S′x2 = Sx − kAx2Sx(1 − Sx),

S′y1 = Sy − kAy1Sy(1 − Sy),

S′y2 = Sy − kAy2Sy(1 − Sy),

where Ax1, Ax2, Ay1 and Ay2 are the local asymme-try features defined by

Ax1 =|fi+1,j − fi−1,j | − |fi+2,j − fi,j |

L − 1,

Ax2 =|fi+1,j+1 − fi−1,j+1| − |fi+2,j+1 − fi,j+1|

L − 1,

Ay1 =|fi,j+1 − fi,j−1| − |fi,j+2 − fi,j |

L − 1,

Ay2 =|fi+1,j+1 − fi+1,j−1| − |fi+1,j+2 − fi+1,j |

L − 1,

in Ax1, Ax2, Ay1 and Ay2, L = 256 for 8-bit lumi-nance images, and then Ax1, Ax2, Ay1 and Ay2 arelimited in the range [−1, 1]. The positive parameterk decides the warping level. When k is large, S′ maybe outside of the range [0, 1]. Since S′ must be kept in[0, 1] in real image interpolations, the outside-rangeS′ will be clipped to 0 or 1.

3.2 Local Gradient Feature Based

Bicubic Method

In digital images, the local gradient features arealso one kind of the important image features. In [7],using the local gradient features around the interpo-lated pixel fi′,j′ , four local gradient weights are pro-posed by the authors. The four local gradient weightsHl, Hr, Vu and Vl are generated from the four pixelmasks around the interpolated pixel, where the fourmasks are shown in Fig. 2 with the dotted line blocks.

図 2: Gradient masks.The local gradient weights Hl, Hr, Vu and Vl are

defined asHl =

1√1 + α(|fi,j − fi−1,j | + |fi,j+1 − fi−1,j+1|)

Hr =1√

1 + α(|fi+1,j − fi+2,j | + |fi+1,j+1 − fi+2,j+1|)Vu =

1√1 + α(|fi,j − fi,j−1| + |fi+1,j − fi+1,j−1|)

Vl =1√

1 + α(|fi,j+1 − fi,j+2| + |fi+1,j+1 − fi+1,j+2|),

where the parameter α is in the range of [0, 1]. Us-ing the local gradient weights Hl, Hr, Vu and Vl, theauthors got 8 new interpolation weights WG

n to re-place the conventional interpolation weights Wn inthe bicubic interpolation. Then, the local gradientfeature based bicubic method is given as

fGi′,j′ =

[WG

−1(Sy) WG0 (Sy) WG

1 (Sy) WG2 (Sy)

]

·

⎡⎢⎢⎣

fi−1,j−1 fi,j−1 fi+1,j−1 fi+2,j−1

fi−1,j fi,j fi+1,j fi+2,j

fi−1,j+1 fi,j+1 fi+1,j+1 fi+2,j+1

fi−1,j+2 fi,j+2 fi+1,j+2 fi+2,j+2

⎤⎥⎥⎦

·

⎡⎢⎢⎣

WG−1(Sx)

WG0 (Sx)

WG1 (Sx)

WG2 (Sx)

⎤⎥⎥⎦ , (5)

where the weights WGn (Sy) are given as

WG−1(Sy) = W−1(Sy)/DV (Sy),

WG0 (Sy) = VuW0(Sy)/DV (Sy),

WG1 (Sy) = VlW1(Sy)/DV (Sy),

WG2 (Sy) = W2(Sy)/DV (Sy),

and the weights WGn (Sx) are given as

WG−1(Sx) = W−1(Sx)/DH(Sx),

図 3: Test images.

WG0 (Sx) = HlW0(Sx)/DH(Sx),

WG1 (Sx) = HrW1(Sx)/DH(Sx),

WG2 (Sx) = W2(Sx)/DH(Sx),

in which

DV (Sy) =

W−1(Sy) + VuW0(Sy) + VlW1(Sy) + W2(Sy)

andDH(Sx) =

W−1(Sx) + HlW0(Sx) + HrW1(Sx) + W2(Sx),

respectively.

3.3 Proposed High Accuracy Bicubic

Interpolation

After the review of the two modified bicubic algo-rithms, we can see that not only the local asymmetryfeature but also the local gradient feature are helpfulto solve the bicubic blurring problem. Thus, in theproposed method we would like to adopt both of thetwo kinds of image features at the same time. Toadopt both the local asymmetry feature and the lo-cal gradient feature, we give the proposed modifiedinterpolation weights W p

n as

W p−1(Sy) = WG

−1(S′y), W p

0 (Sy) = WG0 (S′

y),

W p1 (Sy) = WG

1 (S′y), W p

2 (Sy) = WG2 (S′

y),

and

W p−1(Sx) = WG

−1(S′x), W p

0 (Sx) = WG0 (S′

x),

W p1 (Sx) = WG

1 (S′x), W p

2 (Sx) = WG2 (S′

x).

in which S′x, S′

y are based on the image local asym-metry features, and WG

n are based the image localgradient features. Then, the proposed bicubic inter-polation can be written as

(a)

(b)

図 4: (a) MSE surfaces of two times interpo-lation using the test images (128×128 to256×256). (b) MSE surfaces of four timesinterpolation using the test images (64×64to 256×256).

図 5: Experimental results of the image “Boat”with 256 × 256. (a) Original “Boat” im-age. (b) Subsampled “Boat” image with128 × 128. (c) Subsampled “Boat” imagewith 64 × 64. (d) Result of 2 times con-ventional bicubic interpolation. (e) Resultof 2 times local asymmetry feature basedbicubic interpolation. (f) Result of 2 timeslocal gradient feature based bicubic inter-polation. (g) Result of 2 times proposedbicubic interpolation. (h) Result of 4 timesconventional bicubic interpolation. (i) Re-sult of 4 times local asymmetry featurebased bicubic interpolation. (j) Result of 4times local gradient feature based bicubicinterpolation. (k) Result of 4 times pro-posed bicubic interpolation.

fpi′,j′ =

[W p

−1(Sy) W p0 (Sy) W p

1 (Sy) W p2 (Sy)

]

·

⎡⎢⎢⎣

fi−1,j−1 fi,j−1 fi+1,j−1 fi+2,j−1

fi−1,j fi,j fi+1,j fi+2,j

fi−1,j+1 fi,j+1 fi+1,j+1 fi+2,j+1

fi−1,j+2 fi,j+2 fi+1,j+2 fi+2,j+2

⎤⎥⎥⎦

·

⎡⎢⎢⎣

W p−1(Sx)

W p0 (Sx)

W p1 (Sx)

W p2 (Sx)

⎤⎥⎥⎦ . (6)

4. Parameter Estimation and Experi-

mental Results

Since both the local asymmetry and the local gra-dient features are used in the proposed method, wehave two parameters in the proposed method. One isthe parameter k which decides S′

x and S′y, the other

is the parameter α which decides the local gradientweights Hl, Hr, Vu and Vl. To do a quantitativeestimation of the two parameters in the proposedmethod, we lowpass several different well-known testimages (256×256) with Gaussian lowpass filters, andthen subsample the lowpassed test images to get thesmall size images. Then, the subsampled images areinterpolated to their original size using the proposedbicubic method. The test images are shown in Fig. 3.Using the proposed bicubic method with different pa-rameter values, the mean square error (MSE) sur-faces of each interpolated test image are shown inFig. 4.

From Fig. 4 we can see that all of the minimizedMSE points are not at the coordinate origin, the Xaxis or the Y axis, that means the interpolation ac-curacy of the proposed bicubic method is better thanthose of the conventional bicubic, the local asym-metry feature based bicubic and the local gradientfeature based bicubic methods. At the coordinateorigin, we have k = 0 and α = 0 that means the cal-culation is the conventional bicubic interpolation. Atthe X axis, we have α = 0 that means the calculationis the local asymmetry feature based bicubic interpo-lation. At the Y axis, we have k = 0 that means thecalculation is the local gradient feature based bicubicinterpolation.

Fig. 4 also shows that for both 2 times and 4 timesinterpolations, the optimum parameters of each testimage are almost in the same coordinate range. Thus,we can use the average values of the optimum k, α

as the fixed experimental parameters kp, αp for theproposed bicubic interpolation algorithm. Then, wehave the experimental parameters kp, αp as

kp = 2.1,

αp = 0.05.

To show the effectiveness of the proposed bicubicinterpolation with the experimental parameters, we

give another experiment using a new test image“Boat”. The experimental results are shown inFig. 5. The proposed results are also compared withthe other bicubic results. In [6] the fixed k for the lo-cal asymmetry feature based bicubic method is givenas 5.5, and in [7] the fixed α for the local gradientfeature based bicubic method is given as 0.07.

From Fig. 5, we can see that the accuracy of theproposed bicubic method with kp, αp is higher thanthose of other bicubic methods.

5. Conclusion

In this paper, we propose a novel bicubic interpo-lation using both the local asymmetry features andthe local gradient features of digital images. Exper-imental results show that the proposed method canobtain high accuracy interpolated images.

参考文献

[1] W. K. Pratt, Digital Image Processing, NewYork: Wiley, 1991.

[2] H. Greenspan, C. H. Anderson and S. Akber,“Image enhancement by nonlinear extrapolationin frequency space,” IEEE Trans. on Image Pro-cessing, vol. 9, no. 6, pp. 1035-1048, June 2000.

[3] W. K. Carey, D. B. Chuang and S. S. Hemami,“Regularity-preserving image interpolation,”IEEE Trans. on Image Processing, vol. 8, no.9, pp. 1293-1297, September 1999.

[4] R. Hardie, K. Barnard. and E. Armstrong,“Joint map registration and high resolution im-age estimation using a sequence of undersam-pled measured images,” IEEE Trans. on Im-age Processing, vol. 6, pp. 1621-1632, December1997.

[5] N. R. Shah and A. Zakhor, “Resolution enhance-ment of color video sequences,” IEEE Trans.on Image Processing, vol. 8, no. 6, pp. 879-885,June 1999.

[6] G. Ramponi, “Warped distance for space-variant linear image interpolation,” IEEETrans. on Image Processing, vol. 8, no. 5, pp.629-639, May 1999.

[7] J. W. Hwang and H. S. Lee, “Adaptive imageinterpolation based on local gradient features,”IEEE Signal Processing Letters, vol. 11, no. 3,pp. 359-362, March 2004.