Документ взят из кэша поисковой машины. Адрес оригинального документа : http://imaging.cs.msu.su/pub/2010.ICPR.Nasonov_Krylov.SR_Med.en.pdf Дата изменения: Tue May 11 23:00:00 2010 Дата индексирования: Sat Apr 9 23:25:14 2016 Кодировка:

Fast super-resolution using weighted median filtering
Andrey V. Nasonov, Andrey S. Krylov Laboratory of Mathematical Methods of Image Processing Faculty of Computational Mathematics and Cybernetics Lomonosov Moscow State University nasonov@cs.msu.ru , kryl@cs.msu.ru

Abstract--A non-iterative method of image super-resolution based on weighted median filtering with Gaussian weights is proposed. Visual tests and basic edges metrics were used to examine the method. It was shown that the weighted median filtering reduces the errors caused by inaccurate motion vectors. Keywords-super-resolution; weighted median filtering

I . I N T RO D U C T I O N Image resampling is one of the most important problems in image processing. Despite the increase of the resolution of modern camera sensors, this problem is still actual, for example, for old low-resolution video data and it is very important for surveillance applications. Many image resampling algorithms use a priory information. As an example, image self-similarity at different resolutions is used in NEDI algorithm [1]. Nevertheless this approach improves the image visual quality only if the a priori information is true. Super-resolution is an alternative method to obtain better results. Here several images of an object with subpixel shifts (see figure 1) are used to construct single high-resolution image [2], [3]. Camera sensors have non-zero size, and the observed pixel value is an approximation of image intensity in a certain area. If the object motion and the approximation function are known, then the information from all frames can be used to construct a single high-resolution image. The main problem of super-resolution is the requirement of accurate motion estimation. In this work, we propose a super-resolution method stable enough to the errors of motion vectors estimation. I I . M AT H E M AT I C A L M O D E L The super-resolution task can be posed as an inverse problem. The corresponding direct problem includes a set of the downsampling procedures. It produces the low-resolution images uk after motion transformation and downscaling from the high-resolution image z as: Ak z = uk , k = 1, 2, ..., N .

Figure 1. The correspondence between pixels of low-resolution images (top row) and pixels of high-resolution image (bottom row).

The atmosphere and camera lens blurs are modeled by a single Gauss filter H , and the system of equations takes the form Ak z = DFk H z = uk , k = 1, 2, ..., N . (1)

Motion estimation algorithms [5], [6] are used to calculate motion operators Fk . Many of them use optical flow method for accurate motion estimation [5]. Nevertheless the inaccuracy of the motion estimation remains very important in super-resolution and it is necessary to reduce its influence on the super-resolution result. The super-resolution task described by the inverse problem for the system of equations (1) is an ill-posed problem. Iterative regularization methods based on Tikhonov regularization [7] are used to make this problem wellposed, but these methods are time consuming. Fast superresolution uses non-iterative methods. For example, in [8] the authors calculate the average sum of upsampled and motion compensated low-resolution images zH = H z and then sharpen the result using z = H -1 zH transformation. The idea of the proposed super-resolution method is close to this method but weighted median filtering with Gaussian weights is used. I I I . P RO B L E M D E FI N I T I O N We consider the super-resolution problem (1) with z and uk defined on discrete sets {(i, j ) : i, j Z}. The motion transformation operator Fk defines a set of correspondences

The operator Ak in the general case is represented as Ak = DHcam Fk Hatm z + n [4], where Hatm is the atmosphere blur, Fk is the motion operator, Hcam is the camera lens blur, D is a downscaling operator, n is a noise.


between the coordinates of points of source image and points of motion transformed image:
k Fk z (i, j ) = z (xk , yi,j ). i,j

Operator D performs downscaling: Dz (x, y ) = z (sx, sy ), where s is a downscaling factor, and the operator DFk has the form k DFk z (x, y ) = z (xk , ysi,sj ). si,sj Using notation (x , ) for ( Ak z = DFk (H z ) can be written as
k i,j k yi,j

For every pixel (i, j ) of the high-resolution image (H z ) we take all points (xn , yn ) from a small neighborhood of (i, j ) like shown in figure 2 and perform an averaging of these values. A simple and fast averaging method is based on Gaussian filtering [8]: (H z )(i, j ) =
n

wn e-
n

(xn -i)2 +(yn -j )2 2 2

e

-

(xn -i)2 +(yn -j )2 2 2

,

(3)

xk si,sj

,y

k si,sj

), the operator

k Ak z (i, j ) = DFk (H z )(i, j ) = (H z )(xk , yi,j ), i,j

and the system of equations (1) takes the form
k (H z )(xk , yi,j ) = uk . i,j i,j

By omitting multi-dimensional indexing we compose several upsampled and motion transformed low-resolution k images into a single image and rewrite (xk , yi,j , uk ) i,j i,j as (xn , yn , wn ). Finally we obtain the following superresolution problem statement: to find the high-resolution image (H z ) with the known values wn in the given points (xn , yn ) (see figure 2): (H z )(xn , yn ) = wn . (2)

where radius is chosen experimentally in accordance to the scale factor and accuracy of the motion vectors. Gaussian averaging results in image blur. A robust approach based on median filtering was suggested in [9]. It upsamples the low-resolution images and combining the upsampled images using median averaging. An adaptation of the median averaging to construct (H z ) is presented in [10]. It applies the median averaging to the values of all points in the neighborhood of the target pixel: (H z )(i, j ) = = med(wn : (xn - i)2 + (yn - j )2 < R2 ). (4) Median averaging produces sharp edges but it does not use the spatial distribution of points (xn , yn ). To take the benefits of both Gauss averaging (3) and median averaging (4), we propose a combined method based on weighted median averaging. In weighted median wmed(wn , cn ), every value wn has a weight cn . We choose the weights cn as in the Gauss averaging: (xn - i)2 + (yn - j )2 ). 2 2 If the weights cn are natural numbers, we calculate the weighted median as median average with wn taken cn times. To find the result of the median averaging wmed(wn , cn ) in the general case, the pairs (wn , cn ) are sorted in the ascending order of wn . Next, we find the value m which satisfies the conditions cn = exp(-
m-1 m

ck S/2,
k=1 k=1

ck > S/2,

S=
k

ck ,

Figure 2. The illustration for super-resolution problem statement (2). Pixels of the low-resolution images are shown as , and . Pixels of the highresolution image are shown as ·.

I V. P RO B L E M S O L U T I O N Even small errors in motion estimation result in serious degradation of the reconstructed image. If a super-resolution algorithm does not take into account the inaccuracy of motion vectors, it gives unsatisfactory results. Therefore instead of constructing the high-resolution image (H z ) which satisfies the equation (2) in every point (xn , yn ), we use the following algorithm:

and take the value wm as the result of the weighted median averaging. This process is illustrated in figure 3. We represent every pair (wn , cn ) as a rectangle with the width cn and fixed height. Next we construct a long rectangle with width S by connecting these rectangles in the ascending order of wn . Finally we take the value wm of the rectangle in the middle of the constructed rectangle as the result of the weighted median. The obtained result is the approximation of the blurred image (H z ). But actually the proposed method produces sharp images. This is caused by the behavior of the median filtering. If we increase the radius of the median filtering then


VI. CONCLUSION Non-iterative image super-resolution method based on the weighted median averaging has been proposed. It was shown that weighted median averaging reduces the errors caused by inaccurate motion vectors. The work was supported by RFBR grant 10-01-00535 and by grant of Human Capital Foundation. REFERENCES
[1] X. Li and M. Orchard, "New edge-directed interpolation," IEEE transactions on image processing, vol. 10, no. 10, pp. 1521­1527, 2001. [2] S. Borman and R. Stevenson, "Super-resolution from image sequences - a review," Midwest Symposium on Circuits and Systems, pp. 374­378, 1998. [3] K. Malczewski and R. Stasinski, "Super resolution for multimedia, image, and video processing applications," Recent Advances in Multimedia Signal Processing and Communications, vol. 231, pp. 171­208, 2009. [4] S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multi-frame super-resolution," IEEE Transactions on Image Processing, vol. 13, no. 10, pp. 1327­1344, 2004. [5] A. Bruhn, J. Weickert, and C. Shnorr, "Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods," International Journal on Computer Vision, vol. 61, no. 3, pp. 211­231, 2005. [6] H. Le and G. Seetharaman, "A super-resolution imaging method based on dense subpixel-accurate motion fields," Proc. of the Third International Workshop on Digital and Computational Video, pp. 35­42, 2002. [7] A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-Posed Problems. Washington DC: WH Winston, 1977. [8] A. S. Krylov, A. V. Nasonov, and O. S. Ushmaev, "Video super-resolution with fast deconvolution," Pattern Recognition and Image Analysis, vol. 19, no. 3, pp. 497­500, 2009. [9] A. Zomet, A. Rav-acha, and S. Peleg, "Robust superresolution," Proc. of the IEEE Workshop on Applications of Computer Vision, pp. 645­650, 2001. [10] A. Sanchez-Beato and G. Pajares, "Robust super-resolution using a median filter for irregular samples," Lect. Notes in Comp. Science, vol. 5524, pp. 298­305, 2009. [11] A. Nasonov and A. Krylov, "Adaptive image deringing," Proceedings of GraphiCon'2009, pp. 151­154, 2009.

Figure 3.

Weighted median averaging procedure.

we do not increase the image blur of the resulting image and the resulting image tends to be piece-wise flat. Therefore we not apply deblur algorithms to the result and take the result of the weighted median averaging as the result of superresolution. V. R E S U LT S To test the results of the proposed method we applied random shifts to reference high-resolution images, and the obtained images were downsampled. Then the high-resolution images were reconstructed from these low-resolution images by the super-resolution algorithm and compared with the reference images. To test the stability of the proposed method to the motion vectors errors, we added noise to the motion vectors. A random uniformly distributed value in [-1, 1] range was added to every motion vector for a quarter of the input images. A random value in [-0.25, 0.25] range was added to motion vectors for other images. The results for the proposed super-resolution method are shown in figure 4. It can be seen that the proposed method has better visual quality than of the method based on median averaging and the method based on Gaussian averaging. Median averaging produces noisy edges while Gaussian averaging blurs the edges in the case of errors in motion vectors estimation. To measure numerically the quality of the result of the super-resolution algorithm, we used edge adaptive metrics from [11]. The metrics B E P (Basic Edges Points) calculates the mean square error (MSE) in the points of sharp and isolated edges -- basic edges. The metrics B E N (Basic Edges Neighborhood) calculates the MSE in the basic edges neighborhood. The weighted median averaging shows better B E N and B E P than the Gaussian averaging, but the results are contradictory in comparison to the median averaging. The weighted median averaging gives better results than the median averaging in edge areas (B E P areas), but is worse in the areas near the edges (B E N areas). Nevertheless the problem of edge reconstruction is usually more significant than the reconstruction of non-edge areas in the case of erroneous motion vectors. Thus we make a conclusion that the weighted median averaging method shows better results than the median averaging method or the Gaussian averaging method. The results can be improved by varying the Gauss radius and the radius R for different pixels of the image. This will be part of the future work.


Reference images.

B E P = 55.34, B E P = 17.00, B E N = 0.26 B E N = 5.36 The result of applying erroneous motion vectors to an upsampled input image.

Low-resolution images (shown 4 of 16).

B E P = 35.0, B E P = 9.73, B E N = 5.63 B E N = 4.47 Super-resolution using Gauss averaging (3).

B E P and B E N areas. White areas are B E P areas, gray areas are B E N areas, thin gray lines are the edges.

B E P = 25.73, B E P = 9.08, B E N = 0.22 B E N = 3.66 Super-resolution using median averaging (4).

Example of estimated motion vectors (left image) and corresponding erroneous motion vectors (right image).

B E P = 20.09, B E P = 6.89, B E N = 0.30 B E N = 3.94 Super-resolution using weighted median averaging.

Figure 4.

The illustration of super-resolution methods for scale factor 2 and 16 initial low-resolution images. Parameter = 1.2.