Документ взят из кэша поисковой машины. Адрес оригинального документа : http://imaging.cmc.msu.ru/pub/KrNaUs_PRIA2008.pdf Дата изменения: Fri Oct 31 23:00:00 2008 Дата индексирования: Mon Oct 1 19:30:55 2012 Кодировка:

IMAGE SUPER-RESOLUTION USING FAST DECONVOLUTION
A. Krylov2, A. Nasonov3, O. Ushmaev
2 4

1

Faculty of Computational Mathematics and Cybernetics, Moscow Lomonosov State University, 119991, Russia, Moscow, Leninskie gory, (495) 939-11-29, kryl@cs.msu.ru 3 Faculty of Computational Mathematics and Cybernetics, Moscow Lomonosov State University, 119991, Russia, Moscow, Leninskie gory, (495) 939-11-29, nasonov@cs.msu.ru 4 The institute of Informatics problems of the Russian Academy of sciences, 119333, Russia, Moscow, Vavilova, 44, 2, (499) 135-62-60, olegu@bliolink.ru

Super-resolution problem is posed as an inverse deconvolution problem. Fast noniterative super-resolution algorithm based on this approach is suggested.

Introduction The problem of super-resolution (SR) is to recover a high-resolution image from a set of several degraded low-resolution images. This problem is very helpful in human surveillance, biometrics, etc. because it can significantly improve image quality. There are two groups of video SR algorithms: learning- and reconstruction-based. Learningbased algorithms enhance the resolution of a single image using information on the correspondence of sample low- and highresolution images. Reconstruction-based algorithms use only a set of low-resolution images to construct high-resolution image. More detailed introduction into video SR problems is given in [1], [2]. The majority of reconstruction-based algorithms use camera models [3] for downsampling the high-resolution image. The problem is posed as error minimization problem

neglected, Fk is a warping operator like motion blur or motion deformation, H ca m is camera lens blur which is usually modeled by Gauss filter, D is a decimation operator, n is a noise which is usually ignored. For computational purposes, we exchange H = H ca m and Fk and use Ak z DFk Hz . (2) Warping operator Fk can be calculated, for example, using motion calculation at base points and interpolation at other points [4], [5]. Variational optical flow estimation approaches are also widely used [6], [7], [8], [9]. Problem definition Since z is defined on a discrete set, but pixel coordinates ( x* , y* ) Fk ( x, y) do not always belong to the grid, operator H is used both for filtering and interpolation:

z R arg min
z


k 1

N

Ak z wk

2 2

,

(1)

Hz( x* , y* )

( xi , y j )



z ( xi , y j )e



( x * xi ) 2 ( y * y j ) 2
2

2

where z is reconstructed high-resolution image, wk is k-th low-resolution image, Ak is a downsampling operator which transforms high-resolution image into k-th low-resolution image, 2 is standard Euclidian norm. The operator can be generally represented as Ak z DHcam Fk H atm z n , where H a tm is atmosphere turbulence effect which is often
1

( xi , y j )



e



( x * xi ) 2 ( y * y j ) 2
2

2

, (3)

where ( xi , y j ) are grid points, is chosen in accordance with scale factor s . We use 0.4s . Operator D simply rescales the coordinates. Operator vk Ak z (2) takes the form

vk ( xi , y j ) Hz( xik, j , yik, j ) ,

(4)

The work was supported by grant RFBR 06-01-39006-


where

( xi , y j )

are

grid

points,

( xik, j , yik, j ) Fk (sxi , sy j ) . The super-resolution problem (1) takes the form
zR arg min
z i, j,k



Hz( xik, j , yik, j ) wk ( xi , y j ) . (5)

2

By changing multiple indexes with single index, the formula (5) can be rewritten as
z R arg min
z

In two-dimensional case, we process consequently the rows and the columns of the image. For given super-resolution problem (1), we convolve low-resolution images with Gauss filter and calculate coefficients c j from a given set of images. We seek for


n 1

N

Hz( xn , yn ) wn

2

(6)

{c j } arg min
C


k 1

N

wk wk G C 2 .

2

(12)

The problem (6) is ill-posed, so regularization methods [10] are used: N 2 zR arg min Hz( xn , yn ) wn f ( z ) . (7) z n 1 Iterative method for solving (7) is discussed in [2]. In this paper, a non-iterative algorithm for solving (6) is proposed. Deconvolution We consider the problem of deconvolution on a discrete 1D set {xi : xi ih} and the case of Gauss filter G xi2 1 Gi exp ( 2 ) . (8) 2 2 The convolution looks as y Hz z G , (9) yi z j Gi j .
j

Experiments have shown that adaptive filter (11) does not significantly amplify noise. It depends on given images. If the images are noisy, then filter coefficients are smoothed and noise level does not significantly increase after deconvolution. This also means that regularization term (7) is not necessary due to adaptive filter (11) is automatically tuned to noise level. We have compared adaptive filter with unsharp mask z ( y y G) y G . Unsharp mask shows practically the same results, but it takes more time to estimate its parameters ( , ) . Problem solution If the points ( xn , yn ) in (6) are grid points, then deconvolution method using adaptive filter can be used. But in general case coordinates xn , yn are not discrete. So, the algorithm becomes as follows: 1. Estimate the values of Hz at all grid points ( xi , y j ) . 2. Perform deconvolution using adaptive filter. To estimate the values of Hz at grid points, we use Gauss interpolation (3) with a small enough radius, so it does not significantly influences deconvolution. In Figure 1, a result of the proposed superresolution method is compared with other image resampling and super-resolution methods.

The problem of deconvolution is to reconstruct z from given convolution result Hz (10) z H 1 y . 1 Inverse operator H can be constructed using ^ ^^ ^ ^ ^ Fourier transform: y zG , z y / G , and z can be found as a convolution of y with ^ inverse Fourier transform of 1 / G . Operator

H 1 is unbounded. In the case of noisy data it significantly amplifies noise. To avoid this, we use a finite adaptive filter z y C ,

zi

j k



j k

z j ci j , c j c j .

(11)

Coefficients c j in (11) are chosen to minimize

z y C 2 . Filter length k is chosen in a way to make deconvolution fast, but precise enough. We use k 3 .

a) One of the input images interpolated using nearest neighbor method

b) Regularization-based super-resolution [2]

1

The work was supported by grant RFBR 06-01-39006-


3.

c) The proposed supere) Regularization-based resolution method (6) image interpolation [11] Fig. 1. Super-resolution results using 4 input images and scale factor s=4.

4.

Problem discussion Super-resolution method (6) shows very good results if the warping operator Fk is calculated precisely. If it has errors, the solution becomes unstable. To avoid this, we can formulate super-resolution problem (1) in another way: 1N 2 z R arg min Ak z wk 2 . (13) z N k 1 We make single-image super-resolution for every image and then calculate an average image. Using the approach (13) results in blurred image, but without artifacts caused by the inaccuracy in warping operator determining. An example of the comparison of the approaches is shown in Figure 2.

5.

6.

7.

8.

9.

10. 11. a) The proposed superb) Super-resolution for resolution method (6) unstable data (13) Fig. 2. Super-resolution results for 4 input images and factor s=4 for inaccurately defined warping operator.

uniform illumination // Proc. Int. Conf. Graphicon. 2008. - P. 150­155. S. Farsiu, D. Robinson, M. Elad, P. Milanfar. Fast and Robust Multi-Frame Super-Resolution // IEEE Trans. On Image Processing. - 2004. - Vol. 13, No. 10. - P. 1327­1344. Sung Won Park, Marios Savvides. Breaking the limitation of manifold analysis for super-resolution of facial images // IEEE Int. Conf. on Acoustics, Speech and Signal Processing. - 2007. - Vol. 1. P. 573­576. Ha V. Le, Guna Seetharaman. A Super -Resolution Imaging Method Based on Dense Subpixel -Accurate Motion Fields // Proceedings of the Third International Workshop on Digital and Computational Video. - 2002, - P. 35­42. B.D. Lucas, T. Kanade. An iterative image registration technique with an application to stereo vision // Proc. of Imaging understanding workshop . 1981. - P. 121­130. A. Bruhn, J. Weickert, C. Shnorr. Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods // International Journal of Computer Vision. - 2005. - Vol. 61, No. 3. - P. 211­ 231. J. Weickert, C. Shnorr. Variational Optic Flow Computation with a Spatio -Temporal Smoothness Constraint // J. Math. Im. & Vis. - 2001. - Vol. 14. P. 245­255. T. Brox, A. Bruhn, N. Papenberg, J. Weickert. High Accuracy Optical Flow Estimation Based on a Theory for Warping // Proc. 8th European Conf. on Computer Vision. - 2004. - Vol. 4. - P. 25­36. A.N. Tikhonov, V.Y. Arsenin. Solutions of Ill Posed Problems // WH Winston, Washington DC. - 1977. A.S. Lukin, A.S. Krylov, A.V. Nasonov. Image Interpolation by Super-Resolution // Proc. Int. Conf. Graphicon. - 2006. - P. 239­242.

Conclusion Fast non-iterative method for image superresolution has been suggested. The method shows very good results if the warping operator is accurately estimated like in the case of only sub-pixel shifts in the initial image set. Hybrid method to suppress the problems of warping operator inaccurate determining is under work. References
1. S. Borman, Robert L. Stevenson. Super-Resolution from Image Sequences -- A Review // Midwest Symposium on Circuits and Systems. - 1998. - P. 374­378. 2. A.S. Krylov, A.V. Nasonov, D.V. Sorokin. Face image super-resolution from video data with non1

The work was supported by grant RFBR 06-01-39006-