Документ взят из кэша поисковой машины. Адрес оригинального документа : http://imaging.cs.msu.ru/pub/2012.PRIA.Nasonov_Krylov.EdgeMetrics.en.pdf
Дата изменения: Sat Nov 21 02:49:22 2015
Дата индексирования: Sat Apr 9 23:37:53 2016
Кодировка:

EDGE QUALITY METRICS FOR IMAGE ENHANCEMENT* A.V. Nasonov, A.S. Krylov Laboratory of Mathematical Methods of Image Processing, Faculty of Computational Mathematics and Cybernetics, Lomonosov Moscow State University, 119991, Russia, Moscow, Leninskie Gory, nasonov@cs.msu.ru, kryl@cs.msu.ru

The paper presents a new adaptive full reference method for quality measurement of image enhancement algorithms. The method is based on the analysis of basic edges -- sharp edges which are distant from another edges. The proposed basic edges metrics calculates error values in two areas related to typical artifacts of image enhancement algorithms: basic edges area and basic edges neighborhood. The metrics are illustrated with an application to image resampling and image deblurring but it is also applicable for image deringing and image denoising.

Keywords: image metrics, image enhancement, edges, blur, ringing artifact.

Introduction Development of image metrics is important for the objective analysis of image resampling, deringing, deblurring, denoising and other image enhancement algorithms. Common scheme to estimate the quality of an image enhancement algorithm uses a set of artifact free
*

The work was supported by federal target program "Scientific and scientific-pedagogical personnel of innovative Russia" in 2009-2013 and by RFBR grant 10-01-00535.

reference images. These images are corrupted to simulate the effect which is aimed to be suppressed by the being analyzed image enhancement algorithm. Then the corrupted images are restored using the given algorithm and compared to the corresponding reference images using image metrics. There exist large variety of image metrics [1] ranging from simple but fast approaches like MSE, PSNR to more sophisticated metrics based on the modeling of the human visual system [2]. There also exist no-reference quality estimation algorithms that measure specific artifacts like blur [3] and ringing [4] for certain image restoration algorithms. Most of image metrics can provide an estimation of perceptual image quality but they cannot be used to develop effective image enhancement algorithms. Two image enhancement algorithms can give the same metrics values but the results can be very different if the first algorithm processes edges well and corrupts non-edge area while the second one corrupts only edges. Such an example for image deblurring is shown in Fig.1.

a) Blurred image

b) Results of different image deblurring algorithms

Fig. 1. Application of the unsharp mask with different amount parameter to noisy blurred image. PSNR values are the same, but the edges are sharper in the left result image while non-edge area is better in the right image.

Typical artifacts of image enhancement algorithms are blur and ringing effect near sharp edges. The origin of these artifacts is the loss of the high frequency information during image

corruption and inaccurate reconstruction of the high frequency information by image enhancement algorithms. Using parameters of image corruption and image enhancement method, it is possible to find the areas related to these artifacts and to calculate image quality metrics in these areas separately. This information can be helpful to find the most problem areas of a certain image enhancement algorithm. In this paper, we develop a method to find the areas related to two typical image artifacts: edge blur and ringing effect. An algorithm to find the area related to ringing effect is proposed in [5], but this algorithm has limitations and cannot be applied for most of image enhancement algorithms. Our proposed method is based on the concept of basic edges -- sharp edges which are distant from other edges thus surviving after image corruption. The perceptual metrics for these areas are suggested. The proposed metrics estimate the quality of different image enhancement methods by analyzing the image quality in the areas of blur and ringing effect. Image degradation type and its parameters are supposed to be known. In section 1, we analyze blur and ringing effect for image enhancement of low-resolution images, blurred images and images with ringing effect. In section 2, we find the edges suitable for image quality estimation. In section 3, we introduce our metrics to estimate the quality of image enhancement methods. Application of the proposed metrics to image resampling and image deblurring is shown in section 4.

1. Artifact analysis Since both blur and ringing effect are the results of the loss of high frequency information, these effects should be considered together. If all frequencies above

1 Hz are 2p

truncated in Fourier transform, ringing oscillations appear and edges are blurred. The length of single ringing oscillation and edge width are equal to p pixels. The example of high frequency

truncation is shown in Fig. 2. Although the number of ringing oscillations is unlimited for the high frequency cut off, usually no more than 12 oscillations are noticeable. We will call parameter p as the cut off parameter.

Original image

After high frequency information cut off

Edge profiles:

2D Fourier transform modulus:

Fig. 2. Appearing of blur and ringing effect after high frequency information cut off for p = 4.

In practice, the high frequency information is usually corrupted but not completely absent, and the cut off frequency cannot be obtained directly from Fourier transform. In this case additional investigations are required to estimate blur and ringing effect parameter. We can also predict the parameter's value from the image degradation type a priori. Low-resolution images are constructed using downsampling procedure which includes low-pass antialiasing filtering followed by the decimation procedure. During the decimation with scale factor s , the frequencies greater than

1 are discarded. The cut off is not ideal because of 2s

the two-dimensionality of the image. For any linear image resampling method producing blur and ringing effect, the parameter p depends only on scale factor s and p s . For non-linear image resampling methods we use p s too. In image deringing the parameter p is already known from the definition of the problem. Blurred images are the results of low-pass filtering followed by a noise addition. We consider Gaussian blur with known radius and a noise with Gaussian distribution. There is no frequency cut off, and parameter p depends on image deblurring method. For the unsharp mask, we use p k , where 2.5 k 3 .

We have performed frequency analysis of different image enhancement algorithms to confirm the preposition that parameter p can be estimated from image degradation method. For a pair of reference image v and restored image u we calculate the cumulative spectrum error function A( w) (CSEF):

^ A( w) | f ( w cos , w sin ) |2 d ,
0

2

where f ( w1 , w2 ) is linearly interpolated discrete Fourier transform of the error image f u v . The analysis consists in calculating average CSEFs A( w) for reference images from the set of standard images (baboon, cameraman, house, goldhill, lena, peppers) for popular methods of image resampling, deringing and deblurring. The results of this analysis for p = 2 are shown in Fig. 3. It can be seen that most of the image enhancement methods produce error in highfrequency domain and the change of the curve shape happens in the expected point w

1 1 . 2p 4

Fig. 3. Cumulative spectrum error functions differences for different image corruption and enhancement methods.

2. Basic edges

Blur and ringing effect appears near sharp edges. But an arbitrary sharp edge cannot be used for image quality analysis. Some edges can disappear or can be displaced after image corruption. If these edges are used to analyze blur and ringing effect, the results will be incorrect. There are two effects observed in images with corrupted high frequency information: 1. Masking effect. If an edge with low gradient value is located near an edge with high gradient value, it will disappear after image degradation. 2. Edge displacement. If two edges with the same or close gradient values are located near each other, they will be displaced after image degradation. These effects are illustrated in fig. 4 for image blur.

Fig. 4. The effects of edge masking and edge displacement for image blurring. Left: original edge profiles; right: gradient values with marked local maxima. Top row: low blur; middle: medium blur; bottom: strong blur. Therefore, in our algorithm we find edges in the reference image which satisfy the following conditions: 1. An edge point is not masked by nearby edges:

g

i0 , j0

max g i , j ((i i0 ) 2 ( j j0 ) 2 ) ,
i, j

(1)
t
2 2

where g

i, j

is the gradient modulus, function (t ) is the mask function. We use (t ) he

2p

,

where h

1 . p 2

2. The distance from the edge point to the nearest edge is greater than a threshold R. This operation is performed using mathematical morphology [6]. We use R 3 p . 3. The gradient modulus g
i, j

is greater than a threshold g 0 . The condition is used to

reduce the influence of noise on blur and ringing effect. We call the edges passed the first condition as non-masked edges and the edges passed all these conditions as basic edges. The basic edges persist after image degradation following by reconstruction with cut off parameter less than p.

3. Image quality metrics After detection of basic edges, we calculate two sets: 1. The set M 1 containing all pixels for which the nearest non-masked edge is a basic edge and the distance to this edge is less or equal than 2 p . Blur effect is the most likely to appear in this set. 2. The set M 2 containing all pixels for which the nearest non-masked edge is a basic edge and the distance to this edge is greater than the most likely to appear in this set. The example of finding these sets is shown in Fig. 5.

p and less or equal than 2 p . Ringing effect is 2

Original image.

White edges are the edges passed the condition (1), dark gray edges-- not passed.

White points are points of basic edges, dark lines are non-masked non-basic edges.

White area is the set M 1 , gray area is the set M 2 .

Fig. 5. The result of basic edges detection for p 4 .

To measure image quality, we calculate metrics values in the sets M 1 and M 2 using SSIM [7]:

S S IM

(u, v; M )

(2u v c1 )(2 uv c2 ) , 2 2 ( u v2 c1 )( u v2 c2 )
2 u

where u and v are the means of u and v respectively,

and v2 are the variances,

uv

is

the covariance of u and v, c1 (0.01L) 2 , c2 (0.03L) 2 , L is the dynamic range of pixel values (typically this is L = 255). The values u , v , u2 , v2 and
uv

are calculated only in the set M.

Finally we introduce the image quality value vector for image u with ground truth image v and given cut off parameter p:

Quality (u, v; p) ( 0 , 1 , 2 , ) ,
where 0
S S IM

(u, v) , 1

S S IM

(u, v; M1 ) , 2

S S IM

(u, v; M 2 ) . The value 0 is calculated in

the entire image.

4. Results The problem of optimal constructing of a combination of image enhancement algorithms was used to justify the effectiveness of the proposed metrics. Consider the results of two image enhancement algorithms, where the first result u shows bad 1 but good 2 while the second one v has good 1 and bad 2 . It is natural to take linear combination of these methods to improve the result

wi , j a(di , j )ui , j (1 a(di , j ))vi , j ,
where d
i, j

is the distance to the nearest non-masked edge from the pixel (i, j ) , a(d ) is the

weight coefficient depending on the distance d i , j . We use the following a(d ) function:

p d , 0, 2 p 2d p a(d ) , d p, 2 p d p. 1,
The result for combination of unsharp mask and regularized total variation (TV) deconvolution in low-frequency domain [8] for the problem of image deblurring is shown in Fig. 6. The unsharp mask algorithm performs high frequency amplification

z Hu (1 )(u Hu)
where H is the Gaussian filter with same as was used for image blur, is the amplification parameter. The idea of the regularized total variation deconvolution in low-frequency domain is to take the low-frequency part of the blurred image Hu , perform its sharpening using unsharp mask with high parameter (we use 5 ) and then project the result into the set of bounded total variation and add the high-frequency information without amplification:

z R arg

TV ( z )TV ( Hu )

min

2 HHu (1 )( Hu HHu) z 2 (u Hu) .

For the problem of image resampling, we construct the combined image for bilinear and ideal (sinc) interpolation algorithms. The results are shown in Fig. 7. It can be seen that ringing effect is suppressed while the edges remain sharp.

Reference image.

Blurred ( 3) and noisy observation.

0 0.9828, 1 0.8168, 2 0.9749 .

Regularized deconvolution [8].

Unsharp mask with 2 .

0 0.9844, 1 0.9520, 2 0.9754 .

0 0.9833, 1 0.8791, 2 0.9852 .

Combined method.

0 0.9889, 1 0.9409, 2 0.9901 .
Fig. 6. Application of the proposed metrics to improve the results of deblurring methods.

Reference image.

Low resolution image.

Sinc interpolation.

Bilinear interpolation.

0 0.9848, 1 0.9708, 2 0.9958 .

0 0.9837, 1 0.9665, 2 0.9962 .

Combined method.

0 0.9851, 1 0.9708, 2 0.9962 .
Fig. 7. Application of the proposed metrics to improve the results of image resampling algorithms.

Conclusion New full-reference metrics for quality measurement of image enhancement algorithms based on the analysis of typical artifacts of image enhancements methods have been developed. These metrics were approbated on image resampling and image deblurring. It looks promising for combining two different image enhancement algorithms to obtain better result.

References 1. Ahumada A. J. Computational image quality metrics: a review // SID Digest. 1993. Pp. 305308. 2. Lin W.S. Digital Video Image Quality and Perceptual Coding, chapter Computational Models for Just-Noticeable Difference. CRC Press. 2006. Pp. 281303. 3. Ferzli R., Karam L. J. Human visual system based no-reference objective image sharpness metric // Proceedings of International Conference on Image Processing (ICIP'06). 2006. Pp 29492952. 4. Balasubramanian M., Iyengar S. S., Reynaud J., Beuerman R. W. A ringing metric to evaluate the quality of images restored using iterative deconvolution algorithms. // Proceedings of the 18th International Conference on Systems Engineering (ICSENG'05). 2005. Pp. 483488. 5. Marziliano P., Dufaux F., Winkler S., Ebrahimi T. Perceptual blur and ringing metrics: Application to JPEG2000 // Signal Processing: Image Communication. 2004. Vol. 19, no. 2. Pp. 163172. 6. Nasonov A.V., Krylov A.S. Basic Edges Metrics for Image Deblurring // Proceedings of 10th Conference on Pattern Recognition and Image Analysis: New Information Technologies, St. Petersburg. 2010. Vol. 1. Pp. 243246.

7. Wang Z., Bovik A., Sheikh H., Simoncelli E., Image quality assessment: from error visibility to structural similarity // IEEE Transactions on Image Processing. 2004. Vol. 13, no. 4. Pp. 600612. 8. Krylov A.S., Nasonov A.V. Adaptive image deblurring with ringing control // Fifth International Conference on Image and Graphics (ICIG '09), Xi'an, Shanxi, China. 2009. Pp. 7275.