Документ взят из кэша поисковой машины. Адрес оригинального документа : http://imaging.cs.msu.su/pub/2011.Graphicon.Samsonov_Ignatenko.CameraCalibration.en.pdf Дата изменения: Mon Aug 22 14:17:10 2011 Дата индексирования: Sat Apr 9 23:46:01 2016 Кодировка:

Algorithms for calculating parameters of virtual scenes in computer vision tasks
Pavel Samsonov, Andrey Ilyin, Aleksey Ignatenko Department of Computational Mathematics and Cybernetics
Moscow State University, Moscow, Russia

psamsonov@graphics.cs.msu.ru, ailyin@graphics.cs.msu.ru, aignatenko@graphics.cs.msu.ru

Abstract
In this paper we describe methods for deter mining the parameters of the camera and the light spot, which due to its properties is similar to a point light. The position and the rotation of the camera in the scene coordinates are determined, as well as the position of the light spot, its intensity and color. Parameters are determined from the calibration object in the photograph and some known properties of the camera. Keywords: Camera calibration, determination of illumination parameters, calibration object.

1. INTRODUCTION
In computer graphics many problems are associated with the tasks of three-dimensional reconstruction. For example, a simple synthesis of a textured three-dimensional model of the object by photos. For it not only an array of photographs of the object from different angles is needed, but also the camera positions from which the shots were made. For physically accurate simulation of reflective surface properties of objects we also need light spots' positions, spots' brightness and color. If there are lots of pictures, hand measurements of each of the camera (and maybe a light spot) positions can take a long time. Also, often the accuracy of manual measurements is not enough for such tasks. In this paper algorithms and methods of automatic determining the camera and lighting settings. The input parameters to the algorithms are some camera settings, the size of a calibration object and a set of photographs, which depict the calibration and monitored objects. The positions of cameras and light spots in different photographs may differ. The output is the camera and light spot parameters that were searched for and undistorted images.

relatively to the checkerboard and placing them on a monochrome base, which would greatly improve the accuracy of finding the tips of the shadows. In the method of Roger Y. Tsai, the main problem is to find the calibration points in the image. The authors suggested a method based on the two previous: on the edges of the gray card small checkerboards of different sizes were placed, what greatly accelerated the determination of calibration points and increased the number of n matches in the PnP - problem. Determination of parameters of light is less popular task, and the only method that was found by authors is to determine the parameters of light by a mirror sphere. But in order for results were accurate the method requires perfectly circular mirror sphere of large size, which is difficult to find, and it like the checkerboard takes much space on the photos, substantially degrading the accuracy of reconstruction of the object.

3. THE PROPOSED METHOD
For physical accuracy, some non-linear filters are applied on the original photo. They consider the camera response curve and distortion coefficients. At the stage of camera calibration special calibration points on the calibration object are searched for and then matrix of external camera calibration and coordinate systems of camera are calculated. At the stage of the light parameters determination position of the light spot is restored by cones on the calibration object and by the shadows from them. Then by several photos with known light spot coordinates the intensity and the color of light are restored. When realizing the method the OpenCV library was used.

3.1 Calibration object

2. EXISTING METHODS REVIEW
Currently, we found several existing methods for camera calibration and light parameters determination. The best-known and commonly used approaches in the camera calibration are the standard method, proposed by Roger Y. Tsai ([4]) and the more modern method, developed by Zhengyou Zhang ([5]). Standard calibration method is to search for specific points on the image and to solve PnP-task(match n points on the image and in the scene, where n must be at least 4). The newer method involves using a checkerboard(its cells can be quickly and easily found on the image, and in this method not only points are matched, but also lines, what gives some additional accuracy in determining the camera position and distortion coefficients). Zhengyou Zhang method for calibrating the camera in this task could not be used because calibration object should not take much space in the photo whereas accuracy of Z. Zhang's camera calibration method is highly dependent on the size of the chessboard in the image. There is also difficulty in fixing cones

Figure 1: The calibration object synthesized in 3Ds Max


As part of the problem being solved was the design of calibration object (see Figure 1). The object consists of: Base, which is a rectangular gray card of known dimensions 4 checkerboards in the corners of the gray card of size 4x4, 4x5, 5x5 and 5x6 with known size of cells n cones with a certain height, n 2, for the good accuracy of the desired number n 5; columns are arranged in an arbitrary manner



Approximate these contours with 4-vertex polygons Among these select the quadrangles resembling calibration pattern squares Extract corners of the selected quads, having at least one corner in vicinity Group the corners of the selected quadrangles in lines according to calibration object size

Figure 2: Calibration object with different test objects on it

3.2 Response curve
To achieve physical accuracy, photos should be first processed to restore true colors by non-linear transformation. Response curve is vector function , where P is pixel intensity in the image and M is pixel intensity in the camera matrix. To restore initial colors on the matrix, reverse transformation can be used by formula . In different cameras response curves may differ very much. Figure 4: Found calibration points, highlighted with red circles Then using the method of Roger Y. Tsai from the known relations between gauge points in the scene and the image and from the intrinsic calibration matrix extrinsic and a full calibration are calculated. If necessary, also at this stage the homography matrix and/or distortion coefficients are computed and then distortion is removed from the photographs. Using 3x4 full calibration matrix(F) to calculate image point(ip) from scene point(sp): ( )

Using 3x3 homograph matrix(H) to calculate scene point(sp) = (X; Y; 0), from image point(ip) = (x; y): Figure 3: Response curves of some cameras. Response curve can be achieved from camera's passport characteristics. But sometimes they are incorrect; in this case response curve can be calculated by an external program, for example, "HDRShop" (http://www.debevec.org/HDRShop). ( )

Before camera calibration, also base centers should be marked by user. Because cones in the scene may be of any color and size, automatic detection for them is hardly possible.

3.3 Camera calibration
For camera calibration it is necessary to compare the coordinates of the scene and points on the image. Coordinates of the scene points can be calculated using the input parameters (properties of the calibration object: the size of a gray card, chess cells, etc.). Points on the calibration object are searched through the following steps: Image binarization by thresholding to segment black and white squares Find corners of the black squares Find contours of the boundaries of the black regions Select contours of suitable shape

Figure 5: Yellow points are marked by user cone base centers, green points are restored cone peaks.


3.4 Calculation of distortion coefficients
Distortion is a deviation from rectilinear projection (a projection in which straight lines in the scene remain straight in an image). It appears as a failure of a lens to be rectilinear. By matches between calibration points in the scene and on the image distortion coefficients can be calculated. Then using the distortion coefficients image can be undistorted, so straight lines on the image become actually straight. But sometimes it leads to smoothing image in some regions. More information can be accessed in Brown D.C.'s article ([7]).

will be . So the real point is somewhere in the cloud of these N points. To recover the coordinates of the light spot from the cloud of points 2 formulae were used: arithmetic mean of these points and the "weight" formula, which calculates the sum where those points for which the distance to other points is less are included with larger weights. Tests showed that the "weight" formula often provides with more accurate values of the light spot position than the arithmetic mean.

3.5 Determination of parameters of light
To determine the position of the light spot cone shadow tips should be found in the photo and translated into world coordinates. Initially, the image is segmented using the Mean Shift algorithm ([6]) to find shadow areas. First, those areas which have common points with small circular areas located between the tips of the cones and the centers of their bases are removed from consideration, greatly reducing the probability of error. Then the image is searched for shadows on the following criteria: Distance from the nearest point of the image to the center of the cone base Distance from the furthest point of the center of the cone base The average brightness of pixels in area (the shadow is almost always darker than the surrounding areas) Linear elongation region (cubed distance between the maximum and minimum distance from the center of the cone base points of the domain divided by the square area)

4. RESULTS
Algorithms of camera calibration, distortion correction and determination the position of the light spot have been successfully verified first on synthetic and then on real data.

4.1 Synthetic data
Synthetic data verification was held on two sample images. Known and calculated light spot coordinates were compared to compute error. Error camera position vector length was less than 1.5% of the length of the true position vector of the camera, the error vector of the light spot position length by 5 cones was less than 5% of the true position vector of the spot. Here are presented two tables of errors for each light spot calculating formula when processing synthesized images. Formula to compute errors was Err =


* 100%, where is real

position of light spot and is position, which was calculated by the algorithm. Formula "average mean" 2 3 4 5 n con con con con es es es es Sample 1 90% 25% 14% 5 .8 % Sample 2 14% 5% 2 .8 % 2 .6 %

Formula "with weights " 2 3 4 5 n con con con con es es es es Sample 1 90% 13% 6% 4 .7 % Sample 2 14% 5% 3 .8 % 2 .2 %

4.2 Real data
Calculation errors on real data can't be calculated with the same accuracy as on the synthetic. Measurements were made of the error of both restoring the camera and light spot positions. Tests were performed on a large array of photographs in which the positions of the camera and light spot have not changed as the calibration object moved. We compared the position of light spots in the camera coordinate system. The average error was approximately 4.45% (if more precision is needed - just increase the number of cones).

Figure 6: Restoring the position of the light spot by cone shadow tips We know can calcu Figure 4). midpoints world coordinates of shadow tips and cone peaks, so we late the approximate intersection point of rays (see If number of these rays equals n, then the number of the of the common perpendiculars between every two rays


4.3 Re-projection
Also, the accuracy of the restored position of the light spot can be estimated visually. After the calculation of the light spot position, rays are projected through the cone tops back to the gray card, and the closer they are to their initial positions the more accurate the restored light spot position is.

[5]

[6]

[7]

Z. Zhang, "A Flexible New Technology for Camera Calibration" IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000. Fukunaga, Keinosuke; Larry D. Hostetler (January 1975). "The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition". IEEE Transactions on Information Theory (IEEE) 21 (1): 32­40. Brown DC (1966). "Decentering distortion of lenses". Photogrammetric Engineering, 7: 444­462.

Figure 7: Re-projection Blue points in circles are marked shadow tips. As can be seen in the photographs, the point shift is little, so the accuracy of reconstruction of the light spot is rather high.

5. CONCLUSION
In general, the methods described in this article were already successfully used by several people in their works. Accuracy of reconstruction by 5 cones is large enough, but if you need additional accuracy, you can add several more cones. Only automatic detection of calibration points and the tips of the shadows are problematic features of this work. Averagely, about 80% of points are recognized on each photo as the process takes from several seconds to half a minute. In the future the author intends to develop more accurate and faster methods of recognition of these points in the images.

6. LINKS
[1] James A. Paterson, David Claus, Andrew W. Fitzgibbon, "BRDF and geometry capture from extended inhomogeneous samples using flash photography", EUROGRAPHICS 2005 Volume 24, Number 3 p. 4. Alexey Kravtsov and Vladimir Vezhvenets, "Solve of PnP problem". Computer graphics and multimedia, Volume 1(3)/2003. Alexey Kravtsov and Vladimir Vezhvenets, "General formulation of extrinsic camera calibration task". Computer graphics and multimedia, Volume 1(3)/2003. Roger Y. Tsai, "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses" IEEE Journal of Robotics and Automation 1987 Volume 3 Issue 4.

[2]

[3]

[4]