Introduction

Roadway video passport systems are widely-used for detailed road mapping which allows roadway quality monitoring and repair planning. The system includes a body of video cameras and other sensors mounted on a car as shown below. The car travels along a road and gathers data from the sensors.

Video cameras are mounted on the front side and on the back side of the road laboratory. Example of video frame is shown at the right.

Nowadays video sequences gained by these mobile laboratories are usually processed off-line. Operator manually marks objects like traffic signs, road edges, roadway defects (holes and cracks), and roadway patches on each video frame; the procedure is laborious and takes plenty of time. Typical roadway images are strongly differed to each other in colors and texture, some examples are shown below.

Typical source images for roadway analisys

This project aims at alleviation of road video processing preserving the accuracy of road mapping. Our method provides interactive detection of road-covering defects. We use so-called rectified images of the road covering. These images are obtained from video frames with the aid of perspective plane transformation. Our method is integrated into roadway video passport system.

System description

Overview

Our method proceeds as follows. First video sequence is preprocessed in off-line mode. When preprocessing is completed our algorithm is trained on a part of rectified road image with the help of user input. After that objects on the rest part of the road image are detected automatically.

Outline of road image segmentation

Preprocessing

Preprocessing stage is performed off-line. The first goal of this stage is obtaining rectified images from video sequence with the aid of perspective plane transformation. Another goal of preprocessing is normalization of road images, which includes lighting correction, color correction, image denoising, histogram equalization, etc. in order to reduce the variance in road images appearance.

Automatic object detection

We take advantage of image segments classification for object detection on road images. Features used for classification of image segments include color statistics, area, sizes of a segment projected on coordinate axes, elongation, orientation and other cues that capture shape, color and location.
User Interaction
As long as roadway material, time of survey and weather conditions can vary substantially for different roads, we developed partly automated interactive method for road objects detection. Our method is based on online learning that allows to account for specific characteristics of every section of the road.

The outline of user interaction with the system is as follows:

• automatic object detection is applied to a section of road image
• user checks results of automatic method and corrects some typical errors
• object detection algorithm is re-trained in order to account for new examples

After several iterations of this procedure objects are automatically detected on the rest part of the road image.

Online learning helps to improve the quality of automatic object detection during user interaction. Thus our method requires minimal user effort.

Results

Source image	Segmented image	Analisys results

Source image	Segmented image	Analisys results

Roadway image processing demo.

Road border detection

Most of the time objects of interest are located on a road surface. And it is convenient to cut out road surface from a rectified road image to decrease processing time and error rate. Our algorithm helps user to do this. It works in interactive mode so user can correct obtained results any time.

User perspective of the workflow is following:

1. User draws a part of correct border for initial algoritm learning

2. After that algorithm detects border on the whole image

3. Border can also be detected on other images of the same road

4. Obtained results can be corrected if something has gone wrong. There are two options:  

• User can draw another border line in the area of wrong detection and perform additional learning of the algorithm on it
• Or user can draw part of the correct border line and detection algorithm will take this information into account

5. After user interation border is redetected

Some results for another types of a road are also shown below:

The project team

Principal researchers:

• Dr. Anton Konushin
• Dr. Andrey S. Krylov

Lead researchers:

• Alexander Velizhev
• Olga Barinova

Researchers:

• Sergey Sudakov
• Roman Shapovalov
• Alexander Semashko
• Alexander Chigorin

Contacts

Please, mail all comments, suggestions, problems and contributions to:

Alexander Velizhev
avelizhev@graphics.cs.msu.su

Olga Barinova
obarinova@graphics.cs.msu.su

We appreciate your assistance!

Publications

• Efficient Road Mapping via Interactive Image Segmentation. Barinova, R. Shapovalov, S. Sudakov, A. Velizhev, A. Konushin, in Proceedings of Object Extraction for 3D City Models, Road Databases and Traffic Monitoring (CMRT), pp.1-7, 2009 [pdf] [Slides]
• Semantic segmentation of road images based on cascade classifiers, Sergey Sudakov, Olga Barinova, Alexander Velizhev, Anton Konushin, to appear in Proceedings of PRIA'2008
• Methods for road marking and patches detection, Sergey Sudakov, Alexander Semashko, Olga Barinova, Anton Konushin, Vladislav Kinshakov and Andrey Krylov, in Proceedings of Graphicon’2008 (in Russian)

References

[1] Scientific development and production center “Region”, http://www.nporegion.ru