Automatic detection and understanding the text in natural images, such as photographs of city outdoors or building indoors, is a challenging problem. There is a considerable gap between detecting and understanding text in scanned documents (which is a mature technology) and detecting and understanding text in the natural images. Detection and understanding the text in natural photographs involves localizing the text as well as removing the variation factors, such as varying text orientation, font, color and lighting. Examples of natural scene texts are shown below.

• Pixel-wise annotation of ICDAR dataset (download)
• Matlab toolbox for scene text binarization implementing the methods from ICDAR 2013 paper (download)

Team

• Olga Barinova, Lomonosov Moscow State University
• Sergey Milyaev, Voronezh State University // sergey.milyaev (at) gmail.com
• Tatiana Novikova, Lomonosov Moscow State University
• Victor Lempitsky, Skolkovo Institute of Science and Technology
• Pushmeet Kohli, Microsoft Research Cambridge

Publications

• Novikova T, Barinova O, Kohli P, Lempitsky V. Large-lexicon attribute-consistent text recognition in natural images. ECCV, 2012 (download)
• Milyaev S, Barinova O, Novikova T, Lempitsky V, Kohli P. Image binarization for end-to-end text understanding in natural images. ICDAR, 2013 (download)

Acknowledgements

This project is supported by Microsoft Research programs in Russia.