Efficient Image Segmentation and Segment-Based Analysis in Computer Vision Applications

Abstract

This dissertation focuses on efficient image segmentation and

segment-based object recognition in computer vision

applications. Special attention is devoted to analyzing shape, of

particular importance for our two applications: plant species

identification from leaf photos, and object classification in remote

sensing images. Additionally, both problems are bound by efficiency,

constraining the choice of applicable methods: leaf recognition

results are to be used within an interactive system, while remote

sensing image analysis must scale well over very large image sets.

Leafsnap was the first mobile app to provide automatic recognition of

tree species, currently counting with over 1.7 million downloads. We

present an overview of the mobile app and corresponding back end

recognition system, as well as a preliminary analysis of

user-submitted data. More than 1.7 million valid leaf photos have been

uploaded by users, 1.3 million of which are GPS-tagged. We then focus

on the problem of segmenting photos of leaves taken against plain

light-colored backgrounds. These types of photos are used in practice

within Leafsnap for tree species recognition. A good segmentation is

essential in order to make use of the distinctive shape of leaves for

recognition. We present a comparative experimental evaluation of

several segmentation methods, including quantitative and qualitative

results. We then introduce a custom-tailored leaf segmentation method

that shows superior performance while maintaining computational

efficiency.

The other contribution of this work is a set of attributes for

analysis of image segments. The set of attributes is designed for use

in knowledge-based systems, so they are selected to be intuitive and

easily describable. The attributes can also be computed efficiently,

to allow applicability across different problems. We experiment with

several descriptive measures from the literature and encounter certain

limitations, leading us to introduce new attribute formulations and

more efficient computational methods. Finally, we experiment with the

attribute set on our two applications: plant species identification

from leaf photos and object recognition in remote sensing images.