Fine-Grained Land Use Classification at the City Scale Using Ground-Level Images.

We perform fine-grained land use mapping at the city scale using ground-levelimages. Mapping land use is considerably more difficult than mapping land coverand is generally not possible using overhead imagery as it requires close-upviews and seeing inside buildings. We postulate that the growing collections ofgeoreferenced, ground-level images suggest an alternate approach to thisgeographic knowledge discovery problem. We develop a general framework thatuses Flickr images to map 45 different land-use classes for the City of SanFrancisco. Individual images are classified using a novel convolutional neuralnetwork containing two streams, one for recognizing objects and another forrecognizing scenes. This network is trained in an end-to-end manner directly onthe labeled training images. We propose several strategies to overcome thenoisiness of our user-generated data including search-based training setaugmentation and online adaptive training. We derive a ground truth map of SanFrancisco in order to evaluate our method. We demonstrate the effectiveness ofour approach through geo-visualization and quantitative analysis. Our frameworkachieves over 29% recall at the individual land parcel level which represents astrong baseline for the challenging 45-way land use classification problemespecially given the noisiness of the image data.