License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/LIPIcs.GISCIENCE.2018.17
URN: urn:nbn:de:0030-drops-93450
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2018/9345/
Go to the corresponding LIPIcs Volume Portal

Yan, Bo ; Janowicz, Krzysztof ; Mai, Gengchen ; Zhu, Rui

xNet+SC: Classifying Places Based on Images by Incorporating Spatial Contexts

pdf-format:
LIPIcs-GISCIENCE-2018-17.pdf (0.5 MB)

Abstract

With recent advancements in deep convolutional neural networks, researchers in geographic information science gained access to powerful models to address challenging problems such as extracting objects from satellite imagery. However, as the underlying techniques are essentially borrowed from other research fields, e.g., computer vision or machine translation, they are often not spatially explicit. In this paper, we demonstrate how utilizing the rich information embedded in spatial contexts (SC) can substantially improve the classification of place types from images of their facades and interiors. By experimenting with different types of spatial contexts, namely spatial relatedness, spatial co-location, and spatial sequence pattern, we improve the accuracy of state-of-the-art models such as ResNet - which are known to outperform humans on the ImageNet dataset - by over 40%. Our study raises awareness for leveraging spatial contexts and domain knowledge in general in advancing deep learning models, thereby also demonstrating that theory-driven and data-driven approaches are mutually beneficial.

BibTeX - Entry

@InProceedings{yan_et_al:LIPIcs:2018:9345,
  author =	{Bo Yan and Krzysztof Janowicz and Gengchen Mai and Rui Zhu},
  title =	{{xNet+SC: Classifying Places Based on Images by Incorporating Spatial Contexts}},
  booktitle =	{10th International Conference on Geographic Information  Science (GIScience 2018)},
  pages =	{17:1--17:15},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-083-5},
  ISSN =	{1868-8969},
  year =	{2018},
  volume =	{114},
  editor =	{Stephan Winter and Amy Griffin and Monika Sester},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{http://drops.dagstuhl.de/opus/volltexte/2018/9345},
  URN =		{urn:nbn:de:0030-drops-93450},
  doi =		{10.4230/LIPIcs.GISCIENCE.2018.17},
  annote =	{Keywords: Spatial context, Image classification, Place types, Convolutional neural network, Recurrent neural network}
}

Keywords: Spatial context, Image classification, Place types, Convolutional neural network, Recurrent neural network
Collection: 10th International Conference on Geographic Information Science (GIScience 2018)
Issue Date: 2018
Date of publication: 02.08.2018

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI