License: Creative Commons Attribution 4.0 International license (CC BY 4.0)
When quoting this document, please refer to the following
DOI: 10.4230/LIPIcs.GIScience.2021.II.2
URN: urn:nbn:de:0030-drops-147615
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2021/14761/
Go to the corresponding LIPIcs Volume Portal

Ostermann, Frank O. ; Nüst, Daniel ; Granell, Carlos ; Hofer, Barbara ; Konkol, Markus

Reproducible Research and GIScience: An Evaluation Using GIScience Conference Papers

pdf-format:
LIPIcs-GIScience-2021-II-2.pdf (0.7 MB)

Abstract

GIScience conference authors and researchers face the same computational reproducibility challenges as authors and researchers from other disciplines who use computers to analyse data. Here, to assess the reproducibility of GIScience research, we apply a rubric for assessing the reproducibility of 75 conference papers published at the GIScience conference series in the years 2012-2018. Since the rubric and process were previously applied to the publications of the AGILE conference series, this paper itself is an attempt to replicate that analysis, however going beyond the previous work by evaluating and discussing proposed measures to improve reproducibility in the specific context of the GIScience conference series. The results of the GIScience paper assessment are in line with previous findings: although descriptions of workflows and the inclusion of the data and software suffice to explain the presented work, in most published papers they do not allow a third party to reproduce the results and findings with a reasonable effort. We summarise and adapt previous recommendations for improving this situation and propose the GIScience community to start a broad discussion on the reusability, quality, and openness of its research. Further, we critically reflect on the process of assessing paper reproducibility, and provide suggestions for improving future assessments. The code and data for this article are published at https://doi.org/10.5281/zenodo.4032875.

BibTeX - Entry

@InProceedings{ostermann_et_al:LIPIcs.GIScience.2021.II.2,
  author =	{Ostermann, Frank O. and N\"{u}st, Daniel and Granell, Carlos and Hofer, Barbara and Konkol, Markus},
  title =	{{Reproducible Research and GIScience: An Evaluation Using GIScience Conference Papers}},
  booktitle =	{11th International Conference on Geographic Information Science (GIScience 2021) - Part II},
  pages =	{2:1--2:16},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-208-2},
  ISSN =	{1868-8969},
  year =	{2021},
  volume =	{208},
  editor =	{Janowicz, Krzysztof and Verstegen, Judith A.},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/opus/volltexte/2021/14761},
  URN =		{urn:nbn:de:0030-drops-147615},
  doi =		{10.4230/LIPIcs.GIScience.2021.II.2},
  annote =	{Keywords: reproducible research, open science, reproducibility, GIScience}
}

Keywords: reproducible research, open science, reproducibility, GIScience
Collection: 11th International Conference on Geographic Information Science (GIScience 2021) - Part II
Issue Date: 2021
Date of publication: 14.09.2021
Supplementary Material: The input data for this work are the full texts of GIScience conference proceedings from the years 2012 to 2018. The paper assessment results and source code of figures are published at https://github.com/nuest/reproducible-research-at-giscience and archived on Zenodo [Daniel Nüst et al., 2020]. The used computing environment is https://github.com/rocker-org/binder/ pinning the R version to 3.6.3 and R packages to the https://mran.microsoft.com/timemachine of July 5th 2019.

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI