License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/LIPIcs.ICALP.2019.119
URN: urn:nbn:de:0030-drops-106959
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2019/10695/
Go to the corresponding LIPIcs Volume Portal

Kiefer, Stefan ; Mayr, Richard ; Shirmohammadi, Mahsa ; Totzke, Patrick

Büchi Objectives in Countable MDPs

pdf-format:
LIPIcs-ICALP-2019-119.pdf (0.5 MB)

Abstract

We study countably infinite Markov decision processes with Büchi objectives, which ask to visit a given subset F of states infinitely often. A question left open by T.P. Hill in 1979 [Theodore Preston Hill, 1979] is whether there always exist epsilon-optimal Markov strategies, i.e., strategies that base decisions only on the current state and the number of steps taken so far. We provide a negative answer to this question by constructing a non-trivial counterexample. On the other hand, we show that Markov strategies with only 1 bit of extra memory are sufficient.

BibTeX - Entry

@InProceedings{kiefer_et_al:LIPIcs:2019:10695,
  author =	{Stefan Kiefer and Richard Mayr and Mahsa Shirmohammadi and Patrick Totzke},
  title =	{{B{\"u}chi Objectives in Countable MDPs}},
  booktitle =	{46th International Colloquium on Automata, Languages, and Programming (ICALP 2019)},
  pages =	{119:1--119:14},
  series =	{Leibniz International Proceedings in Informatics (LIPIcs)},
  ISBN =	{978-3-95977-109-2},
  ISSN =	{1868-8969},
  year =	{2019},
  volume =	{132},
  editor =	{Christel Baier and Ioannis Chatzigiannakis and Paola Flocchini and Stefano Leonardi},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{http://drops.dagstuhl.de/opus/volltexte/2019/10695},
  URN =		{urn:nbn:de:0030-drops-106959},
  doi =		{10.4230/LIPIcs.ICALP.2019.119},
  annote =	{Keywords: Markov decision processes}
}

Keywords: Markov decision processes
Collection: 46th International Colloquium on Automata, Languages, and Programming (ICALP 2019)
Issue Date: 2019
Date of publication: 04.07.2019

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI