License: Creative Commons Attribution 4.0 International license (CC BY 4.0)
When quoting this document, please refer to the following
DOI: 10.4230/DagSemProc.07461.3
URN: urn:nbn:de:0030-drops-14032
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2008/1403/
Go to the corresponding Portal

Lambert, Joke ; van Houdt, Benny ; Blondia, Chris

A policy iteration algorithm for Markov decision processes skip-free in one direction

pdf-format:
07461.vanHoudtBenny.ExtAbstract.1403.pdf (0.1 MB)

Abstract

In this paper we present a new algorithm for policy iteration for Markov decision processes (MDP) skip-free in one direction. This algorithm, which is based on matrix analytic methods, is in the same spirit as the algorithm of White (Stochastic Models, 21:785-797, 2005) which was limited to matrices that are skip-free in both directions.

Optimization problems that can be solved using Markov decision processes arise in the domain of optical buffers, when trying to improve loss rates of fibre delay line (FDL) buffers. Based on the analysis of such an FDL buffer we present a comparative study between the different techniques available to solve an MDP. The results illustrate that the exploitation of the structure of the transition matrices places us in a position to deal with larger systems, while reducing the computation times.

BibTeX - Entry

@InProceedings{lambert_et_al:DagSemProc.07461.3,
  author =	{Lambert, Joke and van Houdt, Benny and Blondia, Chris},
  title =	{{A policy iteration algorithm for Markov decision processes skip-free in one direction}},
  booktitle =	{Numerical Methods for Structured Markov Chains},
  pages =	{1--3},
  series =	{Dagstuhl Seminar Proceedings (DagSemProc)},
  ISSN =	{1862-4405},
  year =	{2008},
  volume =	{7461},
  editor =	{Dario Bini and Beatrice Meini and Vaidyanathan Ramaswami and Marie-Ange Remiche and Peter Taylor},
  publisher =	{Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{https://drops.dagstuhl.de/opus/volltexte/2008/1403},
  URN =		{urn:nbn:de:0030-drops-14032},
  doi =		{10.4230/DagSemProc.07461.3},
  annote =	{Keywords: Markov Decision Process, Policy Evaluation, Skip-Free, Optical buffers, Fibre Delay Lines}
}

Keywords: Markov Decision Process, Policy Evaluation, Skip-Free, Optical buffers, Fibre Delay Lines
Collection: 07461 - Numerical Methods for Structured Markov Chains
Issue Date: 2008
Date of publication: 07.04.2008

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI