Markov decision processes with unbounded transition rates: structural properties of the relative value function

Abstract

In this thesis we have studied Markov decision processes with unbounded transition rates. In order to facilitate the search for optimal policies, we are interested in structural properties of the relative value function of these systems. Properties of interest are for example monotonicity or convexity as a function of the input parameters. This can not be done by the standard mathematical tools, since the systems are not uniformizable. In this study we have examined whether a newly developed method called Smoothed Rate Truncation can overcome this problem.

We introduce how this method is used for a processor sharing queue. We have shown that it can be applied to a system with service control. We have also obtained nice results in the framework of event-based dynamic programming. Due to Smoothed Rate Truncation new operators arise. We have shown that for these operators propagation results, similar to results for existing operators, can be derived. We can conclude that Smoothed Rate Truncation can be used to analyse other processes that have unbounded transition rates.