Discrete-Space Markov Processes
A rather detailed study of Markov processes with discrete state space is provided. It focuses on sample path techniques in a perspective inspired by simulation needs. The relationship of these processes with Poisson processes and with discrete-time Markov
- PDF / 453,806 Bytes
- 31 Pages / 441 x 666 pts Page_size
- 83 Downloads / 256 Views
Discrete-Space Markov Processes
Abstract A rather detailed study of Markov processes with discrete state space is provided. It focuses on sample path techniques in a perspective inspired by simulation needs. The relationship of these processes with Poisson processes and with discrete-time Markov chains is shown. Rigorous constructions and results are provided for Markov process with uniformly bounded jump rates. To this end, elements of the theory of bounded operators are introduced, which explain the relation between generator and semigroup, and provide a useful framework for the forward and backward Kolmogorov equations and the Feynman–Kac formula.
Anderson [1] is a reference book on the topic, with a much wider scope than here. Asmussen and Glynn [4] provides a short primer on the theory, and many queueing examples.
5.1 Characterization, Specification, Properties We start with some general properties of a Markov process assuming that it exists. Measure theory on a discrete space reduces to series summation, and we give the definitions in this framework.
5.1.1 Measures, Functions, and Transition Matrices Positive and Signed Measures, Integrals of Functions For μ = (μ(x))x∈V such that μ(x) ∈ R+ , let μ(A) := μ(x) ∈ R+ ∪ {∞} = [0, ∞],
A⊂V ,
x∈A
be defined in the sense of positive series. Such a set-function is called a measure on V , and sometimes a positive measure for precision. The measure μ is said to be finite if its total mass μ(V ) is finite, and to be a probability measure, or a law, if μ(V ) = 1. The space of probability measures is denoted by P = P(V ). C. Graham, D. Talay, Stochastic Simulation and Monte Carlo Methods, Stochastic Modelling and Applied Probability 68, DOI 10.1007/978-3-642-39363-1_5, © Springer-Verlag Berlin Heidelberg 2013
89
90
5 Discrete-Space Markov Processes
For μ = (μ(x))x∈V with μ(x) ∈ R such that μ(A) :=
μ(x) ∈ R,
x∈V
|μ(x)| < ∞, let
A⊂V ,
x∈A
be defined in the sense of absolutely convergent series. Such a set-function is called a signed measure on V . The vector space of signed measures is denoted by M = M (V ). Among positive measures, only finite ones belong to M . A duality bracket between measures μ and functions f is given by, (μ, f ) → μ, f := μ(x)f (x), (5.1) x∈V
in the sense of positive series if μ and f are positive (wide sense), and in the sense of absolutely convergent series if μ belongs to M and f to the Banach space of bounded functions L∞ = L∞ (V ).
Markovian Matrices and Their Actions, Line and Column Vectors In the following, classic matrix multiplication conventions are used. A positive or signed measure μ on V is considered as a line vector (μ(x))x∈V , a function f on V as a column vector (f (x))x∈V , both of infinite length if Card(V ) = ∞, and their matrix product corresponds to the above duality bracket ⎛ ⎞ .. . ⎟ ⎜ ⎟ μf = · · · μ(x) · · · ⎜ μ(x)f (x) = μ, f . (5.2) ⎝f (x)⎠ = .. x∈V . A matrix P is said to be a transition or Markovian matrix if P (x, y) ≥ 0, P (x, y) = 1, P = P (x, y) x,y∈V , y∈V
i.e., if its line
Data Loading...
