Convergence of a jump procedure in a semi-Markov environment in diffusion-approximation scheme
- PDF / 104,814 Bytes
- 10 Pages / 595 x 842 pts (A4) Page_size
- 21 Downloads / 175 Views
CONVERGENCE OF A JUMP PROCEDURE IN A SEMI-MARKOV ENVIRONMENT IN DIFFUSION-APPROXIMATION SCHEME
UDC 519.7
Ya. M. Chabanyuk
The sufficient convergence conditions are obtained for a jump stochastic approximation procedure in a semi-Markov environment in a diffusion approximation scheme with balance conditions for a singular perturbation of the regression function. To this end, a singular perturbation problem is solved for the asymptotic representation of the compensating operator of an augmented Markov renewal process. Keywords: stochastic approximation, semi-Markov process, compensating operator. INTRODUCTION To derive the sufficient convergence conditions for a discrete stochastic approximation procedure (SAP) [1], the sum invariance principle was applied in [1, 2] to the root u 0 of the regression equation C ( u 0 ) = 0 , the SAP being considered as a Markov process with the associated generating operator. The paper [3] addresses a discrete SAP embedded in a jump SAP such that the regression function C ( u, x ) depends on the environment described by a uniformly ergodic Markov process x( t ) , t ³ 0, in a diffusion approximation scheme with a small parameter e ® 0 , e > 0 . Here, we analyze the convergence of a jump SAP in a semi-Markov environment in a diffusion approximation scheme with the use of asymptotic properties of the compensating operator [4] of an augmented Markov renewal process [5] and a solution of a singular perturbation problem [6] for such an operator. PROBLEM FORMULATION -1
We specify a jump SAP in a diffusion approximation scheme (assuming that u e (t ) = u + e 2
n ( t / e 2 ) -1
å
k=0
å a ke C e ( u ke , xke ) : = 0)
as
k=0
a ke C e ( u ke , x ke ), u e ( 0) = u, t > 0 ,
(1)
where the sequence a ne , n ³ 0 , is defined by values of the function a( t ), t > 0 , using the relationships a ne = a( t en ) , t en = e 2 t n , n ³ 0 ,
(2)
where t n , n ³ 0, are Markov renewal times of a uniformly ergodic semi-Markov process (SMP) x( t ) , t ³ 0, in a standard phase space ( X , X ) with a countable process n ( t ) : = max{ n: t n £ t}, t ³ 0. The relationships (3) u ne = u e ( t ne ) , x ne = x( t en ) , t en := e 2 t n , n ³ 0 , hold in SAP (1), (2). National University “L’vivs’ka Politekhnika,” Lviv, Ukraine, [email protected]. Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 124–133, November–December 2007. Original article submitted June 18, 2007. 866
1060-0396/07/4306-0866
©
2007 Springer Science+Business Media, Inc.
The semi-Markov process x( t ), t > 0, is specified by the semi-Markov kernel [5] Q ( x, B , t ) = P ( x, B ) Gx ( t ) , x Î X , B ÎX, t ³ 0 . The stochastic kernel P ( x, B ) , x Î X , B ÎX, specifies transition probabilities of the embedded Markov chain x n : = x( t n ) , n ³ 0 , P ( x, B ) : = P{x n + 1 Î B | x n = x}, with the sojourn distribution Gx ( t ) , x Î X , t ³ 0, Gx ( x ) : = P{q n + 1 £ t | x n = x}, where q x is sojourn in the state x Î X . With the SMP x( t ), t ³ 0, let us consider an associated Markov process x 0 ( t ), t ³ 0, spec
Data Loading...
