On a Markov Chain with Denumerable Number of States and Transition Probabilities Dependent on Probability States

Year 2014, Volume 2, 2014, 1 - 11, 26.05.2016

T. M. Aliyev V. M. Mamedov E. A. Ibayev

Abstract

The authors consider homogeneous Markov chain ξt, t ≥ 0 with a denumerable number of states and transition probabilities dependent on the states of that chain. If the chain ξt, t ≥ 0 is assumed to be ergodic for stationary distribution {p ± k } , k ≥ 0 , it is established that a unique solution to the differential equations system relative to the generating functions P ± (θ) , |θ| ≤ 1 of that distribution { p ± k } , k ≥ 0 exists. This condition is found in the form of the inequality ∥G∥ ≤ e 2 . It is based on Fubini’s theorem from the theory of functions and on the existence of the bound G ≡ G∞ = Gn = limn→∞ Eeθ−η , Eis the identity matrix. Using the principle of the matrix theory by induction, we get that

Keywords

Markov chain, stationary distribution, intensity, difference-differential equation, state, unreliable.

References

• T.A. Aliev, A.P. Magerramov. Study of one class of Markov processes based on the given series of random values constituting the Markov chain. (in Russian) News of ANAS, Physics and Engineering, 23(2):85-89,2006.
• K. B. Datta Matrix and Linear Algebra. New Delhi-110001, 2006, p. 636
• A.N. Kolmogorov, S.V. Fomin. Elements of the Theory of Functions and Functional Analysis, Dover Publications, INC.Mineola, York New 1999. p.257
• V. K. Dzyadyk. Approximation Methods for Solutions of Differential and Integral Equations, VSP, 1995: 325 p.