Math Is Fun Forum

The question:

Suppose an irreducible Markov chain on a state space S with m states in generated by the probability transition matrix (PTM) P. Show that, for any two states i and j, P(i,j)^k >0 for some k with 0<=k<m. In other words, you can get from state i to state j in fewer than m steps. ( Note if i=j then P(i,j)^0=P(i,i)^0=I(i,i)=1 >, and we may take k=0)

I know that I need to use some properties of Markov chain, but I don't really see where I need to start.
would appreciate any help.

Thanks.