Math Is Fun Forum

Ramze

Member

Registered: 2010-06-21

Posts: 2

Markov chain versus Markov Process

Dear Friends,

I was wondering how can I differ between a markov chain and markov process with some examples if possible. I was working in building a formulation for rain occurence probability (what is the probability of raining today if it rained yesterday, or what are the other option such as day 1: rain, day 2: no rain, day 1: no rain, day 2: rain...etc)...As I read this can be solved using 1st order markov chain...what about other orders (2nd or higher)? How this can be soved in this case?
I know it could be a lot of questions, but I believe this may clarify a lot for me and others who are confused regarding this process...Thank you in advance for your help.

Ramze

bobbym

bumpkin

From: Bumpkinland

Registered: 2009-04-12

Posts: 109,606

Re: Markov chain versus Markov Process

Hi Ramze;

A Markov chain is a discrete process in which future events are determined by the state of a process today, a Markov process is the continuous version.

Now what the heck does that mean? Good point! For the sake of working with them you may never know. The good part is, you don't need to. Let me ask you some questions.

!) What is the name of your textbook?
2) Have you had matrices and do you know the basic matrix operations. Cause that is what me and everyone else who is interested in using them to actually solve a problem, uses.
3) Are you just interested in the jargon or in problem solving with them?

Examples:

http://www.mathisfunforum.com/viewtopic … 31#p118031 Post#1, Post#5

http://www.mathisfunforum.com/viewtopic … 68#p118268 Post#5, Post#7

http://www.mathisfunforum.com/viewtopic … 50#p109950 Post#6

http://www.mathisfunforum.com/viewtopic … 82#p110182 Post#2

That is how they are used in small problems. The larger ones are matrices of nxn where n > 100.
Markov chains can solve tree problems and problems with replacement. They are related to recurrences, sums, trees, lattices, graphs and generating functions as pertaining to probability problems.

If you are just interested in knowing when it is doubly stochastic, regular, stochastic, absorbing, in other words the terminology. Then your textbook and the internet (Google) is the best for that. I can't do any better! You have no choice but to swallow the jargon, understand the proofs and memorize them.

If you are interested in problems and their solution using regular or absorbing markov chains like your rain problem. Then please post your problem. Start with your book problems and post them. I will do my best.

I will show you how to do them using simultaneous equations and matrix operations. Later on when you are exposed to computers you will find it very useful to know how to solve them in that manner.

May I recommenf Finite Mathematics 4th ed ... 9th edition by Mizrahi and Sullivan. Go through that and you will be an expert. Not really an expert, just everyone will think you are.

Welcome to the forum!

---

In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.

Ramze

Member

Registered: 2010-06-21

Posts: 2

Re: Markov chain versus Markov Process

Thanks a lot for the info..I will grab the Finite Mathematics textbook and go from there...

bobbym

bumpkin

From: Bumpkinland

Registered: 2009-04-12

Posts: 109,606

Re: Markov chain versus Markov Process

Hi Ramze;

I have the 4th edition that I picked up at a library sale for 50 cents. Someone told me that they were able to get the 9th edition for the same price on ebay. Good luck and bring your questions back to me and we will work them out together.

---

In mathematics, you don't understand things. You just get used to them.
If it ain't broke, fix it until it is.
Always satisfy the Prime Directive of getting the right answer above all else.