Thread: Understanding Markov Chains. Calculating Probabilities from data

Hello,
I have recently discovered the delights of Markov chains and become very interested in them, however everything I have learnt is from reading different texts and I would like to clarify these things by moving away from the mathematical proofs (which is not my background) and into a context that I think would enable me to understand the concept of Markov Chains more thoroughly.

Because the Markov examples I have seen always give you the probabilities (in a math text book) how, given some data, would you calculate the probabilities contained within the transition matrix. Attached is a spreadsheet giving such data. The data describes movement in to (hire) and out of(fired) of a company, as well as individual employee’s “Job Types” over four months.

Consider the current state to be the month of April. From this data I estimate that the transition matrix should have 6 states: Hire, A, B, C, D, and Fired. . How should I and would I populate the probability of the movements between each of the states in the transition matrix? I wouldn’t know where to begin. Any suggestions?
I do have a lot more questions, but I suppose I cannot really ask those until the transition matrix is out

Any help would be greatly appreciated.
Kind regards,
JavaJunkie

From what I've read, if the data is time-homogeneous then I could take the Maximum Likelihood Estimate which is to count, for each row, how many transitions from state i to state j occur in the dataset, and then normalise the rows so they sum to 1.

But not having a mathematical background hinders my ability to do this on my own.

What is time-homogeneous data? and could someone explain what the Maximum Likelihood Estimate is?