马尔科夫过程描述的是空间状态经过一个状态到另一个状态转换的随机过程。该过程要求具备“无记忆”性质，也就是下一状态的概率分布只与当前的状态有关，再时间序列中其他前面的事件无关。

马尔科夫随机序列由元组<S,P>表示。 S 代表有限状态集，P 是状态转移概率矩阵。公式,描述了一个状态转移到另一个状态发生的概率。

本文主要以代码描述MP过程。若要详细看图文例子，可参考知乎https://zhuanlan.zhihu.com/p/35124726

import numpy as np
state = ["sleep","icecream","run"]
transitionName =[["ss","si","sr"],["is","ii","ir"],["rs","ri","rr"]]
transitionMatrix =[[0.2,0.6,0.2],[0.1,0.6,0.3],[0.2,0.7,0.1]]

def activity_forecast(days):
    activitytoday= "sleep"
    print("start state",activitytoday)
    activityList = [activitytoday]
    i =0
    prob =1
    while i < days :
        if activitytoday =="sleep":
            change = np.random.choice(transitionName[0],replace=True,p=transitionMatrix[0])
            if change =='ss':
                prob *= transitionMatrix[0][0]
                activityList.append("sleep")

            elif change == 'si':
                prob *= transitionMatrix[0][1]
                activityList.append("icecream")
                activitytoday ='icecream'
            else :
                prob *= transitionMatrix[0][2]
                activityList.append("run")
                activitytoday='run'

        elif activitytoday == "icecream":
            change = np.random.choice(transitionName[1], replace=True, p=transitionMatrix[1])
            if change == 'is':
                prob *= transitionMatrix[1][0]
                activityList.append("sleep")
                activitytoday = 'sleep'
            elif change == 'ii':
                prob *= transitionMatrix[1][1]
                activityList.append("icecream")
            else:
                prob *= transitionMatrix[1][2]
                activityList.append("run")
                activitytoday = 'run'

        else:
            change = np.random.choice(transitionName[2], replace=True, p=transitionMatrix[2])
            if change == 'rs':
                prob *= transitionMatrix[2][0]
                activityList.append("sleep")
                activitytoday = 'sleep'
            elif change == 'ri':
                prob *= transitionMatrix[2][1]
                activityList.append("icecream")
                activitytoday = 'icecream'
            else:
                prob *= transitionMatrix[2][2]
                activityList.append("run")
        i+=1
        print(" after " + str(i) + " days: " + activitytoday)
        print("Possible states: " + str(activityList))
        print("Probability of the possible sequence of states: " + str(prob))
activity_forecast(2)

运行结果 ：

markov.png