Method for converting dynamical systems with continuous states into Markov decision processes with discrete states

A method for converting a continuous system to a Markov decision process (MDP), wherein the continuous system is dynamical and the MDP has discrete states, comprising the steps of:

• selecting a predetermined number of continuous states of the continuous system, wherein each continuous state corresponds to one discrete state of the MDP;
• and
• applying Delaunay triangulation to the continuous states to produce a set of triangles, wherein vertices of each triangle represent the continuous states, and further comprising for each discrete state;
• determining a next discrete state y=ƒ(x,a), wherein x represents the continuous state corresponding to the discrete state, a is a control action, and ƒ is a non-linear transition function for the continuous;
• identifying a particular triangle containing the next discrete state y;
• and
• expressing the next discrete state y as probabilities of transitioning to the discrete states corresponding to the continuous states x represented by the vertices of the particular triangle, where identifying the particular triangle is performed by means of forming a d×d difference matrix E, such that column j of the difference matrix E contains a differences vm,j−vm.d+1, for j=1, d, between an jth vertex vm.j of triangle m and a d+1st vertex q=vm.d+1 of the triangle m, and solving a linear system of equations Ec=(y−q), for a vector c, and wherein the triangle m is identified as containing y if all cj>0, j=1, d, and cd+1=1−Σj=1Dcj>0, wherein the steps are performed in a processor.