Markov processes using analog crossbar arrays

A computer-implemented method for computing an equilibrium distribution of Markov processes, the method comprising:

• storing weight values in an analog crossbar array of transition probability matrix devices, wherein the weight values in the analog crossbar array of transition probability matrix devices represent a weight matrix with m rows and n columns;
• solving an augmented linear system by solving a stochastic gradient descent-based procedure in which the rows of the weight values of the transition probability matrix devices are used as data samples;
• scaling a solution of the augmented linear system so that a sum of its entries is equal to one;
• determining a probability of changing from one state to another state in a stochastic entity;
• and
• performing a process with an artificial intelligence (AI) model using an AI accelerator chip that employs the equilibrium distribution of Markov processes.

A computer program product for computing an equilibrium distribution of Markov processes, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to:

• store weight values in an analog crossbar array of transition probability matrix devices, wherein the weight values in the analog crossbar array of transition probability matrix devices represent a weight matrix with m rows and n columns;
• compute an eigenvector associated with a real eigenvalue of modulus one for each of the weight values from the transition probability matrix devices;
• apply a gradient-based eigenvalue solver to converge to a dominant eigenpair;
• determine a probability of changing from one state to another state in a stochastic entity based on outcomes of the gradient-based eigenvalue solver;
• and
• performing a process with an artificial intelligence (AI) model using an AI accelerator chip that employs the equilibrium distribution of Markov processes.