US 6,665,651 B2 · Control system and technique employing reinforcement learning having stability and learning phases

Abstract

A feedback control system for automatic on-line training of a controller for a plant, the system having a reinforcement learning agent connected in parallel with the controller. The learning agent comprises an actor network and a critic network operatively arranged to carry out at least one sequence of a stability phase followed by a learning phase. During the stability phase, a multi-dimensional boundary of values is determined. During the learning phase, a plurality of updated weight values is generated in connection with the on-line training, if and until one of the updated weight values reaches the boundary, at which time a next sequence is carried out to determine a next multi-dimensional boundary of values followed by a next learning phase. Also, a method for automatic on-line training of a feedback controller within a system comprising the controller and a plant by employing a reinforcement learning agent comprising a neural network to carry out at least one sequence comprising a stability phase followed by a learning phase. Further included, a computer executable program code on a computer readable storage medium, for on-line training of a feedback controller within a system comprising the controller and a plant.

Claims20

Dependent claims

A feedback control system for automatic on-line training of a controller for a plant, comprising:

a reinforcement learning agent connected in parallel with the controller;
the learning agent comprising an actor network and a critic network operatively arranged to carry out at least one sequence of a stability phase followed by a learning phase;
and
wherein said stability phase comprises determining a multi-dimensional boundary of values, and said learning phase comprises generation of a plurality of updated weight values in connection with the on-line training, if and until one of said updated weight values reaches said boundary, at which time a next sequence is carried out comprising determining a next multi-dimensional boundary of values followed by a next learning phase.

1 more element

A method for automatic on-line training of a feedback controller within a system comprising the controller and a plant, the method comprising the steps of:

employing a reinforcement learning agent comprising a neural network to carry out at least one sequence comprising a stability phase followed by a learning phase;
said stability phase comprising the step of determining a multi-dimensional boundary of neural network weight values for which the system's stability can be maintained;
said learning phase comprising the step of generating a plurality of updated weight values in connection with the on-line training;
and
if, during said learning phase, one of said updated weight values reaches said boundary, carrying out a next sequence comprising the step of determining a next multi-dimensional boundary of weight values followed by a next learning phase.

2 more elements

A computer executable program code on a computer readable storage medium, for on-line training of a feedback controller within a system comprising the controller and a plant, the program code comprising:

a first program sub-code for initializing input and output weight values, respectively, Wi and Vi, of a neural network;
a second program sub-code for instructing a reinforcement agent, comprising said neural network and a critic network, operatively arranged in parallel with the controller, to carry out a stability phase comprising determining a multi-dimensional boundary of neural network weight values for which the system's stability can be maintained;
and
a third program sub-code for instructing said reinforcement agent to carry out a learning phase comprising generating a plurality of updated weight values in connection with the on-line training if and until any one of said updated weight values reaches said boundary, then instructing said reinforcement agent to carry out a next sequence comprising determining a next multi-dimensional boundary of weight values followed by a next learning phase.

1 more element

Dependencies are inferred from the claim text.

Control system and technique employing reinforcement learning having stability and learning phases

Life of this patent

Control system and technique employing reinforcement learning having stability and learning phases

Life of this patent

Abstract

Claims20