Controller with neural network and improved stability
AssigneesRobert Bosch GmbH, Carnegie Mellon University
InventorsJeremy Zieg Kolter, Melrose Roderick, Priya L. Donti, Julia Vinogradska
Life of this patent
Estimating the term…
Abstract
A controller for generating a control signal for a computer-controlled machine. A neural network may be applied to a current sensor signal, the neural network being configured to map the sensor signal to a raw control signal. A projection function may be applied to the raw control signal to obtain a stable control signal to control the computer-controllable machine.
Claims12
A controller for generating a control signal for a computer-controlled machine, the computer-controlled machine including multiple interacting parts, the controller comprising:
- a sensor input interface configured to receive a sensor signal from a sensor system which senses the computer-controlled machine, the sensor signal indicating a current state of the computer-controlled machine;
- a neural network storage configured for storing a trained neural network, the trained neural network defining a raw control policy for the computer-controlled machine;
- and
- a processor system configured to:
- apply the trained neural network to a current sensor signal, the trained neural network being configured to map the current sensor signal to a raw control signal,
- apply a projection function to the raw control signal to obtain a stable control signal, the projection function mapping a raw control policy space to a stable control policy space of predefined stable control policies, and
- control, using the stable control signal, the computer-controlled machine to change the computer-controlled machine to a desired physical state, wherein the control includes controlling an actuator of the computer-controlled machine to move at least a part of the computer-controlled machine to a different position;
- wherein the projection function is defined as
- π
-
- (
- x
- )
- =
- 𝒫
-
- (
- π
- ^
- (
- x
- )
- )
- =
- (
- π
- ^
- (
- x
- )
- if
-
- ρ
- T
-
- π
- ^
- (
- x
- )
- ≤
- η
- ,
- π
- ^
- (
- x
- )
- -
- ρ
- T
-
- π
- ^
- (
- x
- )
- -
- η
- ρ
- T
-
- ρ
-
- ρ
- otherwise
- for sensor input x, stable control signal π(x), raw control signal {circumflex over (π)}(x), vector ρ and value η.
A controlling method for generating a control signal for a computer-controlled machine, the computer-controlled machine including multiple parts interacting in a dynamical system, the controlling method comprising:
- receiving a sensor signal from a sensor system sensing the computer-controlled machine, the sensor signal indicating a current state of the computer-controlled machine;
- applying a neural network to a current sensor signal, the neural network defining a raw control policy for the computer-controlled machine, the neural network being configured to map the current sensor signal to a raw control signal;
- applying a projection function to the raw control signal to obtain a stable control signal, the projection function mapping a raw control policy space to a stable control policy space of predefined stable control policies;
- and
- controlling, using the stable control signal, the computer-controlled machine to move the computer-controlled machine to a desired physical state, wherein the controlling includes controlling an actuator of the computer-controlled machine to move at least a part of the computer-controlled machine to a different position;
- wherein the projection function is defined as
- π
-
- (
- x
- )
- =
- 𝒫
-
- (
- π
- ^
- (
- x
- )
- )
- =
- (
- π
- ^
- (
- x
- )
- if
-
- ρ
- T
-
- π
- ^
- (
- x
- )
- ≤
- η
- ,
- π
- ^
- (
- x
- )
- -
- ρ
- T
-
- π
- ^
- (
- x
- )
- -
- η
- ρ
- T
-
- ρ
-
- ρ
- otherwise
- for sensor input x, stable control signal π(x), raw control signal {circumflex over (π)}(x), vector ρ and value η.
A non-transitory computer readable medium comprising stored instructions that generate a control signal for a computer-controlled machine that includes multiple parts interacting in a dynamical system, the instructions, when executed by a processor system, cause the processor system to perform:
- receiving a sensor signal from a sensor system sensing the computer-controlled machine, the sensor signal indicating a current state of the computer-controlled machine;
- applying a neural network to a current sensor signal, the neural network defining a raw control policy for the computer-controlled machine, the neural network being configured to map the current sensor signal to a raw control signal;
- applying a projection function to the raw control signal to obtain a stable control signal, the projection function mapping a raw control policy space to a stable control policy space of predefined stable control policies;
- and
- controlling, using the stable control signal, the computer-controlled machine the computer-controlled machine to a desired physical state, wherein the controlling includes controlling an actuator of the computer-controlled machine to move at least a part of the computer-controlled machine to a different position;
- wherein the projection function is defined as
- π
-
- (
- x
- )
- =
- 𝒫
-
- (
- π
- ^
- (
- x
- )
- )
- =
- (
- π
- ^
- (
- x
- )
- if
-
- ρ
- T
-
- π
- ^
- (
- x
- )
- ≤
- η
- ,
- π
- ^
- (
- x
- )
- -
- ρ
- T
-
- π
- ^
- (
- x
- )
- -
- η
- ρ
- T
-
- ρ
-
- ρ
- otherwise
- for sensor input x, stable control signal π(x), raw control signal {circumflex over (π)}(x), vector ρ and value η.
Dependencies are inferred from the claim text.