# US 2025/0165732 A1 · Nervous-system emulator for learning with a robot, and associated methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165732 A1
- Document ID: US20250165732A1
- Status: Published application
- Kind: A1
- Application number: 19/032,737
- Filed: 2025-01-21
- Published: 2025-05-22
- Earliest claimed: 2017-09-01
- Inventors: Joseph William Barter
- Web page: https://patentgrep.com/patent/US20250165732A1

## CPC

- B25J 9/1679: {characterised by the tasks executed}
- G06G 7/60: for living beings, e.g. their nervous systems {; for problems in the medical field}
- B25J 9/1605: {Simulation of manipulator lay-out, design, modelling of manipulator}
- B25J 9/1694: {characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion}
- G06N 3/004: Artificial life, i.e. computing arrangements simulating life
- G06N 3/008: based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 7/08: using chaos models or non-linear system models
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/092: Reinforcement learning
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06G: ANALOGUE COMPUTERS (analogue optical computing devices G06E3/00)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A nervous-system emulator engine includes working computational models of the vertebrate nervous system to generate lifelike animal behavior in a robot. These models include functions representing several anatomical features of the vertebrate nervous system, such as spinal cord, brainstem, basal ganglia, thalamus, and cortex. The emulator engine includes a hierarchy of controllers in which controllers at higher levels accomplish goals by continuously specifying desired goals for lower-level controllers. The lowest levels of the hierarchy reflect spinal cord circuits that control muscle tension and length. Moving up the hierarchy into the brainstem and midbrain/cortex, progressively more abstract perceptual variables are controlled. The nervous-system emulator engine may be used to build a robot that generates the majority of animal behavior, including human behavior. The nervous-system emulator engine may also be used to build working models of nervous system functions for clinical experimentation.

## Claims

### Claim 1

A method for learning with a nervous-system emulator of a robot, comprising:
controlling the robot with a plurality of lower-level reference signals such that the robot moves to an initial state;
detecting a perceptual signal from the robot while the robot is in the initial state;
comparing the perceptual signal to a top-level reference signal to generate a most-recent value of an error signal;
comparing the most-recent value of the error signal to a previous value of the error signal to generate an error-velocity value; and
updating each lower-level reference signal of the plurality of lower-level reference signals by:
generating a next step size by (i) reducing a previous step size of the lower-level reference signal if the error-velocity value is less than zero or (ii) increasing the previous step size of the lower-level reference signal if the error-velocity value is greater than zero;
generating a next step direction by (i) setting the next step direction equal to a previous step direction of the lower-level reference signal if the error-velocity value is less than zero or (ii) randomly selecting the next step direction if the error-velocity value is greater than zero; and
adjusting the lower-level reference signal based on both the next step size and the next step direction;
wherein the robot, after said updating, moves to a subsequent state that is different from the initial state.

### Claim 2 (depends on 1)

The method of claim 1, wherein said adjusting comprises adding a next step value to the lower-level reference signal, the next step value being based on both the next step size and the next step direction.

### Claim 3 (depends on 1)

The method of claim 1, wherein said adjusting comprises adding a next step value to a gain value used to transform the error signal into the lower-level reference signal, the next step value being based on both the next step size and the next step direction.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
the gain value is one of a plurality of gain values in one-to-one correspondence with the plurality of lower-level reference signals, each of the plurality of gain values being used to transform the error signal into the corresponding one of the plurality of lower-level reference signals; and
the method further comprises storing the plurality of gain values in a memory as a plurality of saved gain values.

### Claim 5 (depends on 3)

The method of claim 3, wherein:
the gain value is one of a plurality of gain values in one-to-one correspondence with the plurality of lower-level reference signals, each of the plurality of gain values being used to transform the error signal into the corresponding one of the plurality of lower-level reference signals; and
the method further comprises initializing the plurality of gain values by:
retrieving, from a memory, a plurality of saved gain values; and
setting each of the plurality of gain values equal to a respective one of the plurality of saved gain values.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
the method further comprises repeating said controlling the robot, said detecting the perceptual signal, said comparing the perceptual signal, said comparing the most-recent value of the error signal, and said updating the lower-level reference signal over a plurality of iterations; and
the subsequent state for one of the plurality of iterations is the initial state for a next one of the plurality of iterations.

### Claim 7 (depends on 1)

The method of claim 1, wherein said updating the lower-level reference signal occurs in response to the most-recent value of the error signal being greater than an error threshold.

### Claim 8 (depends on 1)

The method of claim 1, further comprising initializing one or both of the previous step size and the previous step direction.

### Claim 9 (depends on 1)

The method of claim 1, further comprising initializing the plurality of lower-level reference signals prior to said controlling the robot.

### Claim 10 (depends on 9)

The method of claim 9, wherein said initializing the plurality of lower-level reference signals comprises:
retrieving, from a memory, a plurality of saved reference-signal values; and
generating the plurality of lower-level reference signals based on the plurality of saved reference-signal values.

### Claim 11 (depends on 1)

The method of claim 1, further comprising storing, in a memory, a plurality of saved reference-signal values of the plurality of lower-level reference signals.

### Claim 12 (depends on 1)

The method of claim 1, wherein:
said controlling the robot, said detecting the perceptual signal, said comparing the perceptual signal, said comparing the most-recent value of the error signal, and said updating the lower-level reference signal are performed to learn one step of a sequence of steps; and
the method further comprises repeating said controlling the robot, said detecting the perceptual signal, said comparing the perceptual signal, said comparing the most-recent value of the error signal, and said updating to learn each other step of the sequence of steps.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
omitting one step of the sequence of steps to create an abbreviated sequence of steps; and
repeating said controlling the robot, said detecting the perceptual signal, said comparing the perceptual signal, said comparing the most-recent value of the error signal, and said updating the lower-level reference signal for each step of the abbreviated sequence of steps.

### Claim 14

A nervous-system emulator for learning with a robot, comprising a learning circuit configured to:
control the robot with a plurality of lower-level reference signals such that the robot moves to an initial state;
detect a perceptual signal from the robot while the robot is in the initial state;
compare the perceptual signal to a top-level reference signal to generate a most-recent value of an error signal;
compare the most-recent value of the error signal to a previous value of the error signal to generate an error-velocity value; and
update each lower-level reference signal of the plurality of lower-level reference signals by:
generating a next step size by (i) reducing a previous step size of the lower-level reference signal if the error-velocity value is less than zero or (ii) increasing the previous step size of the lower-level reference signal if the error-velocity value is greater than zero;
generating a next step direction by (i) setting the next step direction equal to a previous step direction of the lower-level reference signal if the error-velocity value is less than zero or (ii) randomly selecting the next step direction if the error-velocity value is greater than zero; and
adjusting the lower-level reference signal based on both the next step size and the next step direction;
wherein the robot, after the lower-level reference signal is updated, moves to a subsequent state that is different from the initial state.

### Claim 15 (depends on 14)

The nervous-system emulator of claim 14, the learning circuit being configured to adjust the lower-level reference signal by adding a next step value to the lower-level reference signal, the next step value being based on both the next step size and the next step direction.

### Claim 16 (depends on 14)

The nervous-system emulator of claim 14, the learning circuit being configured to adjust the lower-level reference signal by adding a next step value to a gain value used to transform the error signal into the lower-level reference signal, the next step value being based on both the next step size and the next step direction.

### Claim 17 (depends on 14)

The nervous-system emulator of claim 14, the learning circuit comprising:
a processor; and
a memory in electronic communication with the processor;
the memory storing machine-readable instructions that, when executed by the processor, control the learning circuit to control the robot, detect the perceptual signal, compare the perceptual signal, compare the most-recent value of the error signal, and update the lower-level reference signal.

### Claim 18 (depends on 14)

The nervous-system emulator of claim 14, the learning circuit comprising a programmable logic device, a field-programmable gate array (FPGA), a system-on-chip (SoC), an application-specific integrated circuit (ASIC), or a combination thereof.

### Claim 19 (depends on 14)

The nervous-system emulator of claim 14, further comprising a plurality of controllers configured to control the robot based on the plurality of lower-level reference signals.

### Claim 20 (depends on 14)

The nervous-system emulator of claim 14, further comprising the robot.

### Claim 21 (depends on 14)

The nervous-system emulator of claim 14, further comprising a perceptual circuit configured to transform one or more sensor signals into the perceptual signal.

### Claim 22 (depends on 21)

The nervous-system emulator of claim 21, further comprising one or more sensors configured to generate the one or more sensor signals.
