# US 2021/0276187 A1 · Trajectory optimization using neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0276187 A1
- Document ID: US20210276187A1
- Status: Published application
- Kind: A1
- Application number: 17/193,820
- Filed: 2021-03-05
- Published: 2021-09-09
- Earliest claimed: 2020-03-06
- Assignees: Embodied Intelligence Inc.
- Applicants: Embodied Intelligence Inc.
- Inventors: Haoran Tang; Xi Chen; Yan Duan; Nikhil Mishra; Shiyao Wu; Maximilian Sieb; Yide Shentu
- Web page: https://patentgrep.com/patent/US20210276187A1

## CPC

- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor}
- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/096: Transfer learning
- G05B 2219/40499: Reinforcement learning algorithm
- G05B 2219/40519: Motion, trajectory planning
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/04: Inference or reasoning models
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Various embodiments of the technology described herein generally relate to systems and methods for trajectory optimization with machine learning techniques. More specifically, certain embodiments relate to using neural networks to quickly predict optimized robotic arm trajectories in a variety of scenarios. Systems and methods described herein use deep neural networks to quickly predict optimized robotic arm trajectories according to certain constraints. Optimization, in accordance with some embodiments of the present technology, may include optimizing trajectory geometry and dynamics while satisfying a number of constraints, including staying collision-free, and minimizing the time it takes to complete the task.

## Claims

### Claim 1

A method of training a machine learning model to optimize trajectories, the method comprising:
identifying a task, wherein the task comprises a plurality of requirements;
determining a plurality of trajectories based on the task; and
training the machine learning model using the plurality of trajectories.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the plurality of trajectories based on the task comprises generating a dataset of optimized trajectories using a non-convex optimizer.

### Claim 3 (depends on 2)

The method of claim 2, wherein training the machine learning model using the plurality of trajectories comprises training a neural network model to generate the optimized trajectories of the dataset of optimizes trajectories.

### Claim 4 (depends on 3)

The method of claim 3, wherein training the machine learning model using the plurality of trajectories further comprises refining the machine learning model with imitation loss and trajectory constraint costs until a set of constraints are satisfied and the machine learning model predicts an optimal trajectory.

### Claim 5 (depends on 1)

The method of claim 1, wherein the machine learning model comprises one or more deep artificial neural nets.

### Claim 6 (depends on 5)

The method of claim 5, wherein training the machine learning model comprises training the one or more deep artificial neural nets using one or more of reinforcement learning and imitation learning.

### Claim 7 (depends on 1)

The method of claim 1, wherein the plurality of requirements includes at least:
information related to kinematics and joint geometry of a robotic arm;
a start of an optimal trajectory; and
an end of an optimal trajectory.

### Claim 8 (depends on 1)

The method of claim 1, wherein the plurality of requirements includes at least:
a robotic arm, when performing the task, does not collide with one or more of another object and the robot itself;
motion of the robotic arm is smooth throughout completion of the task;
a set of velocity bounds for the robotic arm; and
a set of acceleration bounds for the robotic arm.

### Claim 9 (depends on 1)

The method of claim 1, wherein training the machine learning model using the plurality of trajectories comprises:
training the machine learning model to imitate the plurality of trajectories; and
refining the machine learning model with imitation loss and trajectory constraint costs until a set of constraints for the task are satisfied and the machine learning model predicts an optimal trajectory of the plurality of trajectories.

### Claim 10

A system comprising:
one or more computer-readable storage media;
a processing system operatively coupled to the one or more computer-readable storage media; and
program instructions, stored on the one or more computer-readable storage media, wherein the program instructions, when read and executed by the processing system, direct the processing system to:
identify a task, wherein the task comprises a plurality of requirements;
determine a plurality of trajectories based on the task; and
train a machine learning model using the plurality of trajectories.

### Claim 11 (depends on 10)

The system of claim 10, wherein to determine the plurality of trajectories based on the task, the program instructions, when read and executed by the processing system, direct the processing system to generate a dataset of optimized trajectories using a non-convex optimizer.

### Claim 12 (depends on 11)

The system of claim 11, wherein to train the machine learning model using the plurality of trajectories, the program instructions, when read and executed by the processing system, direct the processing system to train a neural network model to generate the optimized trajectories of the dataset of optimizes trajectories.

### Claim 13 (depends on 12)

The system of claim 12, wherein to train the machine learning model using the plurality of trajectories, the program instructions, when read and executed by the processing system, further direct the processing system to refine the machine learning model with imitation loss and trajectory constraint costs until a set of constraints are satisfied and the machine learning model predicts an optimal trajectory.

### Claim 14 (depends on 10)

The system of claim 10, wherein the machine learning model comprises one or more deep artificial neural nets.

### Claim 15 (depends on 14)

The system of claim 14, wherein to train the machine learning model using the plurality of trajectories, the program instructions, when read and executed by the processing system, further direct the processing system to train the one or more deep artificial neural nets using one or more of reinforcement learning and imitation learning.

### Claim 16 (depends on 10)

The system of claim 10, wherein the plurality of requirements includes at least:
information related to kinematics and joint geometry of a robotic arm;
a start of an optimal trajectory; and
an end of an optimal trajectory.

### Claim 17 (depends on 10)

The system of claim 10, wherein the plurality of requirements includes at least:
a robotic arm, when performing the task, does not collide with one or more of another object and the robot itself;
motion of the robotic arm is smooth throughout completion of the task;
a set of velocity bounds for the robotic arm; and
a set of acceleration bounds for the robotic arm.

### Claim 18

One or more computer-readable storage media having program instructions stored thereon to train a machine learning model to predict optimal trajectories, wherein the program instructions, when read and executed by a processing system, direct the processing system to at least:
identify a task, wherein the task comprises a plurality of requirements;
determine a plurality of trajectories based on the task; and
train the machine learning model using the plurality of trajectories.

### Claim 19 (depends on 18)

The one or more computer-readable storage media of claim 18, wherein to determine the plurality of trajectories based on the task, the program instructions, when read and executed by the processing system, direct the processing system to generate a dataset of optimized trajectories using a non-convex optimizer.

### Claim 20 (depends on 19)

The one or more computer-readable storage media of claim 19, wherein to train the machine learning model using the plurality of trajectories, the program instructions, when read and executed by the processing system, direct the processing system to train a neural network model to generate the optimized trajectories of the dataset of optimizes trajectories.
