# US 12,179,363 B2 · Trajectory optimization using neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,179,363 B2
- Document ID: US12179363B2
- Status: Granted
- Kind: B2
- Application number: 17/193,820
- Filed: 2021-03-05
- Published: 2021-09-09
- Granted: 2024-12-31
- Earliest claimed: 2020-03-06
- Assignees: Embodied Intelligence Inc.
- Applicants: Embodied Intelligence Inc.
- Inventors: Haoran Tang; Xi Chen; Yan Duan; Nikhil Mishra; Shiyao Wu; Maximilian Sieb; Yide Shentu
- Web page: https://patentgrep.com/patent/US12179363B2

## CPC

- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor}
- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/096: Transfer learning
- G05B 2219/40499: Reinforcement learning algorithm
- G05B 2219/40519: Motion, trajectory planning
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/04: Inference or reasoning models
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Various embodiments of the technology described herein generally relate to systems and methods for trajectory optimization with machine learning techniques. More specifically, certain embodiments relate to using neural networks to quickly predict optimized robotic arm trajectories in a variety of scenarios. Systems and methods described herein use deep neural networks to quickly predict optimized robotic arm trajectories according to certain constraints. Optimization, in accordance with some embodiments of the present technology, may include optimizing trajectory geometry and dynamics while satisfying a number of constraints, including staying collision-free, and minimizing the time it takes to complete the task.

## Claims

### Claim 1

A method of training a machine learning model to optimize trajectories, the method comprising:
identifying a task for a robotic arm, wherein the task comprises moving an object from a starting point to an end point with the robotic arm and includes a plurality of requirements and information related to the robotic arm, wherein the information related to the robotic arm includes at least geometric information of the robotic arm and a gripper of the robotic arm;
computing a plurality of trajectories for the robotic arm to move the object from the starting point to the end point based on the task using a non-convex optimizer; and
training the machine learning model using the plurality of trajectories, wherein training the machine learning model comprises refining the machine learning model with imitation loss and trajectory constraint costs until the plurality of requirements are satisfied and the machine learning model predicts an optimal trajectory for moving the object from the starting point to the end point, and wherein training the machine learning model further comprises training the machine learning model using reinforcement learning.

### Claim 2 (depends on 1)

The method of claim 1, wherein computing the plurality of trajectories based on the task comprises generating a dataset of optimized trajectories using the non-convex optimizer.

### Claim 3 (depends on 2)

The method of claim 2, wherein training the machine learning model using the plurality of trajectories comprises training a neural network model to generate the optimized trajectories of the dataset of optimized trajectories.

### Claim 4 (depends on 1)

The method of claim 1, wherein the information related to the robotic arm further includes geometric information of a suction cup of the gripper and positional information of the robotic arm relative to the object with which the robotic arm is to interact.

### Claim 5 (depends on 1)

The method of claim 1, wherein the machine learning model comprises one or more deep artificial neural nets.

### Claim 6 (depends on 5)

The method of claim 5, wherein training the machine learning model comprises training the one or more deep artificial neural nets using reinforcement learning.

### Claim 7 (depends on 1)

The method of claim 1, wherein the plurality of requirements includes a time to complete the task.

### Claim 8 (depends on 1)

The method of claim 1, wherein the plurality of requirements includes at least:
the robotic arm, when performing the task, does not collide with one or more of another object and the robotic arm;
motion of the robotic arm is smooth throughout completion of the task;
a set of velocity bounds for the robotic arm; and
a set of acceleration bounds for the robotic arm.

### Claim 9 (depends on 1)

The method of claim 1, wherein training the machine learning model using the plurality of trajectories comprises training the machine learning model to imitate the plurality of trajectories.

### Claim 10

A system comprising:
one or more computer-readable storage media;
a processing system operatively coupled to the one or more computer-readable storage media; and
program instructions, stored on the one or more computer-readable storage media, wherein the program instructions, when read and executed by the processing system, direct the processing system to:
identify a task for a robotic arm, wherein the task comprises moving an object from a starting point to an end point with the robotic arm and includes a plurality of requirements and information related to the robotic arm, wherein the information related to the robotic arm includes at least geometric information of the robotic arm and a gripper of the robotic arm;
compute a plurality of trajectories for the robotic arm to move the object from the starting point to the end point based on the task using a non-convex optimizer; and
train a machine learning model using the plurality of trajectories, wherein training the machine learning model comprises refining the machine learning model with imitation loss and trajectory constraint costs until the plurality of requirements are satisfied and the machine learning model predicts an optimal trajectory for moving the object from the starting point to the end point, and wherein training the machine learning model further comprises training the machine learning model using reinforcement learning.

### Claim 11 (depends on 10)

The system of claim 10, wherein to compute the plurality of trajectories based on the task, the program instructions, when read and executed by the processing system, direct the processing system to generate a dataset of optimized trajectories using the non-convex optimizer.

### Claim 12 (depends on 11)

The system of claim 11, wherein to train the machine learning model using the plurality of trajectories, the program instructions, when read and executed by the processing system, direct the processing system to train a neural network model to generate the optimized trajectories of the dataset of optimized trajectories.

### Claim 13 (depends on 10)

The system of claim 10, wherein the information related to the robotic arm further includes geometric information of a suction cup of the gripper and positional information of the robotic arm relative to the object with which the robotic arm is to interact.

### Claim 14 (depends on 10)

The system of claim 10, wherein the machine learning model comprises one or more deep artificial neural nets.

### Claim 15 (depends on 10)

The system of claim 10, wherein the plurality of requirements includes a time to complete the task.

### Claim 16 (depends on 10)

The system of claim 10, wherein the plurality of requirements includes at least:
the robotic arm, when performing the task, does not collide with one or more of another object and the robotic arm;
motion of the robotic arm is smooth throughout completion of the task;
a set of velocity bounds for the robotic arm; and
a set of acceleration bounds for the robotic arm.

### Claim 17

One or more computer-readable storage media having program instructions stored thereon to train a machine learning model to predict optimal trajectories, wherein the program instructions, when read and executed by a processing system, direct the processing system to at least:
identify a task for a robotic arm, wherein the task comprises moving an object from a starting point to an end point with the robotic arm and includes a plurality of requirements and information related to the robotic arm, wherein the information related to the robotic arm includes at least geometric information of the robotic arm and a gripper of the robotic arm;
compute a plurality of trajectories for the robotic arm to move the object from the starting point to the end point based on the task using a non-convex optimizer; and
train a machine learning model using the plurality of trajectories, wherein training the machine learning model comprises refining the machine learning model with imitation loss and trajectory constraint costs until the plurality of requirements are satisfied and the machine learning model predicts an optimal trajectory for moving the object from the starting point to the end point, and wherein training the machine learning model further comprises training the machine learning model using reinforcement learning.

### Claim 18 (depends on 17)

The one or more computer-readable storage media of claim 17, wherein to compute the plurality of trajectories based on the task, the program instructions, when read and executed by the processing system, direct the processing system to generate a dataset of optimized trajectories using the non-convex optimizer.

### Claim 19 (depends on 18)

The one or more computer-readable storage media of claim 18, wherein to train the machine learning model using the plurality of trajectories, the program instructions, when read and executed by the processing system, direct the processing system to train a neural network model to generate the optimized trajectories of the dataset of optimized trajectories.
