# US 2021/0276188 A1 · Trajectory optimization using neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0276188 A1
- Document ID: US20210276188A1
- Status: Published application
- Kind: A1
- Application number: 17/193,870
- Filed: 2021-03-05
- Published: 2021-09-09
- Earliest claimed: 2020-03-06
- Assignees: Embodied Intelligence Inc.
- Applicants: Embodied Intelligence Inc.
- Inventors: Haoran Tang; Xi Chen; Yan Duan; Nikhil Mishra; Shiyao Wu; Maximilian Sieb; Yide Shentu
- Web page: https://patentgrep.com/patent/US20210276188A1

## CPC

- B25J 9/1666: {Avoiding collision or forbidden zones}
- B25J 9/1605: {Simulation of manipulator lay-out, design, modelling of manipulator}
- B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor}
- B25J 9/163: {learning, adaptive, model based, rule based expert control}
- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B65G 61/00: Use of pick-up or transfer devices or of manipulators for stacking or de-stacking articles not otherwise provided for
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- B25J 5/007: {mounted on wheels}
- B25J 9/1697: {Vision controlled systems}
- G05B 2219/40499: Reinforcement learning algorithm
- G05B 2219/40519: Motion, trajectory planning
- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G05D 1/227: Handing over between remote control and on-board control; Handing over between remote control arrangements
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- B65G: TRANSPORT OR STORAGE DEVICES, e.g. CONVEYORS FOR LOADING OR TIPPING, SHOP CONVEYOR SYSTEMS OR PNEUMATIC TUBE CONVEYORS (packaging B65B; handling thin or filamentary materials, e.g. paper sheets or thread, B65H; cranes B66C; portable mobile or lifting or hauling appliances, e.g. hoists, B66D; devices for lifting or lowering goods for loading or unloading purposes, e.g. fork-lift trucks, B66F9/00; emptying bottles, jars, casks, barrels or similar containers, not otherwise provided for, B67C9/00; delivering or transferring liquids B67D; filling or discharging vessels for liquefied, solidified or compressed gases F17C; pipe-line systems for fluids F17D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES

## Abstract

Various embodiments of the technology described herein generally relate to systems and methods for trajectory optimization with machine learning techniques. More specifically, certain embodiments relate to using neural networks to quickly predict optimized robotic arm trajectories in a variety of scenarios. Systems and methods described herein use deep neural networks to quickly predict optimized robotic arm trajectories according to certain constraints. Optimization, in accordance with some embodiments of the present technology, may include optimizing trajectory geometry and dynamics while satisfying a number of constraints, including staying collision-free and minimizing the time it takes to complete the task.

## Claims

### Claim 1

A trajectory optimization method for a robotic system, the method comprising:
identifying a task for the robotic system, wherein the task comprises one or more requirements;
providing one or more details related to the task and one or more details related to the robotic system to a machine learning model, wherein the machine learning model determines an optimal trajectory for the robotic system to follow when performing the task and wherein the optimal trajectory satisfies the one or more requirements and minimizes an amount of time it takes the robotic system to successfully complete the task;
receiving an output from the machine learning model, wherein the output comprises the optimal trajectory; and
directing the robotic system to complete the task via the optimal trajectory.

### Claim 2 (depends on 1)

The method of claim 1, wherein the machine learning model comprises one or more deep artificial neural nets.

### Claim 3 (depends on 2)

The method of claim 2, wherein the machine learning model is trained using one or more of reinforcement learning and imitation learning.

### Claim 4 (depends on 1)

The method of claim 1, wherein the details related to the task include at least:
information related to kinematics and geometry of the robotic system;
a start of the optimal trajectory; and
an end of the optimal trajectory.

### Claim 5 (depends on 1)

The method of claim 1, wherein the one or more requirements include at least:
the robotic system does not collide with any objects;
motion of the robotic system is smooth throughout the optimal trajectory;
velocity of the robotic system remain below a maximum velocity; and
acceleration of the robotic system remain below a maximum acceleration.

### Claim 6 (depends on 1)

The method of claim 1, wherein the robotic system comprises one or more of: a robotic arm, a wheeled robot, and a self-driving car.

### Claim 7 (depends on 1)

The method of claim 1, wherein:
the robotic system comprises a robotic arm; and
the task comprises picking up an item and moving the item to a new location.

### Claim 8

A system comprising:
one or more computer-readable storage media;
a processing system operatively coupled to the one or more computer-readable storage media; and
program instructions, stored on the one or more computer-readable storage media, wherein the program instructions, when read and executed by the processing system, direct the processing system to:
identify a task for a robotic system, wherein the task comprises one or more requirements;
provide one or more details related to the task and one or more details related to the robotic system to a machine learning model, wherein the machine learning model determines an optimal trajectory for the robotic system to follow when performing the task and wherein the optimal trajectory satisfies the one or more requirements and minimizes an amount of time it takes the robotic system to successfully complete the task;
receive an output from the machine learning model, wherein the output comprises the optimal trajectory; and
direct the robotic system to complete the task via the optimal trajectory.

### Claim 9 (depends on 8)

The system of claim 8, wherein the machine learning model comprises one or more deep artificial neural nets.

### Claim 10 (depends on 9)

The system of claim 9, wherein the machine learning model is trained using one or more of reinforcement learning and imitation learning.

### Claim 11 (depends on 8)

The system of claim 8, wherein the details related to the task include at least:
information related to kinematics and geometry of the robotic system;
a start of the optimal trajectory; and
an end of the optimal trajectory.

### Claim 12 (depends on 8)

The system of claim 8, wherein the one or more requirements include at least:
the robotic system does not collide with any objects;
motion of the robotic system is smooth throughout the optimal trajectory;
velocity of the robotic system remain below a maximum velocity; and
acceleration of the robotic system remain below a maximum acceleration.

### Claim 13 (depends on 8)

The system of claim 8, wherein the robotic system comprises one or more of: a robotic arm, a wheeled robot, and a self-driving car.

### Claim 14 (depends on 8)

The system of claim 8, wherein:
the robotic system comprises a robotic arm; and
the task comprises picking up an item and moving the item to a new location.

### Claim 15

One or more computer-readable storage media having program instructions stored thereon to predict optimal trajectories, wherein the program instructions, when read and executed by a processing system, direct the processing system to at least:
identify a task for a robotic system, wherein the task comprises one or more requirements;
provide one or more details related to the task and one or more details related to the robotic system to a machine learning model, wherein the machine learning model determines an optimal trajectory for the robotic system to follow when performing the task and wherein the optimal trajectory satisfies the one or more requirements and minimizes an amount of time it takes the robotic system to successfully complete the task;
receive an output from the machine learning model, wherein the output comprises the optimal trajectory; and
direct the robotic system to complete the task via the optimal trajectory.

### Claim 16 (depends on 15)

The one or more computer-readable storage media of claim 15, wherein the machine learning model comprises one or more deep artificial neural nets.

### Claim 17 (depends on 16)

The one or more computer-readable storage media of claim 16, wherein the machine learning model is trained using one or more of reinforcement learning and imitation learning.

### Claim 18 (depends on 15)

The one or more computer-readable storage media of claim 15, wherein the details related to the task include at least:
information related to kinematics and geometry of the robotic system;
a start of the optimal trajectory; and
an end of the optimal trajectory.

### Claim 19 (depends on 15)

The one or more computer-readable storage media of claim 15, wherein the one or more requirements include at least:
the robotic system does not collide with any objects;
motion of the robotic system is smooth throughout the optimal trajectory;
velocity of the robotic system remain below a maximum velocity; and
acceleration of the robotic system remain below a maximum acceleration.

### Claim 20 (depends on 15)

The one or more computer-readable storage media of claim 15, wherein the robotic system comprises one or more of: a robotic arm, a wheeled robot, and a self-driving car.
