# US 2026/0091494 A1 · Techniques for robot control based on differentiable task-and-motion planning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0091494 A1
- Document ID: US20260091494A1
- Status: Published application
- Kind: A1
- Application number: 19/235,084
- Filed: 2025-06-11
- Published: 2026-04-02
- Earliest claimed: 2024-10-02
- Assignees: NVIDIA Corporation
- Applicants: Nvidia Corporation
- Inventors: Fabio TOZETO RAMOS; William SHEN; Caelan Reed GARRETT; Tucker Ryer HERMANS; Ankit GOYAL
- Web page: https://patentgrep.com/patent/US20260091494A1

## CPC

- B25J 9/1664: {characterised by motion, path, trajectory planning}
- B25J 9/161: {Hardware, e.g. neural networks, fuzzy logic, interfaces, processor}
- B25J 9/1661: {characterised by task planning, object-oriented languages}
- B25J 13/08: by means of sensing devices, e.g. viewing or touching devices
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})

## Abstract

Techniques for controlling a robot include receiving a task and motion planning (TAMP)-domain language input, a goal, and sensor data, generating a plan skeleton based on the TAMP-domain language input, the goal, and the sensor data, generating, based on the plan skeleton, one or more feasible particles, generating, based on the plan skeleton and at least one of the one or more feasible particles, a robot plan, and causing the robot to perform at least part of the robot plan.

## Claims

### Claim 1

A computer-implemented method for controlling a robot, the method comprising:
receiving a task and motion planning (TAMP)-domain language input, a goal, and sensor data;
generating a plan skeleton based on the TAMP-domain language input, the goal, and the sensor data;
generating, based on the plan skeleton, one or more feasible particles;
generating, based on the plan skeleton and at least one of the one or more feasible particles, a robot plan; and
causing the robot to perform at least part of the robot plan.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein determining the plan skeleton comprises:
generating, based on the sensor data, a state; and
generating the plan skeleton further based on the state.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein determining the plan skeleton comprises:
selecting, based on the state, one or more first actions included in the plan skeleton;
selecting, based on the goal, one or more second actions included in the first actions; and
generating, based on the one or more second actions, the plan skeleton.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein determining the plan skeleton comprises performing a best-first search of candidate plan skeletons.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein determining the one or more feasible particles comprises:
generating, based on a candidate plan skeleton, one or more particles;
generating one or more optimized particles based on one or more constraints included in the candidate plan skeleton, one or more cost functions included in the candidate plan skeleton, and the one or more particles; and
determining, based on the one or more optimized particles, the one or more feasible particles.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein generating the one or more particles comprises caching one or more particle outputs from one or more samplers keyed at least by one of a constraint context or a parameter binding.

### Claim 7 (depends on 5)

The computer-implemented method of claim 5, wherein generating the one or more optimized particles comprises solving a constraint satisfaction problem.

### Claim 8 (depends on 5)

The computer-implemented method of claim 5, wherein generating the one or more optimized particles comprises:
generating, based on the one or more cost functions included in the candidate plan skeleton, one or more vectorized versions of the cost functions;
performing, based on the one or more vectorized versions of the cost functions, at least one of a batch cost computation or a gradient evaluation in parallel; and
generating, based on at least one of the batch cost computation or the gradient evaluation, the one or more optimized particles.

### Claim 9 (depends on 5)

The computer-implemented method of claim 5, wherein generating the one or more optimized particles comprises at least one of a batch cost computation or a gradient evaluation performed on a graphics processing unit (GPU).

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein generating the robot plan comprises:
computing, based on the one or more feasible particles and the plan skeleton, one or more ranks;
determining, based on the one or more ranks, a first feasible particle with a highest rank; and
generating, based on the first feasible particle with the highest rank and the plan skeleton, the robot plan.

### Claim 11

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:
receiving a TAMP-domain language input, a goal, and sensor data;
determining a plan skeleton based on the TAMP-domain language input, the goal, and the sensor data;
determining, based on the plan skeleton, one or more feasible particles;
generating, based on the plan skeleton and at least one of the one or more feasible particles, a robot plan; and
causing a robot to perform at least part of the robot plan.

### Claim 12 (depends on 11)

The one or more non-transitory computer-readable media of claim 11, wherein determining the one or more feasible particles comprises:
generating, based on a candidate plan skeleton, one or more particles;
generating one or more optimized particles based on one or more constraints included in the candidate plan skeleton, one or more cost functions included in the candidate plan skeleton, and the one or more particles; and
determining, based on the one or more optimized particles, the one or more feasible particles.

### Claim 13 (depends on 12)

The one or more non-transitory computer-readable media of claim 12, wherein generating the one or more particles comprises caching one or more particle outputs from one or more samplers keyed at least by one of a constraint context or a parameter binding.

### Claim 14 (depends on 12)

The one or more non-transitory computer-readable media of claim 12, wherein generating the one or more optimized particles comprises solving a constraint satisfaction problem.

### Claim 15 (depends on 12)

The one or more non-transitory computer-readable media of claim 12, wherein generating the one or more optimized particles comprises:
generating, based on the one or more cost functions included in the candidate plan skeleton, one or more vectorized versions of the cost functions;
performing, based on the one or more vectorized versions of the cost functions, at least one of a batch cost computation or a gradient evaluation in parallel; and
generating, based on at least one of the batch cost computation or the gradient evaluation, the one or more optimized particles.

### Claim 16 (depends on 12)

The one or more non-transitory computer-readable media of claim 12, wherein generating the one or more optimized particles comprises selecting the candidate plan skeleton from a priority queue, wherein each entry in the priority queue is associated with a heuristic value determined based on at least one of an estimated cost or a feasibility of completing the candidate plan skeleton in the entry.

### Claim 17 (depends on 12)

The one or more non-transitory computer-readable media of claim 12, wherein generating the one or more particles comprises sampling a uniform distribution.

### Claim 18 (depends on 12)

The one or more non-transitory computer-readable media of claim 12, wherein generating the one or more optimized particles comprises using at least one of a stochastic first-order optimizer, an augmented Lagrangian, coordinate descent, or a second-order optimizer.

### Claim 19 (depends on 11)

The one or more non-transitory computer-readable media of claim 11, wherein generating the robot plan comprises:
computing, based on the one or more feasible particles and the plan skeleton, one or more ranks;
determining, based on the one or more ranks, a first feasible particle with a highest rank; and
generating, based on the first feasible particle with the highest rank and the plan skeleton, the robot plan.

### Claim 20

A system comprising:
one or more memories storing instructions, and
one or more processors that are coupled to the one or more memories and, when executing the instructions, are configured to:
receive a TAMP-domain language input, a goal, and sensor data,
determine a plan skeleton based on the TAMP-domain language input, the goal, and the sensor data;
determine, based on the plan skeleton, one or more feasible particles,
generate, based on the plan skeleton and at least one of the one or more feasible particles, a robot plan, and
cause a robot to perform at least part of the robot plan.
