# US 2019/0377358 A1 · Systems and methods for simulation utilizing a segmentable monolithic mesh

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377358 A1
- Document ID: US20190377358A1
- Status: Published application
- Kind: A1
- Application number: 16/001,340
- Filed: 2018-06-06
- Published: 2019-12-12
- Earliest claimed: 2018-06-06
- Assignees: Toyota Research Institute, Inc.
- Applicants: Toyota Research Institute, Inc.
- Inventors: Samuel Zapolsky
- Web page: https://patentgrep.com/patent/US20190377358A1

## CPC

- G01S 17/89: for mapping or imaging
- G05D 1/0248: {in combination with a laser (lasers per se H01S)}
- B60W 30/06: Automatic manoeuvring for parking
- G01S 7/4808: {Evaluating distance, position or velocity data}
- G01S 17/08: for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48)
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/931: of land vehicles
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2200/04: involving 3D image data
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods for optimizing the simulation of a robot in an environment are disclosed herein. The system includes an environment sensor configured to generate signals indicative of objects within the environment. The system further includes a computing device having a processor and a non-transitory computer-readable memory and a machine-readable instruction set stored in the non-transitory computer-readable memory. The machine-readable instruction set causes the computing device to: generate a monolithic mesh of the environment such that the objects in the environment form a static rigid body with the environment, create an action plan for the simulation, identify objects within the monolithic mesh that the simulation intends to interact with, segment the objects that the simulation intends to interact with from the monolithic mesh, perform the simulation of the objects segmented from the monolithic mesh, and recombine the objects segmented from the monolithic mesh for the simulation with the monolithic mesh.

## Claims

### Claim 1

A system for optimized simulation of a robot in an environment comprising:
an environment sensor configured to generate one or more signals indicative of one or more objects within the environment;
a computing device communicatively coupled to the environment sensor, the computing device comprising a processor and a non-transitory computer readable memory; and
a machine-readable instruction set stored in the non-transitory computer-readable memory that causes the computing device to perform at least the following when executed by the processor:
generate a monolithic mesh of the environment based on the one or more signals from the environment sensor indicative of the one or more objects within the environment such that the one or more objects in the environment form a static rigid body with the environment,
create an action plan for the simulation,
identify one or more objects within the monolithic mesh that the simulation intends to interact with based on the action plan,
segment the one or more objects that the simulation intends to interact with based on the action plan from the monolithic mesh,
perform the simulation of the one or more objects segmented from the monolithic mesh, and
recombine the one or more objects segmented from the monolithic mesh for the simulation with the monolithic mesh.

### Claim 2 (depends on 1)

The system of claim 1, wherein the environment sensor comprises a camera with a field of view of the environment, the camera generates one or more signals corresponding to image data captured by the camera of the environment.

### Claim 3 (depends on 2)

The system of claim 2, wherein the machine-readable instruction set further causes the computing device to perform the following when executed by the processor:
assign labels to the one or more objects within the field of view of the environment captured by the camera.

### Claim 4 (depends on 3)

The system of claim 3, wherein the one or more objects between the camera and a first distance from the camera are assigned a detailed label, the one or more objects between the first distance and a second distance are assigned a generic label, and the second distance is greater than the first distance from the camera.

### Claim 5 (depends on 1)

The system of claim 1, wherein the environment sensor comprises a LIDAR system, the LIDAR system generates one or more signals corresponding to distance measurements between the LIDAR system and the one or more objects in the environment.

### Claim 6

A method of simulating a robot in an environment comprising:
generating a monolithic mesh of the environment;
identifying one or more objects within the monolithic mesh that a simulation intends to interact with during the simulation;
segmenting the one or more objects that the simulation intends to interact with from the monolithic mesh;
performing the simulation of the one or more objects segmented from the monolithic mesh; and
recombining the one or more objects segmented from the monolithic mesh for the simulation with the monolithic mesh.

### Claim 7 (depends on 6)

The method of claim 6, further comprising determining one or more parameters of the one or more objects segmented from the monolithic mesh wherein the one or more parameters include at least one of the following: an inertia, a shape, a center of mass, a coefficient of friction, a mass, and a dimension.

### Claim 8 (depends on 7)

The method of claim 7, wherein the one or more parameters include an approximation of an actual value determined from a database.

### Claim 9 (depends on 8)

The method of claim 8, further comprising updating the approximation of the actual value of the one or more parameters in response to performing the simulation of the one or more objects.

### Claim 10 (depends on 8)

The method of claim 8, wherein the one or more parameters of the one or more objects are removed from the one or more objects when the one or more objects are recombined with the monolithic mesh.

### Claim 11 (depends on 6)

The method of claim 6, wherein the monolithic mesh of the environment is generated in response to one or more signals from an environment sensor configured to scan the environment.

### Claim 12 (depends on 11)

The method of claim 11, further comprising scanning the environment with the environment sensor, wherein the environment sensor generates one or more signals indicative of objects detected within the environment.

### Claim 13 (depends on 12)

The method of claim 12, further comprising repeating the scan of the environment to identify non-transient objects within the environment.

### Claim 14 (depends on 11)

The method of claim 11, wherein the environment sensor comprises a camera with a field of view of the environment, the camera generates one or more signals corresponding to image data captured by the camera of the environment.

### Claim 15 (depends on 11)

The method of claim 11, wherein the environment sensor comprises a LIDAR system, the LIDAR system generates one or more signals corresponding to distance measurements between the LIDAR system and the one or more objects in the environment.

### Claim 16

A vehicle comprising:
an environment sensor configured to scan an environment and generate one or more signals indicative of one or more objects within the environment;
a computing device communicatively coupled to the environment sensor, the computing device comprising a processor and a non-transitory computer-readable memory; and
a machine-readable instruction set stored in the non-transitory computer-readable memory that causes the computing device to perform at least the following when executed by the processor:
simulate an action plan within the environment, wherein a simulation includes:
generating a monolithic mesh of the environment based on the one or more signals from the environment sensor indicative of the one or more objects within the environment such that the one or more objects in the environment form a static rigid body with the environment,
identifying one or more objects within the monolithic mesh that the simulation intends to interact with based on the action plan,
segmenting the one or more objects that the simulation intends to interact with based on the action plan from the monolithic mesh,
performing the simulation of the one or more objects segmented from the monolithic mesh,
adjusting the action plan in response to one or more results of the simulation, and
recombining the one or more objects segmented from the monolithic mesh for the simulation with the monolithic mesh; and
execute the adjusted action plan within the environment in response to the one or more results of the simulation.

### Claim 17 (depends on 16)

The vehicle of claim 16, wherein the action plan includes automatically navigating an intersection.

### Claim 18 (depends on 16)

The vehicle of claim 16, wherein the environment sensor comprises a camera with a field of view of the environment, the camera generates one or more signals corresponding to image data captured by the camera of the environment.

### Claim 19 (depends on 18)

The vehicle of claim 18, wherein the machine-readable instruction set further causes the computing device to perform the following when executed by the processor:
assign labels to the one or more objects within the field of view of the environment captured by the camera.

### Claim 20 (depends on 19)

The vehicle of claim 19, wherein the one or more objects between the camera and a first distance from the camera are assigned a detailed label, the one or more objects between the first distance and a second distance are assigned a generic label, and the second distance is greater than the first distance from the camera.
