# US 2026/0073631 A1 · Systems and methods for environment mapping based on multi-domain sensor data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073631 A1
- Document ID: US20260073631A1
- Status: Published application
- Kind: A1
- Application number: 19/128,156
- Filed: 2023-02-02
- Published: 2026-03-12
- Earliest claimed: 2023-02-02
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Changhong YANG; Zhixun XIA; Nan JIA; Linkun XU
- Web page: https://patentgrep.com/patent/US20260073631A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 17/05: Geographic models
- G01C 21/3837: {Data obtained from a single source}
- G06T 7/13: Edge detection
- G06T 7/50: Depth or shape recovery
- G06T 15/08: Volume rendering
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data}
- G06V 20/20: in augmented reality scenes
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- B60W 60/00: Drive control systems specially adapted for autonomous road vehicles
- B60W 2420/403: Image sensing, e.g. optical camera
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

Systems and techniques for environment mapping are described. In some examples, a system receives image data and depth data captured using at least one sensor. The image data and the depth data both include respective representations of an environment. The system processes the image data using semantic segmentation to identify segments of the environment that represent different types of objects in the environment in the image data. The system combines the depth data with the semantic segmentation to generate a voxel-based three-dimensional map of the environment.

## Claims

### Claim 1

An apparatus for environment mapping, the apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory and configured to:
receive image data and depth data captured using at least one sensor, the image data and the depth data including respective representations of an environment;
process the image data using semantic segmentation to generate segmented image data that identifies a plurality of segments of the environment, wherein the plurality of segments represent different types of objects in the environment; and
combine the depth data with the segmented image data to generate a voxel-based three-dimensional map of the environment.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the depth data includes a point cloud with a plurality of points, and wherein the at least one processor is configured to omit at least one point from the point cloud from the voxel-based three-dimensional map of the environment based on the segmented image data.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the depth data includes a point cloud with a plurality of points, and wherein the at least one processor is configured to add at least one voxel to the voxel-based three-dimensional map of the environment based on the segmented image data despite a lack of a point in the point cloud corresponding to the at least one voxel.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the depth data identifies an edge of an object of the different types of objects in the environment, wherein the at least one processor is configured to add at least one voxel to the voxel-based three-dimensional map of the environment corresponding to a non-edge portion of the object based on the segmented image data despite a lack of representation of the non-edge portion of the object in the depth data.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured identify a depth of an object in the voxel-based three-dimensional map of the environment based on the depth data, wherein the at least one processor is configured to identify a shape of the object in the voxel-based three-dimensional map of the environment based on the segmented image data.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to identify color information corresponding to voxels of the voxel-based three-dimensional map of the environment based on colors of corresponding portions of the environment as represented in the image data.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to identify respective confidence levels corresponding to the plurality of segments being identified, using the semantic segmentation, as respectively representing the different types of objects.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to identify, based on the segmented image data, different voxels of the voxel-based three-dimensional map of the environment as representing the different types of objects.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the different types of objects in the environment include at least one of ground, sky, plants, structures, people, and vehicles.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the at least one sensor includes an image sensor, wherein the image sensor is configured to capture at least the image data.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the depth data is based on the image data from the image sensor.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the at least one sensor includes a depth sensor, wherein the depth sensor is configured to capture at least the depth data.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to output an indication of the voxel-based three-dimensional map of the environment.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to cause display of at least a portion of the voxel-based three-dimensional map of the environment using a display.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to cause transmission of at least a portion of the voxel-based three-dimensional map of the environment to a recipient device using a communication interface.

### Claim 16 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to generate a route through the environment for a vehicle based on the voxel-based three-dimensional map of the environment.

### Claim 17 (depends on 1)

The apparatus of claim 1, wherein the at least one processor is configured to modify movement of a vehicle through the environment based on the voxel-based three-dimensional map of the environment.

### Claim 18 (depends on 1)

The apparatus of claim 1, wherein the apparatus includes at least one of a head-mounted display (HMD), a mobile handset, or a wireless communication device.

### Claim 19

A method for environment mapping, the method comprising:
receiving image data and depth data captured using at least one sensor, the image data and the depth data including respective representations of an environment;
processing the image data using semantic segmentation to generate segmented image data that identifies a plurality of segments of the environment, wherein the plurality of segments represent different types of objects in the environment; and
combining the depth data with the segmented image data to generate a voxel-based three-dimensional map of the environment.

### Claim 20

(canceled)

### Claim 21

(canceled)

### Claim 22

(canceled)

### Claim 23

(canceled)

### Claim 24

(canceled)

### Claim 25

(canceled)

### Claim 26 (depends on 19)

The method of claim 19, further comprising:
identifying, based on the segmented image data, different voxels of the voxel-based three-dimensional map of the environment as representing the different types of objects.

### Claim 27

(canceled)

### Claim 28

(canceled)

### Claim 29

(canceled)

### Claim 30

(canceled)