# US 12,762,825 B2 · Adaptive learnable pooling for overlapping camera features

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,825 B2
- Document ID: US12762825B2
- Status: Granted
- Kind: B2
- Application number: 18/506,583
- Filed: 2023-11-10
- Published: 2025-05-15
- Granted: 2026-10-13
- Earliest claimed: 2023-11-10
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Varun Ravi Kumar; Meysam Sadeghigooghari; Senthil Kumar Yogamani
- Web page: https://patentgrep.com/patent/US12762825B2

## CPC

- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06T 7/11: Region-based segmentation
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/806: {of extracted features}
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06V 2201/07: Target detection
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An apparatus is configured to fuse features from a plurality of camera images using a set of learnable parameters. The apparatus may extract features from a respective image from each camera of a plurality of cameras, and fuse the features into a fused image having a grid structure. To fuse the features the apparatus may determine a contribution of the respective image from each of the plurality of cameras to a respective cell of the grid structure, and aggregate, based on the contribution to the respective cell and a respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features.

## Claims

### Claim 1

An apparatus for processing image data, the apparatus comprising:
a memory for storing the image data; and
processing circuitry in communication with the memory, wherein the processing circuitry is configured to:
extract features from a respective image from each camera of a plurality of cameras; and
fuse the features into a fused image having a grid structure, wherein to fuse the features, the processing circuitry is configured to:
determine a contribution of the respective image from each of the plurality of cameras to a respective cell of the grid structure, and
aggregate, based on the contribution to the respective cell and a respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features, wherein the set of learnable parameters comprises a unique weight assigned to each camera of the plurality of cameras that contributes to the respective cell.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein to determine the contribution, the processing circuitry is configured to:
determine, from a lookup table, the contribution of the respective image from each of the plurality of cameras to the respective cell of the grid structure.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the lookup table indicates contribution of the respective image from each of the plurality of cameras to the respective cell of the grid structure based on one or more of a configuration of the plurality of cameras or a type of the plurality of cameras.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein the processing is circuitry is further configured to:
generate the lookup table based on one or more of a configuration of the plurality of cameras or a type of the plurality of cameras.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein to aggregate, based on the contribution to the respective cell and the respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features, the processing circuitry is configured to:
calculate the aggregated features according to a function:
aggregated_features
i
=
W
i
[
i
]
·
F
i
⁢
1
+
W
i
[
2
]
·
F
i
⁢
2
+
…
⁢
W
i
[
N
i
]
·
F
i
⁢
N
⁢
i
,
[
1
]
wherein i is the respective cell, aggregated_featuresi are the aggregated features at cell i, Wi[1] is the weight of a first camera of the plurality of cameras that contributes to cell i, Fi1 is the respective features of the first camera of the plurality of cameras at cell i, Wi[2] is the weight of a second camera of the plurality of cameras that contributes to cell i, Fi2 is the respective features of the second camera of the plurality of cameras at cell i, Wi[Ni] is the weight of an Nth camera of the plurality of cameras that contributes to cell i, and FiNi is the respective features of the Nth camera of the plurality of cameras at cell i.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the processing is circuitry is further configured to:
apply, to the fused image, object detection decoder to generate a set of bounding boxes that indicate a location of one or more objects within the fused image.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the processing is circuitry is further configured to:
apply, to the fused image, a segmentation decoder to identify types of objects in the fused image.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the processing circuitry and the memory are part of an advanced driver assistance system (ADAS).

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the apparatus further comprises:
the plurality of cameras.

### Claim 10

A method for processing image data, the method comprising:
extracting features from a respective image from each camera of a plurality of cameras; and
fusing the features into a fused image having a grid structure, wherein fusing the features comprises:
determining a contribution of the respective image from each of the plurality of cameras to a respective cell of the grid structure, and
aggregating, based on the contribution to the respective cell and a respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features, wherein the set of learnable parameters comprises a unique weight assigned to each camera of the plurality of cameras that contributes to the respective cell.

### Claim 11 (depends on 10)

The method of claim 10, wherein determining the contribution comprises:
determining, from a lookup table, the contribution of the respective image from each of the plurality of cameras to the respective cell of the grid structure.

### Claim 12 (depends on 11)

The method of claim 11, wherein the lookup table indicates contribution of the respective image from each of the plurality of cameras to the respective cell of the grid structure based on one or more of a configuration of the plurality of cameras or a type of the plurality of cameras.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
generating the lookup table based on one or more of a configuration of the plurality of cameras or a type of the plurality of cameras.

### Claim 14 (depends on 10)

The method of claim 10, wherein aggregating, based on the contribution to the respective cell and the respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features comprises:
calculating the aggregated features according to a function:
aggregated_features
i
=
W
i
[
1
]
·
F
i
⁢
1
+
W
i
[
2
]
·
F
i
⁢
2
+
…
⁢
W
i
[
N
i
]
·
F
i
⁢
N
⁢
i
,
wherein i is the respective cell, aggregated_featuresi are the aggregated features at cell i, Wi[1] is the weight of a first camera of the plurality of cameras that contributes to cell i, Filis the respective features of the first camera of the plurality of cameras at cell i, Wi[2] is the weight of a second camera of the plurality of cameras that contributes to cell i, Fi2 is the respective features of the second camera of the plurality of cameras at cell i, Wi[Ni] is the weight of an Nth camera of the plurality of cameras that contributes to cell i, and FiNi is the respective features of the Nth camera of the plurality of cameras at cell i.

### Claim 15 (depends on 10)

The method of claim 10, further comprising:
applying, to the fused image, object detection decoder to generate a set of bounding boxes that indicate a location of one or more objects within the fused image.

### Claim 16 (depends on 10)

The method of claim 10, further comprising:
applying, to the fused image, a segmentation decoder to identify types of objects in the fused image.

### Claim 17 (depends on 10)

The method of claim 10, wherein the method is performed by an advanced driver assistance system (ADAS).

### Claim 18

A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to:
extract features from a respective image from each camera of a plurality of cameras; and
fuse the features into a fused image having a grid structure, wherein to fuse the features, the instructions further cause the one or more processors to:
determine a contribution of the respective image from each of the plurality of cameras to a respective cell of the grid structure, and
aggregate, based on the contribution to the respective cell and a respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features, wherein the set of learnable parameters comprises a unique weight assigned to each camera of the plurality of cameras that contributes to the respective cell.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein to determine the contribution, the instructions further cause the one or more processors to:
determine, from a lookup table, the contribution of the respective image from each of the plurality of cameras to the respective cell of the grid structure.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the lookup table indicates contribution of the respective image from each of the plurality of cameras to the respective cell of the grid structure based on one or more of a configuration of the plurality of cameras or a type of the plurality of cameras.

### Claim 21 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the instructions further cause the one or more processors to:
generate the lookup table based on one or more of a configuration of the plurality of cameras or a type of the plurality of cameras.

### Claim 22 (depends on 20)

The non-transitory computer-readable storage medium of claim 20, wherein to aggregate, based on the contribution to the respective cell and the respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features, the instructions further cause the one or more processors to:
calculate the aggregated features according to a function:
aggregated_features
i
=
W
i
[
1
]
·
F
i
⁢
1
+
W
i
[
2
]
·
F
i
⁢
2
+
…
⁢
W
i
[
N
i
]
·
F
i
⁢
N
⁢
i
,
wherein i is the respective cell, aggregated_featuresi are the aggregated features at cell i, Wi[1] is the weight of a first camera of the plurality of cameras that contributes to cell i, Filis the respective features of the first camera of the plurality of cameras at cell i, Wi[2] is the weight of a second camera of the plurality of cameras that contributes to cell i, Fi2 is the respective features of the second camera of the plurality of cameras at cell i, Wi[Ni] is the weight of an Nth camera of the plurality of cameras that contributes to cell i, and FiNi is the respective features of the Nth camera of the plurality of cameras at cell i.

### Claim 23

An apparatus configured for processing image data, the apparatus comprising:
means for extracting features from a respective image from each camera of a plurality of cameras; and
means for fusing the features into a fused image having a grid structure, wherein the means for fusing the features comprises:
means for determining a contribution of the respective image from each of the plurality of cameras to a respective cell of the grid structure, and
means for aggregating, based on the contribution to the respective cell and a respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features, wherein the set of learnable parameters comprises a unique weight assigned to each camera of the plurality of cameras that contributes to the respective cell.

### Claim 24 (depends on 23)

The apparatus of claim 23, wherein the means for determining the contribution comprises:
means for determining, from a lookup table, the contribution of the respective image from each of the plurality of cameras to the respective cell of the grid structure.

### Claim 25 (depends on 24)

The apparatus of claim 24, wherein the lookup table indicates contribution of the respective image from each of the plurality of cameras to the respective cell of the grid structure based on one or more of a configuration of the plurality of cameras or a type of the plurality of cameras.

### Claim 26 (depends on 23)

The apparatus of claim 23, wherein the means for aggregating, based on the contribution to the respective cell and the respective set of learnable parameters for each cell, the features from each of the respective images to each respective cell of the fused image to generate aggregated features comprises:
means for calculating the aggregated features according to a function:
aggregated_features
i
=
W
i
[
1
]
·
F
i
⁢
1
+
W
i
[
2
]
·
F
i
⁢
2
+
…
⁢
W
i
[
N
i
]
·
F
i
⁢
N
⁢
i
,
wherein i is the respective cell, aggregated_featuresi are the aggregated features at cell i, Wi[1] is the weight of a first camera of the plurality of cameras that contributes to cell i, Filis the respective features of the first camera of the plurality of cameras at cell i, Wi[2] is the weight of a second camera of the plurality of cameras that contributes to cell i, Fi2 is the respective features of the second camera of the plurality of cameras at cell i, Wi[Ni] is the weight of an Nth camera of the plurality of cameras that contributes to cell i, and FiNi is the respective features of the Nth camera of the plurality of cameras at cell i.