# US 2021/0341613 A1 · Sensor pod calibration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0341613 A1
- Document ID: US20210341613A1
- Status: Published application
- Kind: A1
- Application number: 16/864,138
- Filed: 2020-04-30
- Published: 2021-11-04
- Earliest claimed: 2020-04-30
- Assignees: Zoox, Inc.
- Applicants: Zoox, Inc.
- Inventors: Derek Adams; Daniel Glenn Johnson; Christopher William Labadie; Ryan McMichael; Daniel Miller; Peter Thomas Mitros; Anubhav Thakur; Joseph Patrick Warga; Austin In-Jei Yi
- Web page: https://patentgrep.com/patent/US20210341613A1

## CPC

- B60R 11/04: Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
- G01S 17/89: for mapping or imaging
- G01S 7/4813: {Housing arrangements}
- G01S 7/4912: Receivers
- G01S 17/931: of land vehicles
- B60R 2011/004: {outside the vehicle}
- B60R 2300/802: for monitoring and displaying vehicle exterior blind spot views
- G01S 2013/93273: {on the top of the vehicles}
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A sensor pod system includes one or more sensor pods with a plurality of sensors configured to collect data from an environment. A sensor pod may have an effective field of view created by individual sensors with overlapping fields of view. The sensor pod system may include sensors of different types and modalities. Sensor pods of the sensor pod system may be modularly installed on a vehicle, for example, an autonomous vehicle and collect and provide data of the environment during operation of the vehicle. A combination of techniques may be used to calibrate a sensor pod prior to installation in the vehicle and may be used to determine if the sensor pod is compatible with other sensor pods and may also be used to calibrate other sensors when integrating the sensor pod into the vehicle.

## Claims

### Claim 1

A system comprising:
a frame comprising a vehicle mounting interface, a first mounting location and a second mounting location;
a first sensor coupled to the frame at the first mounting location;
a second sensor coupled to the frame at the second mounting location;
one or more processors; and
non-transitory computer-readable media storing one or more instructions that, when executed, cause the one or more processors to perform acts comprising:
receiving, from the first sensor, first sensor data of an environment of the system;
receiving, from the second sensor, second sensor data of the environment of the system;
segmenting, as first segmented data, the first sensor data based at least in part on a representation of a first invariant object in the first sensor data;
segmenting, as second segmented data, the second sensor data based at least in part on a representation of a second invariant object in the second sensor data;
determining, based at least in part on the first segmented data, a subset of the first sensor data associated with the first invariant object;
determining, based at least in part on the second segmented data, a subset of the second sensor data associated with the second invariant object;
determining, based at least in part on the subset of the first sensor data, a first calibration error associated with the first sensor;
determining, based at least in part on the subset of the second sensor data, a second calibration error associated with the second sensor; and
determining, based at least in part on the first calibration error associated with the first sensor and the second calibration error associated with the second sensor, a system calibration factor associated with the system.

### Claim 2 (depends on 1)

The system of claim 1, further comprising a mount coupled to the frame at the vehicle mounting interface, the mount fixed relative to the environment.

### Claim 3 (depends on 1)

The system of claim 1, the environment containing the first invariant within a 90-degree field of view.

### Claim 4 (depends on 1)

The system of claim 1, the mount having indexed orientations, the first sensor data received at a first index, the second sensor data received at a second index, the determining further based at least in part on the first index orientation and the second index orientation.

### Claim 5 (depends on 1)

The system of claim 1, further comprising a third sensor coupled to the frame at a third mounting location; and
the acts further comprising:
receiving, from the third sensor, third sensor data of the environment of the system;
segmenting, as third segmented data, the third sensor data based at least in part on a representation of a third invariant object in the third sensor data;
determining, based at least in part on the third segmented data, a subset of the third sensor data associated with the third invariant object;
determining, based at least in part on the subset of the third sensor data, a third calibration error associated with the third sensor; and
the determining the system calibration factor associated with the system being further based at least in part on the third calibration error associated with the third sensor.

### Claim 6 (depends on 5)

The system of claim 5, wherein:
the mount has indexed orientations;
the first sensor data and second sensor data received with the frame oriented at a first index;
the third sensor data received with the frame oriented at a second index;
the acts further comprising:
receiving fourth sensor data from the second sensor with the frame at a second index; and
the determining the system calibration factor associated with the system being further based at least in part on the third sensor data and the fourth sensor data.

### Claim 7 (depends on 1)

The system of claim 1, the first invariant being the same as the second invariant.

### Claim 8 (depends on 1)

The system of claim 1, the first invariant comprising a fiducial, a portion of a building, wall, corner, pole, window, roof/ceiling line, wall-ceiling/roof intersection, or combinations thereof.

### Claim 9 (depends on 1)

The system of claim 1, wherein the first sensor is a different type of sensor from the second sensor.

### Claim 10 (depends on 1)

The system of claim 1, wherein the first sensor comprises a camera, and the second sensor comprises a lidar.

### Claim 11 (depends on 1)

The system of claim 1, the acts further comprising:
receiving, from a sensor pod, sensor pod data representative of a third invariant;
determining, based at least in part on the sensor pod data, a sensor pod calibration factor; and
determining, based at least in part on the system calibration factor and the sensor pod calibration factor, whether the sensor pod is compatible with the first sensor and the second sensor mounted to the frame, when the sensor pod and the frame are mounted to a vehicle.

### Claim 12

A vehicle comprising:
a sensor pod comprising a first sensor and a second sensor;
one or more processors; and
non-transitory computer-readable media storing one or more instructions that, when executed, cause the one or more processors to perform acts comprising:
receiving, from the first sensor, sensor data of an environment of the vehicle;
segmenting, as segmented data, the sensor data based at least in part on a representation of an invariant object in the sensor data;
determining, based at least in part on the segmented data, a subset of the sensor data associated with the invariant object;
determining, based at least in part on the subset of the sensor data, a calibration error associated with the first sensor; and
determining, based at least in part on the calibration error and a sensor pod calibration factor, an estimated calibration error associated with the second sensor.

### Claim 13 (depends on 12)

The vehicle of claim 12, wherein the sensor pod calibration factor is based at least in part on a first calibration error associated with the first sensor based at least in part on first data received from the first sensor, representative of a first invariant, and a second calibration error associated with the second sensor based at least in part on second data received from the second sensor, representative of a second invariant.

### Claim 14 (depends on 13)

The vehicle of claim 13, wherein the first invariant is the same as the second invariant.

### Claim 15 (depends on 12)

The vehicle of claim 12, wherein the first sensor comprises a camera, and the second sensor comprises a lidar.

### Claim 16 (depends on 12)

The vehicle of claim 12, the acts further comprising:
receiving, from another sensor pod, sensor pod data representative of a third invariant;
determining, based at least in part on the sensor pod data, another sensor pod calibration factor; and
determining, based at least in part on the sensor pod calibration factor and the other sensor pod calibration factor, whether the sensor pod and the other sensor pod have compatible effective fields of view when the sensor pod and the other sensor pod are mounted to the vehicle.

### Claim 17

A method comprising:
receiving, from a first sensor pod, first sensor data representative of a first invariant;
determining, based at least in part on the first sensor data, a first calibration factor;
receiving, from a second sensor pod, second sensor data representative of a second invariant;
determining, based at least in part on the second sensor data, a second calibration factor; and
determining, based at least in part on the first calibration factor and the second calibration factor, whether the first sensor pod and the second sensor pod have compatible effective fields of view.

### Claim 18 (depends on 17)

The method of claim 17, further comprising pairing the first sensor pod with the second sensor pod for installation on a vehicle when determined to have compatible effective fields of view.

### Claim 19 (depends on 17)

The method of claim 17, further comprising installing, when determined to have compatible fields of view, the first sensor pod on a vehicle where the second on a vehicle is already installed.

### Claim 20 (depends on 17)

The method of claim 17, wherein the first sensor data comprises data from a plurality of sensors of the first sensor pod.
