# US 2025/0168313 A1 · Multi-camera 3d content creation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0168313 A1
- Document ID: US20250168313A1
- Status: Published application
- Kind: A1
- Application number: 19/029,976
- Filed: 2025-01-17
- Published: 2025-05-22
- Earliest claimed: 2013-08-28
- Assignees: Outward, Inc.
- Applicants: Outward, Inc.
- Inventors: Clarence Chui
- Web page: https://patentgrep.com/patent/US20250168313A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- G01B 11/245: using a plurality of fixed, simultaneously operating transducers ({G01B11/2408 - G01B11/2425, } G01B11/255 take precedence)
- H04N 23/60: Control of cameras or camera modules
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS

## Abstract

Techniques for generating three-dimensional content from the recordings of one or more independently operated cameras that are not constrained to fixed positions and orientations are disclosed. In some embodiments, data associated with a camera recording a scene is received, wherein camera pose with respect to the scene is not known; camera pose with respect to the scene is determined based on received image data captured by the camera and received sensor data associated with the camera; and the received data and determined camera pose are used to facilitate generation of at least a portion of a point cloud corresponding to the scene.

## Claims

### Claim 1

A method, comprising:
receiving data from a plurality of cameras configured to capture a scene;
for each of the plurality of cameras, determining a relative pose of a given camera with respect to the scene for a frame based at least in part on image data captured by that camera and sensor data associated with that camera for the frame;
determining relative poses of cameras with respect to one or more other cameras comprising the plurality of cameras for the frame based on independently determined relative poses of individual cameras with respect to the scene for the frame; and
generating at least a partial three-dimensional reconstruction of the scene for the frame based on received image data and determined relative camera poses for the frame.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of cameras is independently operated.

### Claim 3 (depends on 1)

The method of claim 1, wherein no foreknowledge exists of relative poses of cameras with respect to the scene and with respect to each other.

### Claim 4 (depends on 1)

The method of claim 1, wherein relative poses of cameras with respect to the scene and with respect to each other are not fixed and are time variant.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating at least the partial three-dimensional reconstruction of the scene for the frame comprises determining correspondences between images comprising the frame captured by the plurality of cameras.

### Claim 6 (depends on 1)

The method of claim 1, wherein generating at least the partial three-dimensional reconstruction of the scene for the frame comprises facilitating registration of images comprising the frame captured by the plurality of cameras by feature correspondence between nearest neighbor cameras.

### Claim 7 (depends on 1)

The method of claim 1, wherein generating at least the partial three-dimensional reconstruction of the scene for the frame comprises rectifying images comprising the frame captured by the plurality of cameras.

### Claim 8 (depends on 1)

The method of claim 1, wherein generating at least the partial three-dimensional reconstruction of the scene for the frame comprises estimating depths in images comprising the frame captured by the plurality of cameras.

### Claim 9 (depends on 1)

The method of claim 1, wherein generating at least the partial three-dimensional reconstruction of the scene for the frame comprises generating at least a corresponding portion of a point cloud for the frame.

### Claim 10 (depends on 1)

The method of claim 1, wherein determining relative pose of a given camera with respect to the scene for the frame comprises determining a first estimate of camera pose with respect to the scene based on image data received from the given camera and a second estimate of camera pose with respect to the scene based on sensor data received from the given camera.

### Claim 11 (depends on 10)

The method of claim 10, wherein the second estimate is employed to verify and provide a parallel estimate to the first estimate.

### Claim 12 (depends on 10)

The method of claim 10, wherein the second estimate is employed to fill gaps in pose estimation when pose cannot be determined from the first estimate.

### Claim 13 (depends on 1)

The method of claim 1, wherein determined relative pose of a given camera with respect to the scene is with respect to features or fiducials of the scene.

### Claim 14 (depends on 1)

The method of claim 1, wherein data is received, relative poses are determined, and at least the partial three-dimensional reconstruction of the scene is generated for each of a plurality of times slices or frames.

### Claim 15 (depends on 1)

The method of claim 1, wherein received image data comprises frames of a video recording of the scene.

### Claim 16 (depends on 1)

The method of claim 1, wherein generating at least the partial three-dimensional reconstruction of the scene for the frame is based on synchronizing received data from the plurality of cameras that have captured different perspectives of the scene.

### Claim 17 (depends on 1)

The method of claim 1, wherein generating at least the partial three-dimensional reconstruction of the scene for the frame is based on correspondence between sets of features seen in common among multiple cameras.

### Claim 18 (depends on 1)

The method of claim 1, wherein generating at least the partial three-dimensional reconstruction of the scene for the frame is based on establishing correspondence of features between frames of a video sequence.

### Claim 19

A system, comprising:
a processor configured to:
receive data from a plurality of cameras configured to capture a scene;
for each of the plurality of cameras, determine a relative pose of a given camera with respect to the scene for a frame based at least in part on image data captured by that camera and sensor data associated with that camera for the frame;
determine relative poses of cameras with respect to one or more other cameras comprising the plurality of cameras for the frame based on independently determined relative poses of individual cameras with respect to the scene for the frame; and
generate at least a partial three-dimensional reconstruction of the scene for the frame based on received image data and determined relative camera poses for the frame; and
a memory coupled to the processor and configured to provide instructions to the processor.

### Claim 20

A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
receiving data from a plurality of cameras configured to capture a scene;
for each of the plurality of cameras, determining a relative pose of a given camera with respect to the scene for a frame based at least in part on image data captured by that camera and sensor data associated with that camera for the frame;
determining relative poses of cameras with respect to one or more other cameras comprising the plurality of cameras for the frame based on independently determined relative poses of individual cameras with respect to the scene for the frame; and
generating at least a partial three-dimensional reconstruction of the scene for the frame based on received image data and determined relative camera poses for the frame.
