# US 2025/0148644 A1 · Dynamic generation of data sets for training machine-trained network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148644 A1
- Document ID: US20250148644A1
- Status: Published application
- Kind: A1
- Application number: 18/737,459
- Filed: 2024-06-07
- Published: 2025-05-08
- Earliest claimed: 2017-12-29
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Andrew C. Mihal; Steven L. Teig
- Web page: https://patentgrep.com/patent/US20250148644A1

## CPC

- G06T 7/85: {Stereo camera calibration}
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 7/55: from multiple images
- G06T 7/593: from stereo images
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08)
- H04N 13/128: Adjusting depth or disparity
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 13/243: using three or more two-dimensional [2D] image sensors
- G06N 3/048: Activation functions
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30261: Obstacle
- H04N 13/246: Calibration of cameras
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Some embodiments of the invention provide a novel method for training a multi-layer node network. Some embodiments train the multi-layer network using a set of inputs generated with random misalignments incorporated into the training data set. In some embodiments, the training data set is a synthetically generated training set based on a three-dimensional ground truth model as it would be sensed by a sensor array from different positions and with different deviations from ideal alignment and placement. Some embodiments dynamically generate training data sets when a determination is made that more training is required. Training data sets, in some embodiments, are generated based on training data sets for which the multi-layer node network has produced bad results.

## Claims

### Claim 25

A method comprising:
capturing first image data comprising a first image captured by a first image capture device and a second image captured by a second image capture device;
computing, using the first image data and a machine-trained (MT) network, first misalignment data, wherein the first misalignment data represents a first misalignment of the first image capture device;
computing, using the first image data and the first misalignment data, first output data comprising a first depth value for a first object; and
storing the first depth value in memory.

### Claim 26 (depends on 25)

The method of claim 25, wherein the first misalignment data comprises a first vector comprising distance data representing a distance of a first image capture device from a prior position of the first image capture device.

### Claim 27 (depends on 25)

The method of claim 25, wherein the first misalignment data comprises a second vector comprising orientation data, wherein the orientation data comprises quaternion data representing a change in orientation of the first image capture device from a prior orientation of the first image capture device.

### Claim 28 (depends on 25)

The method of claim 25, the method comprising:
computing, using the first image data, first disparity data representing a pixel shift between a first location of the first object in the first image and a second location of the first object in the second image.

### Claim 29 (depends on 28)

The method of claim 28, wherein computing the first output data comprises:
computing, using the first misalignment data, inter-device alignment data comprising distance data representing a distance between the first image capture device and the second image capture device;
computing, using the first disparity data and the inter-device alignment data, the first output data.

### Claim 30 (depends on 25)

The method of claim 25, comprising:
capturing second image data comprising a third image captured by the first image capture device and a fourth image captured by the second image capture device;
computing, using the second image data and the first misalignment data, second output data, wherein the second output data comprises at least a second depth value for the first object; and
computing, using the first output data and the second output data, a position and speed of the first object.

### Claim 31 (depends on 30)

The method of claim 30, comprising:
executing, in response to the position and the speed of the first object, a collision avoidance operation, the collision avoidance operation including at least one of (i) outputting a warning, (ii) automatic braking, or (iii) automatic steering.

### Claim 32 (depends on 31)

The method of claim 31, comprising:
triggering a misalignment data update for the first image capture device based on configuration data specifying trigger events for misalignment data update, wherein updating misalignment data comprises:
capturing, second image data comprising third image captured by the first image capture device and a fourth image captured by the second image capture device; and
computing, using the second image data and the MT network, second misalignment data.

### Claim 33 (depends on 32)

The method of claim 32, comprising:
determining gyroscopic data;
determining acceleration data;
detecting, based on the gyroscopic data and acceleration data, an impact; and
triggering, based on the impact and the configuration data, an update of the misalignment data.

### Claim 34 (depends on 32)

The method of claim 32, comprising:
determining a number of times that depth information has been determined using the first misalignment data; and
triggering, based on the number of times that depth information has been determined using the first misalignment data and the configuration data, an update of the misalignment data.

### Claim 35 (depends on 32)

The method of claim 32, comprising:
capturing ambient condition data;
determining, based on the ambient condition data, a change in ambient conditions; and
triggering, based on the change in ambient conditions and the configuration data, an update of the misalignment data.

### Claim 36

A system comprising:
one or more processors;
a plurality of image capture devices;
one or more computer-readable media, the one or more computer-readable media storing processor-executable instructions which, when executed using the one or more processors, perform operations comprising:
capturing, first image data comprising a first image captured by a first image capture device and a second image captured by a second image capture device;
computing, using the first image data and a machine-trained (MT) network, first misalignment data, wherein the first misalignment data represents a first misalignment of the first image capture device;
computing, using the first image data and the first misalignment data, first output data comprising a first depth value for a first object; and
storing the first depth value in memory.

### Claim 37 (depends on 36)

The system of claim 36, comprising a pre-processor, the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
generating, using the pre-processor and the first image data, a cropped first image and a cropped second image, wherein the first image data is not in a correct format for computations using the MT network.

### Claim 38 (depends on 36)

The system of claim 36, the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
computing, using the first image data, first disparity data representing a pixel shift between a first location of the first object in the first image and a second location of the first object in the second image.

### Claim 39 (depends on 38)

The system of claim 38, the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
computing, using the first misalignment data, inter-device alignment data comprising distance data representing a distance between the first image capture device and the second image capture device; and
computing, using the first disparity data and the inter-device alignment data, the first output data.

### Claim 40 (depends on 36)

The system of claim 36, comprising an orientation sensor, the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
determining, using the orientation sensor, gyroscopic data;
detecting, based on the gyroscopic data, an impact; and
triggering, based on the impact, an update of the first misalignment data.

### Claim 41 (depends on 36)

The system of claim 36, comprising an accelerometer, the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
determining, using the accelerometer, acceleration data;
detecting, based on the acceleration data, an impact; and
triggering, based on the impact, an update of the first misalignment data.

### Claim 42 (depends on 36)

The system of claim 36, comprising a sensor array comprising a plurality of image capture devices, wherein the first image capture device and second image capture device positions are determined relative to a center point of the sensor array.

### Claim 43

A computer program product comprising at least one non-transitory computer-readable storage medium storing software instructions that, when executed, cause a system to:
capture first image data comprising a first image captured by a first image capture device and a second image captured by a second capture device;
compute, using the first image data and a machine-trained (MT) network, first misalignment data, wherein the first misalignment data represents a first misalignment of the first image capture device;
compute, using the first image data and the first misalignment data, first output data comprising a first depth value for a first object; and
store the first depth value in memory.

### Claim 44 (depends on 43)

The computer program product of claim 43, the at least one non-transitory computer-readable storage medium storing software instructions that, when executed, cause the system to:
trigger a misalignment data update for the first image capture device based on configuration data specifying trigger events for misalignment data update, wherein updating misalignment data causes the system to:
capture, second image data comprising third image captured by the first image capture device and a fourth image captured by the second capture device; and
compute, using the second image data and the MT network, second misalignment data.
