# US 2019/0320115 A1 · Dynamic Batch Size Selection for Vehicle Camera Image Processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320115 A1
- Document ID: US20190320115A1
- Status: Published application
- Kind: A1
- Application number: 15/950,587
- Filed: 2018-04-11
- Published: 2019-10-17
- Earliest claimed: 2018-04-11
- Assignees: GM Global Technology Operations LLC
- Applicants: GM Global Technology Operations LLC
- Inventors: Unmesh Dutta Bordoloi; Shige Wang; Stephen G. Lusko; Jinsong Wang
- Web page: https://patentgrep.com/patent/US20190320115A1

## CPC

- H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
- H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- H04N 23/62: Control of parameters via user interfaces
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- B60R 2300/105: using multiple cameras
- B60R 2300/303: using joined images, e.g. multiple camera images
- B60R 2300/607: from a bird's eye viewpoint
- B60R 2300/806: for aiding parking
- B60R 2300/8066: for monitoring rearward traffic
- B60R 2300/8093: for obstacle warning
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR

## Abstract

Examples of techniques for dynamically selecting a batch size used in vehicle camera image processing are disclosed. In one example implementation, a method includes generating, by a processing device, a batch table and a mode table. The method further includes determining, by the processing device, image processing performance requirements for a current mode of a vehicle using the mode table, the vehicle comprising a plurality of cameras configured to capture a plurality of images. The method further includes selecting, by the processing device, a batch size and a processing frequency based at least in part on the image processing performance requirements for the current mode of the vehicle. The method further includes processing, by an accelerator, at least a subset of the plurality of images based at least in part on the batch size and processing frequency.

## Claims

### Claim 1

A computer-implemented method comprising:
generating, by a processing device, a batch table and a mode table;
determining, by the processing device, image processing performance requirements for a current mode of a vehicle using the mode table, the vehicle comprising a plurality of cameras configured to capture a plurality of images;
selecting, by the processing device, a batch size and a processing frequency based at least in part on the image processing performance requirements for the current mode of the vehicle; and
processing, by an accelerator, at least a subset of the plurality of images based at least in part on the batch size and processing frequency.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
determining, by the processing device, whether the batch size is greater than a number of the plurality of cameras.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further comprising:
based at least in part on determining that the batch size is greater than the number of the plurality of cameras, setting a batch size b equal to a number of frames remaining in a current column and (b−r)% n frames from a next column, where b represents the batch size, r represents the frames remaining in a current column, and n represents a number of cameras.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, further comprising:
based at least in part on determining that the batch size is not greater than the number of the plurality of cameras, setting a batch size b equal to a number of r frames remaining in a current column and (b−r) frames in a next column, where b represents the batch size and r represents the frames remaining in a current column.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the current mode of the vehicle is selected from a group comprising a highway driving mode, an urban driving mode, a parking mode, and a degraded mode.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein processing the at least a subsect of the plurality of images is performed using a convolutional neural network.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the batch table comprises a plurality of batch sizes, each of the plurality of batch sizes having a frequency value and a latency value associated therewith.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the mode table comprises a plurality of modes of the vehicle, each of the plurality of modes having a camera configuration value and a latency requirement associated therewith.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the plurality of cameras comprises at least one surround view camera and at least one long range camera.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, further comprising:
prior to the processing, determining, by the processing device, whether a latency requirement is met based at least in part on the batch size.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, further comprising:
based at least in part on determining that the latency requirement is not met, reverting, by the processing device, to a safe batch size, wherein the safe batch size is used to perform the processing.

### Claim 12

A system comprising:
a memory comprising computer readable instructions; and
a processing device for executing the computer readable instructions for performing a method comprising:
generating, by the processing device, a batch table and a mode table;
determining, by the processing device, image processing performance requirements for a current mode of a vehicle using the mode table, the vehicle comprising a plurality of cameras configured to capture a plurality of images;
selecting, by the processing device, a batch size and a processing frequency based at least in part on the image processing performance requirements for the current mode of the vehicle; and
processing, by an accelerator, at least a subset of the plurality of images based at least in part on the batch size and processing frequency.

### Claim 13 (depends on 12)

The system of claim 12, wherein the method further comprises:
determining, by the processing device, whether the batch size is greater than a number of the plurality of cameras.

### Claim 14 (depends on 13)

The system of claim 13, wherein the method further comprises:
based at least in part on determining that the batch size is greater than the number of the plurality of cameras, setting a batch size b equal to a number of frames remaining in a current column and (b−r)% n frames from a next column, where b represents the batch size, r represents the frames remaining in a current column, and n represents a number of cameras.

### Claim 15 (depends on 13)

The system of claim 13, wherein the method further comprises:
based at least in part on determining that the batch size is not greater than the number of the plurality of cameras, setting a batch size b equal to a number of r frames remaining in a current column and (b−r) frames in a next column, where b represents the batch size and r represents the frames remaining in a current column.

### Claim 16 (depends on 12)

The system of claim 12, wherein the current mode of the vehicle is selected from the group consisting of a highway driving mode, an urban driving mode, a parking mode, and a degraded mode.

### Claim 17 (depends on 12)

The system of claim 12, wherein processing the at least a subsect of the plurality of images is performed using a convolutional neural network.

### Claim 18 (depends on 12)

The system of claim 12, wherein the batch table comprises a plurality of batch sizes, each of the plurality of batch sizes having a frequency value and a latency value associated therewith.

### Claim 19 (depends on 12)

The system of claim 12, wherein the mode table comprises a plurality of modes of the vehicle, each of the plurality of modes having a camera configuration value and a latency requirement associated therewith.

### Claim 20

A computer program product comprising:
a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processing device to cause the processing device to perform a method comprising:
generating, by the processing device, a batch table and a mode table;
determining, by the processing device, image processing performance requirements for a current mode of a vehicle using the mode table, the vehicle comprising a plurality of cameras configured to capture a plurality of images;
selecting, by the processing device, a batch size and a processing frequency based at least in part on the image processing performance requirements for the current mode of the vehicle; and
processing, by an accelerator, at least a subset of the plurality of images based at least in part on the batch size and processing frequency.
