# US 2026/0030712 A1 · Sub-image streaming and processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0030712 A1
- Document ID: US20260030712A1
- Status: Published application
- Kind: A1
- Application number: 18/787,408
- Filed: 2024-07-29
- Published: 2026-01-29
- Earliest claimed: 2024-07-29
- Assignees: Mellanox Technologies, Ltd.
- Applicants: Mellanox Technologies, Ltd.
- Inventors: Dotan David Levi; Yechiel Wiesel; Wojciech Wawrzyniec Wasko; Natan Manevich; Elad Mentovich
- Web page: https://patentgrep.com/patent/US20260030712A1

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- H04N 19/127: Prioritisation of hardware or computational resources
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/42: characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Systems and methods herein are for distributed image processing by at least a data processing unit (DPU) and using at least a graphics processing unit (GPU) in possible association with a field programmable gate array (FPGA). For example, the FPGA may be used to perform physical layer processing for images captured by the image sensor or from a simulation and can provide a media stream for the DPU and the DPU can provide payload of only image sections from the images in a media stream for the GPU to perform content layer processing for only the image sections of the images.

## Claims

### Claim 1

A system comprising a data processing unit (DPU) to receive image data associated with captured images of at least one media stream, the image data comprising payload and headers, the DPU further to separate the headers from the payload, to provide sequence numbers for the payload to represent only image sections of the images, and to provide only the image sections in a shared buffer for access by a graphics processing unit (GPU) to enable the GPU to process the payload representing only the image sections using the sequence numbers.

### Claim 2 (depends on 1)

The system of claim 1, further comprising:
an image sensor to capture the images; and
a field programmable gate array (FPGA) to receive the images and to provide the image data as concurrent media streams of the at least one media stream.

### Claim 3 (depends on 2)

The system of claim 2, wherein the DPU is further to store the headers for the payload of a first one of the concurrent media stream and additional headers for additional payload of a second one of the concurrent media streams in different ones of a plurality of buffers that are distinct from the shared buffer, wherein the DPU is further to arrange the payload and the additional payload, belonging to the image sections and to additional image sections of the images, in contiguous ones of the designated locations of the shared buffer, and wherein the arrangement of the payload and the additional payload enables the GPU to stitch the image sections and the additional image sections together for use by at least one application or for further processing by the GPU.

### Claim 4 (depends on 3)

The system of claim 3, wherein the shared buffer is local to the GPU and wherein the plurality of buffers are local to a central processing unit (CPU) of a host machine or the DPU.

### Claim 5 (depends on 3)

The system of claim 3, wherein the shared buffer is on a GPU card which comprises the GPU and wherein the plurality of buffers are in the host machine or the DPU.

### Claim 6 (depends on 3)

The system of claim 3, wherein the shared buffer is on an accelerator card or a converged card which comprises the GPU and the DPU and wherein the plurality of buffers are in the host machine or the DPU.

### Claim 7 (depends on 3)

The system of claim 3, wherein the DPU is further to discard other payload that are other than the image sections following the arrangement of the payload representing only the image sections for the GPU.

### Claim 8 (depends on 2)

The system of claim 2, wherein the image sensor comprises a multi-array sensor, and wherein different sensors of the multi-array sensor provide different and concurrent media streams of the at least one media stream.

### Claim 9 (depends on 2)

The system of claim 2, wherein the image sensor is further to communicate concurrent media streams of the at least one media stream to the FPGA, and wherein the concurrent media streams are associated with different User Datagram Protocol (UDP) ports of the FPGA.

### Claim 10 (depends on 1)

The system of claim 1, wherein the at least one media stream includes two media streams, wherein the headers are associated with a first one of the two media streams and are provided in a first buffer for access by a host machine comprising a central processing unit (CPU), wherein additional headers are associated with a second one of the two media streams, and wherein the additional headers are provided for in a second buffer for separate access, relative to the headers associated with the first one of the two media streams, by the host machine.

### Claim 11 (depends on 8)

The system of claim 8, wherein the CPU or the GPU is to use information from an application to inform the DPU of the payload representing only the image sections of the images to be provided by the DPU to the GPU.

### Claim 12 (depends on 1)

The system of claim 1, wherein the at least one media stream includes two media streams, wherein the payload are associated with a first one of the two media streams, representing only the image sections of the first one of the two media streams, and are provided in the shared buffer for access by the GPU, and wherein additional payload are associated with a second one of the two media streams, representing only additional image sections of the second one of the two media streams, and are provided in the shared buffer or a different buffer for contiguous access, with the payload associated with the first one of the two media streams, by the GPU.

### Claim 13 (depends on 1)

The system of claim 1, further comprising a central processing unit (CPU) of a host machine, the CPU to use information from an application to cause the GPU to process the payload representing only the image sections of the images.

### Claim 14 (depends on 1)

The system of claim 1, wherein the headers are received for local access using a central processing unit (CPU) of a host machine, wherein the CPU enables the DPU to provide the payload representing only the image sections for local access by the GPU, and wherein the CPU enables the GPU to process the payload representing only the image sections based in part on the sequence numbers.

### Claim 15 (depends on 1)

The system of claim 1, wherein the DPU is to control the provision of the payload over a stream bit rate and burst size which are associated with predictable workloads at a known consumption rate for the GPU.

### Claim 16

A plurality of circuits comprising at least a data processing unit (DPU) to receive image data associated with captured images of at least one media stream, the image data comprising payload and headers, the DPU further to separate the headers from the payload, to provide sequence numbers for the payload to represent only image sections of the images, and to provide only the image sections in a shared buffer for access by a graphics processing unit (GPU) to enable the GPU to process the payload representing only the image sections using the sequence numbers.

### Claim 17 (depends on 16)

The plurality of circuits of claim 16, further comprising:
an image sensor to capture the images, the image sensor comprising a multi-array sensor, wherein different sensors of the multi-array sensor provide different and concurrent media streams of the at least one media stream; and
a field programmable gate array (FPGA) to receive the images and to provide the image data as the different and concurrent media streams of the at least one media stream.

### Claim 18 (depends on 16)

The plurality of circuits of claim 16, wherein the shared buffer is one of: local to the GPU, on a GPU card which comprises the GPU, or on an accelerator card or a converged card which comprises the GPU and the DPU.

### Claim 19 (depends on 16)

The plurality of circuits of claim 16, wherein the headers are to be stored in a first one of a plurality of buffers, wherein the plurality of buffers are distinct from the shared buffers and are local to a central processing unit (CPU) of a host machine or the DPU or are in the host machine or the DPU, wherein the headers are for the payload of a first one of concurrent media streams of the at least one media stream, and wherein additional headers for additional payload of a second one of the concurrent media streams are in second one of a plurality of buffers.

### Claim 20

A method for image processing comprising:
receiving, in a data processing unit (DPU) image data associated with captured images of at least one media stream, the image data comprising payload and headers,
separating, by the DPU, the headers from the payload;
providing sequence numbers for the payload to represent only image sections of the images; and
providing only the image sections in a shared buffer for access by a graphics processing unit (GPU) to enable the GPU to process the payload representing only the image sections using the sequence numbers.

### Claim 21 (depends on 20)

The method of claim 20, further comprising:
capturing the images using an image sensor, the image sensor comprising a multi-array sensor, and wherein different sensors of the multi-array sensor provide different and concurrent media streams of the at least one media stream;
receiving the images in a field programmable gate array (FPGA); and
providing, from the FPGA, the image data as the different and concurrent media streams of the at least one media stream.

### Claim 22 (depends on 20)

The method of claim 20, wherein the shared buffer is one of: local to the GPU, on a GPU card which comprises the GPU, or on an accelerator card or a converged card which comprises the GPU and the DPU.

### Claim 23 (depends on 20)

The method of claim 20, further comprising:
storing, by the DPU, the headers for the payload of a first one of the concurrent media stream and additional headers for additional payload of a second one of the concurrent media streams in different ones of a plurality of buffers that are distinct from the shared buffer;
arranging, by the DPU the payload and the additional payload, belonging to the image sections and to additional image sections of the images, in contiguous ones of the designated locations of the shared buffer; and
enabling, using the arrangement of the payload and the additional payload, the GPU to stitch the image sections and the additional image sections together for use by at least one application or for further processing by the GPU.
