# US 12,111,778 B2 · Image processing accelerator

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,111,778 B2
- Document ID: US12111778B2
- Status: Granted
- Kind: B2
- Application number: 17/558,252
- Filed: 2021-12-21
- Published: 2022-04-14
- Granted: 2024-10-08
- Earliest claimed: 2018-12-27
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Mihir Mody; Niraj Nandan; Hetul Sanghvi; Brian Chae; Rajasekhar Reddy Allu; Jason A. T. Jones; Anthony Lell; Anish Reghunath
- Web page: https://patentgrep.com/patent/US12111778B2

## CPC

- G06F 13/1668: {Details of memory controller}
- G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence)
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- H04N 5/765: Interface circuits between an apparatus for recording and another apparatus
- H04N 5/77: between a recording apparatus and a television camera
- H04N 9/79: Processing of colour television signals in connection with recording
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

A processing accelerator includes a shared memory, and a stream accelerator, a memory-to-memory accelerator, and a common DMA controller coupled to the shared memory. The stream accelerator is configured to process a real-time data stream, and to store stream accelerator output data generated by processing the real-time data stream in the shared memory. The memory-to-memory accelerator is configured to retrieve input data from the shared memory, to process the input data, and to store, in the shared memory, memory-to-memory accelerator output data generated by processing the input data. The common DMA controller is configured to retrieve stream accelerator output data from the shared memory and transfer the stream accelerator output data to memory external to the processing accelerator; and to retrieve the memory-to-memory accelerator output data from the shared memory and transfer the memory-to-memory accelerator output data to memory external to the processing accelerator.

## Claims

### Claim 1

A processing accelerator, comprising:
a thread scheduler;
a stream accelerator coupled to the thread scheduler;
a shared memory; and
a memory controller coupled to the shared memory;
wherein the stream accelerator includes processing circuitry, and a load/store engine coupled to the processing circuitry, the load/store engine including a buffer, and shared memory access circuitry.

### Claim 2 (depends on 1)

The processing accelerator of claim 1, further comprising:
a memory-to-memory accelerator.

### Claim 3 (depends on 1)

The processing accelerator of claim 1, wherein:
the stream accelerator is configured to process a line of data as the line of data is received by the stream accelerator.

### Claim 4 (depends on 2)

The processing accelerator of claim 2, wherein:
the memory-to-memory accelerator is configured to process a two-dimensional block of data from the shared memory.

### Claim 5 (depends on 1)

The processing accelerator of claim 1, wherein:
the processing circuitry of the stream accelerator includes:
a first processing circuit including a first input coupled to the load/store engine, and a first output;
a second processing circuit including a second input coupled to the first output, and a second output; and
a third processing circuit including a third input coupled to the second output, and a third output coupled to the load/store engine.

### Claim 6 (depends on 1)

The processing accelerator of claim 1, further comprising:
a block processing accelerator.

### Claim 7 (depends on 1)

The processing accelerator of claim 1, wherein:
the shared memory access circuitry includes:
arbitration circuitry;
line/block conversion circuitry; and
virtual line conversion circuitry.

### Claim 8 (depends on 7)

The processing accelerator of claim 7, wherein:
the arbitration circuitry is configured to arbitrate access of load/store engine to the shared memory.

### Claim 9 (depends on 7)

The processing accelerator of claim 7, wherein:
the line/block conversion circuitry is configured to convert a line of data to a block of data.

### Claim 10 (depends on 7)

The processing accelerator of claim 7, wherein:
the line/block conversion circuitry is configured to convert a block of data to a line of data.

### Claim 11 (depends on 7)

The processing accelerator of claim 7, wherein:
the virtual line conversion circuitry is configured to partition a data stream into a set of data lines each having a length; and
the length is determined based on an efficiency of processing by the hardware accelerator.

### Claim 12 (depends on 1)

The processing accelerator of claim 1, wherein:
the shared memory includes a depth that is configurable.

### Claim 13 (depends on 12)

The processing accelerator of claim 12, wherein:
the shared memory is configured based on configuration information received from a processor.

### Claim 14 (depends on 1)

The processing accelerator of claim 1, wherein:
the memory controller is configured based on configuration information received from a processor.

### Claim 15 (depends on 14)

The processing accelerator of claim 14, wherein:
the configuration information includes one of transfer destination and length.

### Claim 16 (depends on 1)

The processing accelerator of claim 1, wherein:
the hardware accelerator is one of a plurality of hardware accelerators; and
each of the plurality of hardware accelerators functions independently.

### Claim 17

A processing accelerator, comprising:
a thread scheduler;
a hardware accelerator coupled to the thread scheduler;
a shared memory having a depth that is configurable; and
a memory controller coupled to the shared memory;
wherein the depth of the shared memory is configurable based on one of a size of data, a format of data, and a transfer latency.

### Claim 18

A method, comprising:
receiving, by a first hardware accelerator, a first set of data;
processing, by the first hardware accelerator, the first set of data to generate a first output data;
storing, by the first hardware accelerator, the first output data in a shared memory coupled to a second hardware accelerator;
retrieving, by the second hardware accelerator, the first output data from the shared memory;
processing, by the second hardware accelerator, the first output data to produce a second output data;
storing, by the second accelerator, the second output data in the shared memory; and
synchronizing, by a scheduler coupled to the first hardware accelerator and the second hardware accelerator, the retrieving of the first output data stored in the shared memory by the second hardware accelerator based on availability of the first output data in the shared memory.

### Claim 19 (depends on 18)

The method of claim 18, further comprising:
initiating processing, by the scheduler, of a next set of data by the first hardware accelerator on completion of processing of the first set of data by the first hardware accelerator.

### Claim 20

A method, comprising:
receiving by a first hardware accelerator, a first set of data;
processing, by the first hardware accelerator, the first set of data to generate first output data;
storing, by the first hardware accelerator, the first output data in a shared memory coupled to a second hardware accelerator;
retrieving, by the second hardware accelerator, the first output data from the shared memory;
processing, by the second hardware accelerator, the first output data to produce second output data; and
storing, by the second accelerator, the second output data in the shared memory;
wherein the shared memory includes a first variable depth circular buffer accessible by the first hardware accelerator; and
wherein the shared memory includes a second variable depth circular buffer accessible by the second hardware accelerator.