# US 2017/0256016 A1 · Hardware architecture for acceleration of computer vision and imaging processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2017/0256016 A1 - Document ID: US20170256016A1 - Status: Published application - Kind: A1 - Application number: 15/059,175 - Filed: 2016-03-02 - Published: 2017-09-07 - Earliest claimed: 2016-03-02 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd - Inventors: Seungjin Lee; Seok-Jun Lee - Web page: https://patentgrep.com/patent/US20170256016A1 ## CPC - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06F 9/544: {Buffers; Shared memory; Pipes} - G06T 1/60: Memory management - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An image and vision processing architecture included a plurality of image processing hardware accelerators each configured to perform a different one of a plurality of image processing operations on image data. A multi-port memory shared by the hardware accelerators stores the image data and is configurably coupled by a sparse crossbar interconnect to one or more of the hardware accelerators depending on a use case employed. The interconnect processes accesses of the image data by the hardware accelerators. Two or more of the hardware accelerators are chained to operate in sequence in a first order for a first use case, and at least one of the hardware accelerators is set to operate for a second use case. Portions of the memory are allocated to the hardware accelerators based on the use case employed, with an allocated portion of the memory configured as a circular buffer. ## Claims ### Claim 1 A system, comprising: a plurality of image processing hardware accelerators each configured to perform a different one of a plurality of image processing operations on image data; a multi-port memory shared by the plurality of image processing hardware accelerators and configured to store the image data; and an interconnect configurably coupling the memory to one or more of the image processing hardware accelerators and configured to process accesses of the image data by the image processing hardware accelerators, wherein two or more of the image processing hardware accelerators are configurably chained to operate in sequence in a first order for a first use case for the image processing hardware accelerators, and at least one of the image processing hardware accelerators is configurably set to operate for a second use case for the image processing hardware accelerators, wherein at least one of the image processing hardware accelerators is configurably allocated a portion of the memory based on whether the first use case or the second use case is configured, the allocated portion of the memory configured as a circular buffer, and wherein the interconnect includes a head pointer bus configured to carry a head pointer for the circular buffer and a tail pointer bus configured to carry a tail pointer for the circular buffer. ### Claim 2 (depends on 1) The system according to claim 1, wherein different combinations of the image processing hardware accelerators are configurably chained to operate in sequence in different orders for different use cases for the image processing hardware accelerators. ### Claim 3 (depends on 2) The system according to claim 2, wherein each of the combinations of the image processing hardware accelerators is configurably chained by allocation of portions of the shared memory as buffers between consecutive ones of the image processing hardware accelerators in the corresponding sequence for the respective one of the use cases. ### Claim 4 (depends on 1) The system according to claim 1, wherein the shared memory comprises multiple, concurrently accessible static random access memory (SRAM) banks configured to be allocated in portions to each of the one or more image processing hardware accelerators employed in a current use case. ### Claim 5 (depends on 4) The system according to claim 4, wherein portions of at least two of the SRAM banks are allocated as one of an input buffer or an output buffer for each of the image processing hardware accelerators employed in the current use case. ### Claim 6 (depends on 4) The system according to claim 4, wherein the interconnect is configured to connect a read port for one of the image processing hardware accelerators employed in the current use case to one or more of the SRAM banks for memory allocated to store image data on which the one image processing hardware accelerator is to operate and to connect a write port for the one image processing hardware accelerators to one or more of the SRAM banks for memory allocated to store image data produced by operation of the one image processing hardware accelerator. ### Claim 7 (depends on 1) The system according to claim 1, wherein a portion of the shared memory allocated to at least one of the image processing hardware accelerators employed in a current use case is configured as a tile buffer. ### Claim 8 (depends on 1) The system according to claim 1, wherein the interconnect is a sparse crossbar. ### Claim 9 An image processing circuit for processing a plurality of image streams, the image processing circuit comprising: a plurality of hardware accelerators, each hardware accelerator of the plurality of hardware accelerators having an input and an output; a multi-ported memory subsystem configured as a shared circular buffer and comprising an interconnect element configured to enable a stream at an output of any first one of the hardware accelerators to be coupled to an input of any second one of the hardware accelerators; a plurality of circular buffer write adapters configured to convert incoming image streams into circular buffer data; a plurality of circular buffer read adapters configured to convert circular buffer data into outgoing data streams; and at least one bus synchronizing data within the circular buffer across the plurality of circular buffer write adapters and the plurality of circular buffer read adapters. ### Claim 10 (depends on 9) The image processing circuit of claim 9, wherein the multi-port memory subsystem comprises multiple, concurrently accessible static random access memory (SRAM) banks configured to be connected to each of the plurality of circular buffer write adapters and each of the plurality of circular buffer read adapters. ### Claim 11 (depends on 9) The image processing circuit of claim 9, wherein the at least one bus comprises: a write status bus configured to allow one of the circular buffer write adapters to broadcast a write status message to all of the circular circuit buffer read adapters. ### Claim 12 (depends on 9) The image processing circuit of claim 9, wherein the at least one bus comprises: a read status bus configured to allow one of the circular buffer read adapters to broadcast a read status message to each of the circular buffer read adapters. ### Claim 13 (depends on 9) The image processing circuit of claim 9, wherein each of the circular buffer write adapters comprises: a write status transmitter configured to transmit write status updates with a programmable writer identifier at programmable time intervals. ### Claim 14 (depends on 9) The image processing circuit of claim 9, wherein each of the circular buffer write adapters comprises: a read status receiver configured to receive read status updates from a read status bus, and, when a reader identifier of a received read status update matches one of multiple programmed reader identifiers, store information for the read status update in a local tail pointer register corresponding to the matched reader identifier. ### Claim 15 (depends on 9) The image processing circuit of claim 9, wherein each of the circular buffer read adapters comprises: a read status transmitter configured to transmit read status updates with a programmable reader identifier at programmable time intervals. ### Claim 16 A method, comprising: for a plurality of image processing hardware accelerators each configured to perform a different one of a plurality of image processing operations on image data, one of chaining two or more of the image processing hardware accelerators to operate in sequence in a first order for a first use case for the image processing hardware accelerators, and setting at least one of the image processing hardware accelerators to operate for a second use case for the image processing hardware accelerators; for a multi-port memory shared by the plurality of image processing hardware accelerators and configured to store the image data, configuring a portion of the shared memory allocated to one of the image processing hardware accelerators as a circular buffer based on whether the first use case or the second use case is employed; and configuring an interconnect coupling the memory to one or more of the image processing hardware accelerators to process accesses of the image data by the image processing hardware accelerators, wherein the interconnect includes a head pointer bus configured to carry a head pointer for the circular buffer and a tail pointer bus configured to carry a tail pointer for the circular buffer. ### Claim 17 (depends on 16) The method according to claim 16, wherein different combinations of the image processing hardware accelerators are configurably chained to operate in sequence in different orders for different use cases for the image processing hardware accelerators. ### Claim 18 (depends on 17) The method according to claim 17, wherein each of the combinations of the image processing hardware accelerators is configurably chained by allocation of portions of the shared memory as buffers between consecutive ones of the image processing hardware accelerators in the corresponding sequence for the respective one of the use cases. ### Claim 19 (depends on 17) The method according to claim 17, wherein the shared memory comprises multiple, concurrently accessible static random access memory (SRAM) banks configured to be allocated in portions to each of the one or more image processing hardware accelerators employed in a current use case. ### Claim 20 (depends on 19) The method according to claim 19, wherein portions of at least two of the SRAM banks are allocated as one of an input buffer or an output buffer for each of the image processing hardware accelerators employed in the current use case. ### Claim 21 (depends on 19) The method according to claim 19, wherein the interconnect is configured to connect a read port for one of the image processing hardware accelerators employed in the current use case to one or more of the SRAM banks for memory allocated to store image data on which the one image processing hardware accelerator is to operate and to connect a write port for the one image processing hardware accelerators to one or more of the SRAM banks for memory allocated to store image data produced by operation of the one image processing hardware accelerator. ### Claim 22 (depends on 16) The method according to claim 16, wherein a portion of the shared memory allocated to at least one of the image processing hardware accelerators employed in a current use case is configured as a tile buffer. ### Claim 23 (depends on 16) The method according to claim 16, wherein the interconnect is a sparse crossbar.