# US 11,907,144 B1 · Early semaphore update > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,144 B1 - Document ID: US11907144B1 - Status: Granted - Kind: B1 - Application number: 17/805,410 - Filed: 2022-06-03 - Granted: 2024-02-20 - Earliest claimed: 2022-06-03 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Raymond S. Whiteside; Thomas A. Volpe - Web page: https://patentgrep.com/patent/US11907144B1 ## CPC - G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence) - G06F 2213/28: DMA - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Techniques to reduce the latency in notifying that space in a memory has been freed up are described. For example, when moving data from on-chip memory of a computing engine to system memory, the computing engine can be notified that its on-chip memory is free before an acknowledgment is provided by the system memory that the data being moved has been written into the system memory. The computing engine can be given access to the on-chip memory sooner by generating an early semaphore update based on a determination that the set of data being moved to system memory has been read out from the on-chip memory. The early semaphore update need not wait for the acknowledgement from the system memory, thus reducing the latency of notifying the computing engine that the on-chip memory is free. ## Claims ### Claim 1 A system comprising: system memory operable to: store a set of data; and generate a respective acknowledgement to indicate that a corresponding set of data has been stored in the system memory; an integrated circuit device including: an acceleration circuit; an on-chip memory; and a semaphore that is written to control operation of the acceleration circuit; and a direct memory access (DMA) engine including: a read descriptor queue operable to store read descriptors; a write descriptor queue operable to store write descriptors; and a controller operable to: process the read descriptors to obtain the set of data from the on-chip memory for storage into the system memory; insert a first semaphore update without a memory barrier in a processing flow of the DMA engine to indicate that the on-chip memory is free to store a new set of data before the set of data has been completely written into the system memory; process the write descriptors to write the set of data into the system memory; receive an acknowledgement from the system memory that the set of data has been stored in the system memory; and perform a second semaphore update with a memory barrier to indicate that the set of data is available in the system memory in response to the acknowledgement. ### Claim 2 (depends on 1) The system of claim 1, wherein the first semaphore update is inserted by inserting a first semaphore update descriptor in the write descriptor queue between a last write descriptor of the write descriptors for writing the set of data and a second semaphore update descriptor corresponding to the second semaphore update. ### Claim 3 (depends on 1) The system of claim 1, wherein the first semaphore update is inserted by inserting a first semaphore update descriptor in a separate queue which is triggered by a tail pointer increment after processing a last read descriptor of the read descriptors for reading the set of data. ### Claim 4 (depends on 1) The system of claim 1, wherein the DMA engine includes a semaphore update generator circuit, and wherein the first semaphore update is inserted by configuring the semaphore update generator circuit to insert a semaphore update transaction at an output interface of the DMA engine in response to detecting that a last data element of the set of data has been read. ### Claim 5 A computer-implemented method comprising: reading a set of data stored in an on-chip memory of an integrated circuit device to be written into a system memory; inserting a first semaphore update in a processing flow of the integrated circuit device to indicate that the on-chip memory is free to store a new set of data; writing the set of data read from the on-chip memory to the system memory; receiving an acknowledgement from the system memory that the set of data has been stored in the system memory; and in response to the acknowledgement, performing a second semaphore update to indicate that the set of data is available in the system memory. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein the first semaphore update is performed by a direct memory access (DMA) engine based on a first semaphore update descriptor without a memory barrier, and the second semaphore update is performed by the DMA engine based on a second semaphore update descriptor with a memory barrier. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, wherein reading the set of data stored in the on-chip memory is performed by the DMA engine based on a set of read descriptors stored in a read descriptor queue, and wherein writing the set of data to the system memory is performed based on a set of write descriptors in a write descriptor queue. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein inserting the first semaphore update includes inserting the first semaphore update descriptor in the write descriptor queue between a last write descriptor of the set of write descriptors for writing the set of data and the second semaphore update descriptor. ### Claim 9 (depends on 7) The computer-implemented method of claim 7, wherein inserting the first semaphore update includes inserting the first semaphore update descriptor in a separate queue which is triggered by a tail pointer increment after processing a last read descriptor of the set of read descriptors for reading the set of data. ### Claim 10 (depends on 5) The computer-implemented method of claim 5, wherein the first semaphore update is inserted by a semaphore update generator circuit in a DMA engine, the semaphore update generator circuit inserting a semaphore update transaction at an output interface of the DMA engine in response to detecting a last data element of the set of data has been read. ### Claim 11 (depends on 10) The computer-implemented method of claim 10, wherein the semaphore update generator circuit determines that the last data element of the set of data has been read out from the on-chip memory based on snooping an address associated with a last read descriptor on an input interface. ### Claim 12 (depends on 5) The computer-implemented method of claim 5, wherein the first semaphore update and the second semaphore update increment a same semaphore location in the integrated circuit device. ### Claim 13 (depends on 5) The computer-implemented method of claim 5, wherein the first semaphore update and the second semaphore update increment different semaphore locations in the integrated circuit device. ### Claim 14 (depends on 13) The computer-implemented method of claim 13, wherein the integrated circuit device is a neural network processor, and the on-chip memory is a state buffer of the neural network processor. ### Claim 15 (depends on 14) The computer-implemented method of claim 14, wherein the set of data includes weight values or feature map values associated with a neural network model. ### Claim 16 A direct memory access (DMA) engine comprising: a read descriptor queue operable to store read descriptors; a write descriptor queue operable to store write descriptors; an input interface operable to read data based on the read descriptors; an output interface operable to write data based on the write descriptors; and a controller operable to: process the read descriptors to obtain a set of data from an on-chip memory of an integrated circuit device; insert a first semaphore update in a processing flow of the DMA engine to indicate that the on-chip memory is free to store a new set of data; process the write descriptors to write the set of data read from the on-chip memory to system memory; receive an acknowledgement from the system memory that the set of data has been stored in the system memory; and perform a second semaphore update to indicate that the set of data is available in the system memory in response to the acknowledgement. ### Claim 17 (depends on 16) The DMA engine of claim 16, wherein the first semaphore update is inserted by processing a first semaphore update descriptor placed in the write descriptor queue between a last write descriptor of the write descriptors for writing the set of data and a second semaphore update descriptor corresponding to the second semaphore update. ### Claim 18 (depends on 16) The DMA engine of claim 16, wherein the first semaphore update is inserted by processing a first semaphore update descriptor in a separate queue which is triggered by a tail pointer increment after processing a last read descriptor of the read descriptors for reading the set of data. ### Claim 19 (depends on 16) The DMA engine of claim 16, further comprising: a semaphore update generator circuit, wherein the first semaphore update is inserted by configuring the semaphore update generator circuit to insert a semaphore update transaction at the output interface in response to detecting that a last data element of the set of data has been read. ### Claim 20 (depends on 16) The DMA engine of claim 16, wherein the first semaphore update and the second semaphore update increment a same semaphore location.