# US 2021/0240526 A1 · Full asynchronous execution queue for accelerator hardware > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0240526 A1 - Document ID: US20210240526A1 - Status: Published application - Kind: A1 - Application number: 16/861,367 - Filed: 2020-04-29 - Published: 2021-08-05 - Earliest claimed: 2020-02-05 - Assignees: NEC Laboratories Europe GmbH - Applicants: NEC Laboratories Europe GmbH - Inventors: Nicolas Weber - Web page: https://patentgrep.com/patent/US20210240526A1 ## CPC - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 9/3004: {to perform operations on memory} - G06F 9/3806: {using address prediction, e.g. return stack, branch history buffer} - G06F 9/5016: {the resource being the memory} - G06F 12/023: {Free address space management} - G06F 12/0875: with dedicated cache, e.g. instruction or stack - G06F 12/1009: using page tables, e.g. page table structures - G06F 13/4239: {with asynchronous protocol} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for providing an asynchronous execution queue for accelerator hardware includes replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer. Execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address. ## Claims ### Claim 1 A method for providing an asynchronous execution queue for accelerator hardware, the method comprising: replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer, whereby execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address. ### Claim 2 (depends on 1) The method according to claim 1, further comprising: replacing a free operation in the execution queue to be sent to the accelerator with an asynchronous free operation that uses the reference pointer and schedules a free instruction into the execution queue to be sent to the accelerator, whereby execution of the asynchronous free operation in the execution queue by the accelerator frees the corresponding memory address and removes the entry from the look-up table. ### Claim 3 (depends on 1) The method according to claim 1, wherein the reference pointer includes a reference id and an offset. ### Claim 4 (depends on 3) The method according to claim 3, wherein the reference pointer is 64 bits in which 32 bits comprise the reference id and 32 bits comprise the offset. ### Claim 5 (depends on 1) The method according to claim 1, further comprising providing an asynchronous variadic operation which copies data in a variadic argument from original code in the execution queue into a variadic arguments stack and returns a memory offset, the variadic arguments stack being copied together to the accelerator along with the execution queue. ### Claim 6 (depends on 1) The method according to claim 1, wherein the reference pointer is used by kernel calls and memcopy operations to determine the corresponding memory address using the look-up table. ### Claim 7 (depends on 1) The method according to claim 1, wherein the accelerator is decoupled from a host of the accelerator and is housed in a separate enclosure together with other accelerators, the host being remotely connected to the accelerator enclosure using external interconnects. ### Claim 8 (depends on 1) The method according to claim 1, wherein the look-up table is implemented in the accelerator by software or as an extension to a memory management unit (MMU) of the accelerator. ### Claim 9 A system comprising one or more processors which, alone or in combination, are configured to provide for execution of the following steps: replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer, whereby execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address. ### Claim 10 (depends on 9) The system according to claim 9, being further configured to provide for execution of the following steps: replacing a free operation in the execution queue to be sent to the accelerator with an asynchronous free operation that uses the reference pointer and schedules a free instruction into the execution queue to be sent to the accelerator, whereby execution of the asynchronous free operation in the execution queue by the accelerator frees the corresponding memory address and removes the entry from the look-up table. ### Claim 11 (depends on 9) The system according to claim 9, wherein the system is decoupled from the accelerator, which is housed in a separate enclosure together with other accelerators, the system being remotely connected to the accelerator enclosure using external interconnects. ### Claim 12 (depends on 9) The system according to claim 9, being configured to implement the look-up table in the accelerator using software. ### Claim 13 (depends on 9) The system according to claim 9, wherein the look-up table is an extension to a memory management unit (MMU) of the accelerator. ### Claim 14 A tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more processors, alone or in combination, provide for execution of the following steps: replacing a malloc operation in an execution queue to be sent to an accelerator with an asynchronous malloc operation that returns a unique reference pointer, whereby execution of the asynchronous malloc operation in the execution queue by the accelerator allocates a requested memory size and adds an entry to a look-up table accessible by the accelerator that maps the reference pointer to a corresponding memory address. ### Claim 15 (depends on 14) The tangible, non-transitory computer-readable medium according to claim 14, having further instructions thereon which, upon being executed by the one or more processors, alone or in combination, provide for execution of the following additional steps: replacing a free operation in the execution queue to be sent to the accelerator with an asynchronous free operation that uses the reference pointer and schedules a free instruction into the execution queue to be sent to the accelerator, whereby execution of the asynchronous free operation in the execution queue by the accelerator frees the corresponding memory address and removes the entry from the look-up table.