Back to results
US 12,608,225 B2Granted
Hierarchical programming model for artificial intelligence hardware
AssigneeMicrosoft Technology Licensing, LLC
InventorsHaishan Zhu, Eric S. Chung
Life of this patent
Estimating the term…
Abstract
A system includes a set of lower-level control threads. The system also includes a higher-level control thread configured to receive a command from a device, generate a set of commands based on the command, and provide the set of commands to a subset of the set of lower-level control threads. A lower-level control thread in the subset of the set of lower-level control threads is configured to instruct, based on a particular command in the set of commands, a subset of a plurality of processing threads to perform a set of operations.
Claims19
A system comprising:
- one or more processors;
- a memory;
- a set of lower-level control threads stored in the memory and executable by the one or more processors;
- and
- a higher-level control thread stored in the memory and executed by the one or more processors, the higher-level control thread receiving a first command from a device, generating a set of second commands based on the first command for implementing the first command, and providing the set of second commands to a subset of the set of lower-level control threads, wherein generating the set of second commands based on the first command includes selecting the subset of the set of lower-level control threads in accordance with types of the set of second commands,
- wherein a lower-level control thread in the subset of the set of lower-level control threads instructs, based on a particular second command of the set of second commands, a subset of a plurality of processing threads to perform a set of operations for implementing the particular second command, the plurality of processing threads being separate and distinct from the higher-level control thread and the set of lower-level control threads, the subset of the plurality of processing threads including at least a memory thread that exclusively performs memory access operations and one or more compute threads that exclusively perform compute operations, wherein synchronization is provided between the one or more compute threads and the memory thread.
3 more elements
A first device, comprising:
- a memory storing computer-executable program instructions including a higher-level control thread, a set of lower-level control threads, and a plurality of processing threads;
- and
- a processor executing the computer-executable program instructions stored in the memory to cause the higher-level control thread to receive a first command from a second device, generate a set of second commands based on the first command for implementing the first command, and provide the set of second commands to a subset of the set of lower-level control threads, wherein generating the set of second commands based on the first command includes selecting the subset of the set of lower-level control threads in accordance with types of the set of second commands,
- the processor further executing the computer-executable program instructions stored in the memory to cause the lower-level control thread in the subset of the set of lower-level control threads to instruct, based on a particular second command of the set of second commands, a subset of a plurality of processing threads to perform a set of operations for implementing the particular second command, the plurality of processing threads being separate and distinct from the higher-level control thread and the set of lower-level control threads, the subset of the plurality of processing threads including at least a memory thread that exclusively performs memory access operations and one or more compute threads that exclusively perform compute operations, wherein synchronization is provided between the one or more compute threads and the memory thread.
1 more element
A method comprising:
- executing a higher-level control thread to generate, upon receiving a first command from a device, a set of second commands based on the first command for implementing the first command wherein generating the set of second commands based on the first command includes selecting a subset of a set of lower-level control threads in accordance with types of the set of second commands;
- executing the higher-level control thread to provide the set of second commands to the subset of the set of lower-level control threads;
- and
- executing a lower-level control thread in the subset of the set of lower-level control threads to instruct, based on a particular second command of the set of second commands, a subset of a plurality of processing threads to execute a set of operations for implementing the particular second command, the plurality of processing threads being separate and distinct from the higher-level control thread and the set of lower-level control threads, the subset of the plurality of processing threads including at least a memory thread that exclusively executes memory access operations and one or more compute threads that exclusively execute compute operations, wherein synchronization is provided between the one or more compute threads and the memory thread.
1 more element
Dependencies are inferred from the claim text.