Back to results
US 11,961,007 B2Granted
Split network acceleration architecture
AssigneeQUALCOMM Incorporated
InventorsColin Beaton Verrilli, Rashid Ahmed Akbar Attar, Raghavendar Bhavansikar
Life of this patent
Estimating the term…
Abstract
A method for accelerating machine learning on a computing device is described. The method includes hosting a neural network in a first inference accelerator and a second inference accelerator. The neural network split between the first inference accelerator and the second inference accelerator. The method also includes routing intermediate inference request results directly between the first inference accelerator and the second inference accelerator. The method further includes generating a final inference request result from the intermediate inference request results.
Claims20
A method for accelerating machine learning on a computing device comprising a first inference accelerator, a second inference accelerator, and a host processor, the method comprising:
- implementing, in memory of the host processor, a first request queue associated with the first inference accelerator, the first request queue receiving a pointer to inference data received at the host processor, the inference data being transmitted from the host processor, via a virtual channel, to the first inference accelerator for processing in response to the first request queue receiving the pointer;
- implementing, in the memory of the host processor, a completion queue for receiving a final inference request result, via a first direct memory access (DMA) transfer, in response to transmitting the inference data;
- hosting a neural network in the first inference accelerator and the second inference accelerator, the neural network split between the first inference accelerator and the second inference accelerator;
- routing intermediate inference request results directly between the first inference accelerator and the second inference accelerator, the routing of the intermediate interface request results being performed independently of the host processor;
- generating a final inference request result from the intermediate inference request results;
- and
- implementing, in the first inference accelerator or the second inference accelerator, a global synchronization manager (GSM) to route the final inference request result to the host processor, via the first DMA transfer, in accordance with satisfying a pre-synchronization condition.
4 more elements
A system for accelerating machine learning, comprising:
- a first inference accelerator and a second inference accelerator to host a neural network, the neural network split between the first inference accelerator and the second inference accelerator;
- a host processor to receive inference data and a final inference request result, the final inference request result being generated from intermediate inference request results, the host processor including:
- a first request queue associated with the first inference accelerator, the first request queue receiving a pointer to the inference data, the inference data being transmitted from the host processor, via a virtual channel, to the first inference accelerator for processing in response to the first request queue receiving the pointer, and
- a completion queue for receiving a final inference request result, via a first direct memory access (DMA) transfer, in response to transmitting the inference data;
- a switch to route the intermediate inference request results directly between the first inference accelerator and the second inference accelerator, the routing of the intermediate interface request results being performed independently of the host processor;
- and
- a global synchronization manager (GSM), included in the first inference accelerator or the second inference accelerator, to route the final inference request result to the host processor, via the first DMA transfer, in accordance with satisfying a pre-synchronization condition.
4 more elements
A system for accelerating machine learning, comprising:
- a first inference accelerator and a second inference accelerator to host a neural network, the neural network split between the first inference accelerator and the second inference accelerator;
- a host processor to receive inference data and a final inference request result, the final inference request result being generated from intermediate inference request results, the host processor including:
- a first request queue associated with the first inference accelerator, the first request queue receiving a pointer to the inference data, the inference data being transmitted from the host processor, via a virtual channel, to the first inference accelerator for processing in response to the first request queue receiving the pointer, and
- a completion queue for receiving a final inference request result, via a first direct memory access (DMA) transfer, in response to transmitting the inference data;
- means for routing the intermediate inference request results directly between the first inference accelerator and the second inference accelerator, the routing of the intermediate interface request results being performed independently of the host processor;
- and
- means for transmitting, the final inference request result to the host processor, via the first DMA transfer, the transmission being initiated by a global synchronization manager (GSM) included in the first inference accelerator or the second inference accelerator in accordance with satisfying a pre-synchronization condition.
4 more elements
Dependencies are inferred from the claim text.