Back to results
US 11,899,551 B1Granted
On-chip software-based activity monitor to configure throttling at a hardware-based activity monitor
AssigneeAmazon Technologies, Inc.
InventorThomas A. Volpe
Life of this patent
Estimating the term…
Abstract
On-chip software-based activity monitoring is implemented to configure hardware-based activity throttling. A software-based activity monitor implemented on an integrated circuit obtains data from on-chip components to determine throttling modifications for a processing engine of the integrated circuit. The throttling modifications are applied to throttling criteria that is used by a hardware-based activity monitor on the integrated circuit which is responsible for directly evaluating and throttling processing at the processing engine of the integrated circuit.
Claims20
A system, comprising:
- at least one processor;
- a memory, storing program instructions that when executed by the at least one processor cause the at least one processor to submit one or more machine learning tasks for processing to an inference accelerator;
- and
- wherein the inference accelerator comprises an integrated circuit, wherein the integrated circuit comprises a software-based activity monitor implemented by firmware executing on a first portion of the integrated circuit, wherein the integrated circuit further comprises a tensor processing unit that comprises a hardware-based activity monitor implemented on a second portion of the integrated circuit and a systolic array implemented on a third portion of the integrated circuit that is configured to perform the one or more machine learning tasks;
- and
- wherein the software-based activity monitor is configured to:
- monitor data obtained from a component of the integrated circuit;
- determine a modification to throttling criteria used by the hardware-based activity monitor based on the monitored data;
- and
- cause the modification to the throttling criteria used by the hardware-based activity monitor;
- and
- wherein the hardware-based activity monitor is configured to:
- monitor processing performed by the systolic array according to the modified throttling criteria;
- and
- cause, according to the monitoring, a modification to a rate at which data enters the systolic array for processing.
12 more elements
A method, comprising:
- monitoring, by a software-based activity monitor of an integrated circuit, data obtained from a component of the integrated circuit, wherein the software-based activity monitor is implemented using programming instructions executing on a first portion of the integrated circuit;
- determining, by the software-based activity monitor, a modification to a throttling criteria used by a hardware-based activity monitor based on the monitored data, wherein the hardware-based activity monitor is implemented on a second portion of the integrated circuit, and wherein the hardware-based activity monitor uses the throttling criteria to evaluate processing performed on a third portion of the integrated circuit and to modify, according to the evaluation, a rate at which data enters the third portion of the integrated circuit for processing;
- and
- applying, by the software-based activity monitor, the modification to the throttling criteria used by the hardware-based activity monitor.
1 more element
A device, comprising circuitry that implements an integrated circuit, the integrated circuit comprising:
- a first portion that implements a processing engine;
- a second portion that implements a hardware-based activity monitor, configured to:
- monitor processing performed by the first portion of the integrated circuit according to throttling criteria;
- and
- modify, according to the monitoring, a rate at which data enters the first portion of the integrated circuit for processing;
- and
- a third portion executing programing instructions that implement a software-based activity monitor, configured to:
- monitor data obtained from a component of the integrated circuit;
- determine a modification to the throttling criteria used by the hardware-based activity monitor based on the monitored data;
- and
- apply the modification to the throttling criteria used by the hardware-based activity monitor.
8 more elements
Dependencies are inferred from the claim text.