Back to results
US 12,737,243 B2Granted
Automatic error prediction in data centers
AssigneeNVIDIA Corporation
InventorsTamar Viclizki, Fay Wang, Divyansh Jain, Avighan Majumder, Vadim Gechman, Vibhor Agrawal
Life of this patent
Estimating the term…
Abstract
Apparatuses, systems, and techniques to predict a probability of an error or anomay in processing units, such as those of a data center. In at least one embodiment, the probability of an error occurring in a processing unit is identified using multiple trained machine learning models, in which the trained machine learning models each outputs, for example, the probability of an error occurring within a different predetermined time period.
Claims20
A method comprising:
- processing telemetry data corresponding to a device using a plurality of machine learning models, wherein the plurality of machine learning models are trained on historical telemetry data from devices of a data center, wherein each machine learning model of the plurality of machine learning models outputs a respective error prediction of a plurality of error predictions, wherein the respective error prediction comprises a probability of a type of potential error occurring within a unique future time period different from future time periods of other error predictions of the plurality of error predictions;
- selecting, by a processing device and based on the plurality of error predictions, at least one preventative action of a plurality of candidate preventative actions corresponding to the device;
- and
- automatically performing the preventative action for the device in response to selecting the at least one preventative action, wherein at least one of the plurality of preventative actions comprises adjusting a workload of the device.
1 more element
A non-transitory computer-readable medium comprising instructions that, responsive to execution by a processing device, cause the processing device to perform operations comprising:
- processing telemetry data for a device by a first machine learning model that outputs a first error prediction comprising a first probability of a first type of potential error occurring within a first future time period, wherein the first machine learning model is trained on historical telemetry date from devices of a data center;
- processing the telemetry data for the device by a second machine learning model that outputs a second error prediction comprising a second probability of a second type of potential error occurring within a second future time period, wherein the first future time period is different from the second future time period, wherein the second machine learning model is trained on historical telemetry date from devices of the data center;
- selecting, by the processing device and based on the first error prediction and the second error predictions, at least one preventative action of a plurality of candidate preventative actions corresponding to the device;
- and
- automatically performing the preventative action for the device in response to selecting the at least one preventative action, wherein at least one of the plurality of preventative actions comprises adjusting a workload of the device.
2 more elements
A system comprising:
- a memory device;
- and
- a processing device coupled to the memory device, wherein the processing device is to perform operations comprising:
- receiving, by a plurality of machine learning models, telemetry data for a processor;
- processing the telemetry data to generate a plurality of error predictions, wherein the plurality of machine learning models are trained on historical telemetry data from devices of a data center, wherein each error prediction of the plurality of error predictions comprises a probability of the processor experiencing a type of error within a unique future time period different from a future time period of other error predictions;
- selecting, by the processing device and based on the plurality of error predictions, at least one preventative action of a plurality of candidate preventative actions corresponding to the processor;
- and
- automatically performing the preventative action for the processor in response to selecting the at least one of the plurality of preventative actions comprises adjusting a workload of the processor.
5 more elements
Dependencies are inferred from the claim text.