Back to results
US 12,639,616 B1Granted
Exploiting learned saliency to optimize machine learning training
AssigneeXilinx, Inc.
InventorsErwei Wang, Samuel R. Bayliss
Life of this patent
Estimating the term…
Abstract
Embodiments herein describe techniques for setting and using saliency values to modify how a ML model is trained. In one embodiment, different blocks of data (referred to herein as tiles) are assigned respective saliency values. After performing one or more iterations, a training application can modify the default saliency values assigned to the tiles to reflect the importance of the tile. In one embodiment, the training application evaluates a weight gradient that indicates how a weight (or weights) corresponding to each tile are modified and modifies the saliency values accordingly. The ML training system can then use the saliency values to affect future training iterations to reduce the time required to train the ML model or save power.
Claims17
An integrated circuit, comprising:
- an accelerator circuit comprising data processing engines (DPEs) configured to train a multi-layer neural network machine-learning model (ML model) based on training data and associated labels:
- segment the training data into tiles, wherein the tiles comprise one or more of multiple pixels and multiple syllables of speech;
- assign salience values to the tiles;
- assign weights to layers of the ML model;
- perform a first ML training iteration on the tiles, including to process the tiles based on the weights of the respective layers and at levels of precision that are based on the salience values of the respective tiles;
- adjust the weights based on validation results of the first ML training iteration;
- adjust the salience values of the tiles based on the adjustments to the weights, including to increase the salience value of a tile for which a weight gradient meets a salience threshold, such that the adjusted salience value reflects importance of the tile in training the ML model;
- and
- perform a second ML training iteration on the tiles, including to process the tiles based on the adjusted weights of the respective layers and levels of precision that are based on the adjusted salience values of the respective tiles, including to increase the level of precision at which the tile is processed.
6 more elements
A method, comprising:
- training a multi-layer neural network machine-learning model (ML model) based on training data and associated labels, comprising:
- segmenting the training data into tiles, wherein the tiles comprise one or more of multiple pixels and multiple syllables of speech;
- assigning salience values to the tiles;
- assigning weights to layers of the ML model;
- performing a first ML training iteration on the tiles, comprising processing the tiles based on the weights of the respective layers and at levels of precision that are based on the salience values of the respective tiles adjusting the weights based on validation results of the first ML training iteration;
- adjusting the salience values of the tiles based on the adjustments to the weights, including increasing the salience value of a tile for which a weight gradient meets a salience threshold, such that the adjusted salience value reflects importance of the tile in training the ML model;
- and
- performing a second ML training iteration on the tiles, comprising processing the tiles based on the adjusted weights of the respective layers and at levels of precision that are based on the adjusted salience values of the respective tiles including increasing the level of precision at which the tile is processed.
5 more elements
A system, comprising:
- a host comprising a host processor and memory encoded with an application program that includes instructions for training a multi-layer neural network machine-learning model (ML model);
- and
- an accelerator circuit comprising a plurality of data processing engines (DPEs) configured to train the ML model on behalf of the host processor based on training data and associated labels, including to
- segment the training data into tiles, wherein the tiles comprise one or more of multiple pixels and multiple syllables of speech;
- assign salience values to the tiles;
- assign weights to layers of the ML model;
- perform a first ML training iteration on the tiles, including to process the tiles based on the weights of the respective layers and at levels of precision that are based on the salience values of the respective tiles;
- adjust the weights based on validation results of the first training iteration;
- adjust the salience values of the tiles based on the adjustments to the respective weights, including to increase the salience value of a tile for which a weight gradient meets a salience threshold, such that the adjusted salience value reflects importance of the tile in training the ML model;
- and
- perform a second ML training iteration on the tiles, including to process the tiles based on the adjusted weights of the respective layers and levels of precision that are based on the adjusted salience values of the respective tiles, including to increase the level of precision at which the tile is processed.
8 more elements
Dependencies are inferred from the claim text.