US 11,385,863 B2Granted

Adjustable precision for multi-stage compute processes

AssigneeHewlett Packard Enterprise Development LP

InventorsSai Rahul Chalamalasetti, Paolo Faraboschi, Martin Foltin, Catherine Graves, Dejan S. Milojicic, Sergey Serebryakov, John Paul Strachan

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Disclosed techniques provide for dynamically changing precision of a multi-stage compute process. For example, changing neural network (NN) parameters on a per-layer basis depending on properties of incoming data streams and per-layer performance of an NN among other considerations. NNs include multiple layers that may each be calculated with a different degree of accuracy and therefore, compute resource overhead (e.g., memory, processor resources, etc.). NNs are usually trained with 32-bit or 16-bit floating-point numbers. Once trained, an NN may be deployed in production. One approach to reduce compute overhead is to reduce parameter precision of NNs to 16 or 8 for deployment. The conversion to an acceptable lower precision is usually determined manually before deployment and precision levels are fixed while deployed. Disclosed techniques and implementations address automatic rather than manual determination or precision levels for different stages and dynamically adjusting precision for each stage at run-time.

Claims20

Dependent claims

A computer-implemented method of processing a multi-stage compute process, the method comprising:

obtaining a multi-stage compute process for processing in a training mode, wherein results of a first stage of the multi-stage compute process provide input to a second stage of the multi-stage compute process;
processing the first stage on a first hardware processor at a first degree of precision to obtain first initial results;
analyzing the first initial results to determine a first accuracy result of the first stage;
processing the first stage on the first hardware processor at a second degree of precision to obtain first secondary results, wherein the second degree of precision is less than the first degree of precision;
analyzing the first secondary results to determine a second accuracy result of the first stage;
determining that the second accuracy result is within a first threshold of accuracy relative to the first accuracy result;
storing the second degree of precision as a first acceptable degree of precision for the first stage;
processing the second stage on the first hardware processor at a third degree of precision to obtain second initial results;
analyzing the second initial results to determine a third accuracy result of the second stage;
processing the second stage on the first hardware processor at a fourth degree of precision to obtain second secondary results, wherein the fourth degree of precision is less than the third degree of precision;
analyzing the second secondary results to determine a fourth accuracy result of the second stage;
determining that the fourth accuracy result is within a second threshold of accuracy relative to the third accuracy result;
and
storing the fourth degree of precision as a second acceptable degree of precision for the second stage,
wherein analyzing the first and second initial results and the first and second secondary results is performed automatically by a processor of a computer system.

12 more elements

A non-transitory computer readable medium comprising computer executable instructions stored thereon that when executed by one or more processing units, perform a method to process a multi-stage compute process, the method comprising:

obtaining a multi-stage compute process for processing in a training mode, wherein results of a first stage of the multi-stage compute process provide input to a second stage of the multi-stage compute process;
processing the first stage on a first hardware processor at a first degree of precision to obtain first initial results;
analyzing the first initial results to determine a first accuracy result of the first stage;
processing the first stage on the first hardware processor at a second degree of precision to obtain first secondary results, wherein the second degree of precision is less than the first degree of precision;
analyzing the first secondary results to determine a second accuracy result of the first stage;
determining that the second accuracy result is within a first threshold of accuracy relative to the first accuracy result;
storing the second degree of precision as a first acceptable degree of precision for the first stage;
processing the second stage on the first hardware processor at a third degree of precision to obtain second initial results;
analyzing the second initial results to determine a third accuracy result of the second stage;
processing the second stage on the first hardware processor at a fourth degree of precision to obtain second secondary results, wherein the fourth degree of precision is less than the third degree of precision;
analyzing the second secondary results to determine a fourth accuracy result of the second stage;
determining that the fourth accuracy result is within a second threshold of accuracy relative to the third accuracy result;
and
storing the fourth degree of precision as a second acceptable degree of precision for the second stage,
wherein analyzing the first and second initial results and the first and second secondary results is performed automatically by the one or more processing units.

12 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/052,218Filed 1 Aug 2018
Published: US 2020/0042287 A16 Feb 2020
Granted: 12 Jul 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification13 codes

G06F 7/483
Using non-contact-making devices, e.g. tube, solid state device; using unspecified devices: computations with numbers represented by a non-linear…
G06F 7/5443
For evaluating functions by calculation: sum of products
G06N 3/0464
Architecture, e.g. interconnection topology: convolutional networks [CNN, ConvNet]
G06N 3/0495
Architecture, e.g. interconnection topology: quantised networks; Sparse networks; Compressed networks