# US 2023/0058159 A1 · Bayesian Bits Joint Mixed-Precision Quantization And Structured Pruning Using Decomposed Quantization And Bayesian Gates

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0058159 A1
- Document ID: US20230058159A1
- Status: Published application
- Kind: A1
- Application number: 17/759,725
- Filed: 2021-04-29
- Published: 2023-02-23
- Earliest claimed: 2020-04-29
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Marinus Willem VAN BAALEN; Christos LOUIZOS; Markus NAGEL; Tijmen Pieter Frederik BLANKEVOORT; Rana Ali AMJAD
- Web page: https://patentgrep.com/patent/US20230058159A1

## CPC

- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/09: Supervised learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Various embodiments include methods and devices for joint mixed-precision quantization and structured pruning. Embodiments may include determining whether a plurality of gates of quantization and pruning gates are selected for combination, and in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor in which the next bit-width increases for each successive iteration, and adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor in which the next quantized tensor has the next bit-width, and in which the next quantized tensor is the quantized tensor for a successive iteration.

## Claims

### Claim 1

A method for joint mixed-precision quantization and structured pruning, comprising:
determining whether a plurality of gates of quantization and pruning gates are selected for combination; and
in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination:
quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, wherein the next bit-width increases for each successive iteration; and
adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor, wherein the next quantized tensor has the next bit-width, and wherein the next quantized tensor is the quantized tensor for a successive iteration.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining whether a successive gate of the quantization and pruning gates to the plurality of gates is selected for combination; and
outputting the quantized tensor of a last iteration in response to determining that the successive gate is not selected for combination.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving a tensor by the quantization and pruning gates;
clipping the tensor to a range of values producing a clipped tensor; and
quantizing the clipped tensor to a lowest bit-width producing a clipped quantized tensor, wherein quantizing a residual error of a quantized tensor to a scale of a next bit-width for a first iteration comprises quantizing a residual error of the clipped quantized tensor to the scale of the next bit-width, wherein the next bit-width is greater than the lowest bit-width.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
determining whether a gate of the quantization and pruning gates preceding the plurality of gates is selected for combination; and
outputting the clipped quantized tensor in response to determining that the gate preceding the plurality of gates is not selected for combination.

### Claim 5 (depends on 4)

The method of claim 4, wherein quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, and adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor is further performed in response to determining that the gate preceding the plurality of gates is selected for combination.

### Claim 6 (depends on 3)

The method of claim 3, further comprising, in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination computing a residual error of a quantized tensor by subtracting the quantized tensor from the clipped tensor.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
receiving a weight tensor by the quantization and pruning gates;
determining whether a gate of the quantization and pruning gates preceding the plurality of gates is selected for combination; and
outputting a tensor configured to prune the weight tensor in response to determining that that the gate preceding the plurality of gates is not selected for combination.

### Claim 8 (depends on 1)

The method of claim 1, further comprising training the quantization and pruning gates using a regularizer configured to achieve a target balance of efficiency of a neural network and accuracy of the neural network by selecting fewer gates for combination producing a lower bit-width and more pruned quantized tensor favoring efficiency in comparison to selecting more gates for combination producing a higher bit-width and less pruned quantized tensor favoring accuracy.

### Claim 9 (depends on 8)

The method of claim 8, further comprising setting a hyperparameter for the regularizer such that the hyperparameter strengthens regularization on a basis of an efficiency factor of the neural network.

### Claim 10 (depends on 8)

The method of claim 8, further comprising setting a hyperparameter for the regularizer configured to control the target balance of efficiency of a neural network and accuracy of the neural network.

### Claim 11 (depends on 8)

The method of claim 8, wherein training the quantization and pruning gates using a regularizer comprises training the quantization and pruning gates to select no gates for combination for a group of weights of a weight tensor, wherein selecting no gates prunes the group of weights reducing a cost of an efficiency factor of the neural network.

### Claim 12 (depends on 1)

The method of claim 1, further comprising training the quantization and pruning gates for determining values of gates, including the plurality of gates, based on variational approximation of a posterior distribution of the quantization and pruning gates, wherein the values of the gates are configured to indicate to the quantization and pruning gates whether the gates are selected for combination.

### Claim 13 (depends on 1)

The method of claim 1, wherein quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor comprises quantizing the residual error of the quantized tensor to the scale of the next bit-width producing the residual error quantized tensor, wherein the next bit-width doubles for each successive iteration.

### Claim 14

A computing device, comprising a processing device configured with executable instructions to cause the processing device to execute operations comprising:
determining whether a plurality of gates of quantization and pruning gates are selected for combination; and
in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination:
quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, wherein the next bit-width increases for each successive iteration; and
adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor, wherein the next quantized tensor has the next bit-width, and wherein the next quantized tensor is the quantized tensor for a successive iteration.

### Claim 15 (depends on 14)

The computing device of claim 14, wherein the processing device is configured with executable instructions to perform operations further comprising:
determining whether a successive gate of the quantization and pruning gates to the plurality of gates is selected for combination; and
outputting the quantized tensor of a last iteration in response to determining that the successive gate is not selected for combination.

### Claim 16 (depends on 14)

The computing device of claim 14, wherein the processing device is configured with executable instructions to perform operations further comprising:
receiving a tensor by the quantization and pruning gates;
clipping the tensor to a range of values producing a clipped tensor; and
quantizing the clipped tensor to a lowest bit-width producing a clipped quantized tensor, wherein quantizing a residual error of a quantized tensor to a scale of a next bit-width for a first iteration comprises quantizing a residual error of the clipped quantized tensor to the scale of the next bit-width, wherein the next bit-width is greater than the lowest bit-width.

### Claim 17 (depends on 16)

The computing device of claim 16, wherein the processing device is configured with executable instructions to perform operations further comprising:
determining whether a gate of the quantization and pruning gates preceding the plurality of gates is selected for combination; and
outputting the clipped quantized tensor in response to determining that the gate preceding the plurality of gates is not selected for combination.

### Claim 18 (depends on 17)

The computing device of claim 17, wherein the processing device is configured with executable instructions to perform operations such that quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, and adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor is further performed in response to determining that the gate preceding the plurality of gates is selected for combination.

### Claim 19 (depends on 16)

The computing device of claim 16, wherein the processing device is configured with executable instructions to perform operations further comprising, in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination computing a residual error of a quantized tensor by subtracting the quantized tensor from the clipped tensor.

### Claim 20 (depends on 14)

The computing device of claim 14, wherein the processing device is configured with executable instructions to perform operations further comprising:
receiving a weight tensor by the quantization and pruning gates;
determining whether a gate of the quantization and pruning gates preceding the plurality of gates is selected for combination; and
outputting a tensor configured to prune the weight tensor in response to determining that the gate preceding the plurality of gates is not selected for combination.

### Claim 21 (depends on 14)

The computing device of claim 14, wherein the processing device is configured with executable instructions to perform operations further comprising training the quantization and pruning gates using a regularizer configured to achieve a target balance of efficiency of a neural network and accuracy of the neural network by selecting fewer gates for combination producing a lower bit-width and more pruned quantized tensor favoring efficiency in comparison to selecting more gates for combination producing a higher bit-width and less pruned quantized tensor favoring accuracy.

### Claim 22 (depends on 21)

The computing device of claim 21, wherein the processing device is configured with executable instructions to perform operations further comprising setting a hyperparameter for the regularizer such that the hyperparameter strengthens regularization on a basis of an efficiency factor of the neural network.

### Claim 23 (depends on 21)

The computing device of claim 21, wherein the processing device is configured with executable instructions to perform operations further comprising setting a hyperparameter for the regularizer configured to control the target balance of efficiency of a neural network and accuracy of the neural network.

### Claim 24 (depends on 21)

The computing device of claim 21, wherein the processing device is configured with executable instructions to perform operations such that training the quantization and pruning gates using a regularizer comprises training the quantization and pruning gates to select no gates for combination for a group of weights of a weight tensor, wherein selecting no gates prunes the group of weights reducing a cost of an efficiency factor of the neural network.

### Claim 25 (depends on 14)

The computing device of claim 14, wherein the processing device is configured with executable instructions to perform operations further comprising training the quantization and pruning gates for determining values of gates, including the plurality of gates, based on variational approximation of a posterior distribution of the quantization and pruning gates, wherein the values of the gates are configured to indicate to the quantization and pruning gates whether the gates are selected for combination.

### Claim 26 (depends on 14)

The computing device of claim 14, wherein the processing device is configured with executable instructions to perform operations such that quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor comprises quantizing the residual error of the quantized tensor to the scale of the next bit-width producing the residual error quantized tensor, wherein the next bit-width doubles for each successive iteration.

### Claim 27

A non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a computing device to perform operations comprising:
determining whether a plurality of gates of quantization and pruning gates are selected for combination; and
in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination:
quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, wherein the next bit-width increases for each successive iteration; and
adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor, wherein the next quantized tensor has the next bit-width, and wherein the next quantized tensor is the quantized tensor for a successive iteration.

### Claim 28 (depends on 27)

The non-transitory processor-readable storage medium of claim 27, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising:
determining whether a successive gate of the quantization and pruning gates to the plurality of gates is selected for combination; and
outputting the quantized tensor of a last iteration in response to determining that the successive gate is not selected for combination.

### Claim 29 (depends on 27)

The non-transitory processor-readable storage medium of claim 27, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising:
receiving a tensor by the quantization and pruning gates;
clipping the tensor to a range of values producing a clipped tensor; and
quantizing the clipped tensor to a lowest bit-width producing a clipped quantized tensor, wherein quantizing a residual error of a quantized tensor to a scale of a next bit-width for a first iteration comprises quantizing a residual error of the clipped quantized tensor to the scale of the next bit-width, wherein the next bit-width is greater than the lowest bit-width.

### Claim 30 (depends on 29)

The non-transitory processor-readable storage medium of claim 29, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising:
determining whether a gate of the quantization and pruning gates preceding the plurality of gates is selected for combination; and
outputting the clipped quantized tensor in response to determining that the gate preceding the plurality of gates is not selected for combination.

### Claim 31 (depends on 30)

The non-transitory processor-readable storage medium of claim 30, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations such that quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, and adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor is further performed in response to determining that the gate preceding the plurality of gates is selected for combination.

### Claim 32 (depends on 29)

The non-transitory processor-readable storage medium of claim 29, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising, in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination computing a residual error of a quantized tensor by subtracting the quantized tensor from the clipped tensor.

### Claim 33 (depends on 27)

The non-transitory processor-readable storage medium of claim 27, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising:
receiving a weight tensor by the quantization and pruning gates;
determining whether a gate of the quantization and pruning gates preceding the plurality of gates is selected for combination; and
outputting a tensor configured to prune the weight tensor in response to determining that that the gate preceding the plurality of gates is not selected for combination.

### Claim 34 (depends on 27)

The non-transitory processor-readable storage medium of claim 27, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising training the quantization and pruning gates using a regularizer configured to achieve a target balance of efficiency of a neural network and accuracy of the neural network by selecting fewer gates for combination producing a lower bit-width and more pruned quantized tensor favoring efficiency in comparison to selecting more gates for combination producing a higher bit-width and less pruned quantized tensor favoring accuracy.

### Claim 35 (depends on 34)

The non-transitory processor-readable storage medium of claim 34, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising setting a hyperparameter for the regularizer such that the hyperparameter strengthens regularization on a basis of an efficiency factor of the neural network.

### Claim 36 (depends on 34)

The non-transitory processor-readable storage medium of claim 34, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising setting a hyperparameter for the regularizer configured to control the target balance of efficiency of a neural network and accuracy of the neural network.

### Claim 37 (depends on 34)

The non-transitory processor-readable storage medium of claim 34, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations such that training the quantization and pruning gates using a regularizer comprises training the quantization and pruning gates to select no gates for combination for a group of weights of a weight tensor, wherein selecting no gates prunes the group of weights reducing a cost of an efficiency factor of the neural network.

### Claim 38 (depends on 27)

The non-transitory processor-readable storage medium of claim 27, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations further comprising training the quantization and pruning gates for determining values of gates, including the plurality of gates, based on variational approximation of a posterior distribution of the quantization and pruning gates, wherein the values of the gates are configured to indicate to the quantization and pruning gates whether the gates are selected for combination.

### Claim 39 (depends on 27)

The non-transitory processor-readable storage medium of claim 27, wherein the stored processor-executable instructions are configured to cause the processor of the computing device to perform operations such that quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor comprises quantizing the residual error of the quantized tensor to the scale of the next bit-width producing the residual error quantized tensor, wherein the next bit-width doubles for each successive iteration.

### Claim 40

A computing device, comprising:
means for determining whether a plurality of gates of quantization and pruning gates are selected for combination; and
means for iteratively performing operations for each successive gate of the plurality of gates selected for combination in response to determining that the plurality of gates are selected for combination comprising:
means for quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, wherein the next bit-width increases for each successive iteration; and
means for adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor, wherein the next quantized tensor has the next bit-width, and wherein the next quantized tensor is the quantized tensor for a successive iteration.

### Claim 41 (depends on 40)

The computing device of claim 40, further comprising:
means for determining whether a successive gate of the quantization and pruning gates to the plurality of gates is selected for combination; and
means for outputting the quantized tensor of a last iteration in response to determining that the successive gate is not selected for combination.

### Claim 42 (depends on 40)

The computing device of claim 40, further comprising:
means for receiving a tensor by the quantization and pruning gates;
means for clipping the tensor to a range of values producing a clipped tensor; and
means for quantizing the clipped tensor to a lowest bit-width producing a clipped quantized tensor, wherein means for quantizing a residual error of a quantized tensor to a scale of a next bit-width for a first iteration comprises means for quantizing a residual error of the clipped quantized tensor to the scale of the next bit-width, wherein the next bit-width is greater than the lowest bit-width.

### Claim 43 (depends on 42)

The computing device of claim 42, further comprising:
means for determining whether a gate of the quantization and pruning gates preceding the plurality of gates is selected for combination; and
means for outputting the clipped quantized tensor in response to determining that the gate preceding the plurality of gates is not selected for combination.

### Claim 44 (depends on 43)

The computing device of claim 43, wherein means for iteratively performing operations for each successive gate of the plurality of gates selected for combination of quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, and adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor further in response to determining that the gate preceding the plurality of gates is selected for combination.

### Claim 45 (depends on 42)

The computing device of claim 42, further comprising means for computing a residual error of a quantized tensor by subtracting the quantized tensor from the clipped tensor iteratively for each successive gate of the plurality of gates selected for combination in response to determining that the plurality of gates are selected for combination.

### Claim 46 (depends on 40)

The computing device of claim 40, further comprising:
means for receiving a weight tensor by the quantization and pruning gates;
means for determining whether a gate of the quantization and pruning gates preceding the plurality of gates is selected for combination; and
means for outputting a tensor configured to prune the weight tensor in response to determining that the gate preceding the plurality of gates is not selected for combination.

### Claim 47 (depends on 40)

The computing device of claim 40, further comprising means for training the quantization and pruning gates using a regularizer configured to achieve a target balance of efficiency of a neural network and accuracy of the neural network by selecting fewer gates for combination producing a lower bit-width and more pruned quantized tensor favoring efficiency in comparison to selecting more gates for combination producing a higher bit-width and less pruned quantized tensor favoring accuracy.

### Claim 48 (depends on 47)

The computing device of claim 47, further comprising means for setting a hyperparameter for the regularizer such that the hyperparameter strengthens regularization on a basis of an efficiency factor of the neural network.

### Claim 49 (depends on 47)

The computing device of claim 47, further comprising means for setting a hyperparameter for the regularizer configured to control the target balance of efficiency of a neural network and accuracy of the neural network.

### Claim 50 (depends on 47)

The computing device of claim 47, wherein means for training the quantization and pruning gates using a regularizer comprises means for training the quantization and pruning gates to select no gates for combination for a group of weights of a weight tensor, wherein selecting no gates prunes the group of weights reducing a cost of an efficiency factor of the neural network.

### Claim 51 (depends on 40)

The computing device of claim 40, further comprising means for training the quantization and pruning gates for determining values of gates, including the plurality of gates, based on variational approximation of a posterior distribution of the quantization and pruning gates, wherein the values of the gates are configured to indicate to the quantization and pruning gates whether the gates are selected for combination.

### Claim 52 (depends on 40)

The computing device of claim 40, wherein means for quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor comprises means for quantizing the residual error of the quantized tensor to the scale of the next bit-width producing the residual error quantized tensor, wherein the next bit-width doubles for each successive iteration.

### Claim 53

A method for generating a mixed-precision artificial neural network, comprising:
quantizing a first layer of the mixed-precision artificial neural network at a first precision determined using corresponding stochastic gates of quantization and pruning gates; and
quantizing a second layer of the mixed-precision artificial neural network at a second precision determined using corresponding stochastic gates of the quantization and pruning gates, wherein the first precision and the second precision are different.

### Claim 54 (depends on 53)

The method of claim 53, further comprising:
determining whether a plurality of stochastic gates of the quantization and pruning gates are selected for combination; and
in response to determining that the plurality of stochastic gates are selected for combination, iteratively for each successive stochastic gate selected for combination:
quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, wherein the next bit-width increases for each successive iteration; and
adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor, wherein the next quantized tensor has the next bit-width, and wherein the next quantized tensor is the quantized tensor for a successive iteration.

### Claim 55 (depends on 54)

The method of claim 54, further comprising:
determining whether a successive stochastic gate to the plurality of stochastic gates is selected for combination; and
outputting the quantized tensor of a last iteration in response to determining that the successive stochastic gate is not selected for combination.

### Claim 56 (depends on 54)

The method of claim 54, further comprising:
receiving a tensor by the quantization and pruning gates;
clipping the tensor to a range of values producing a clipped tensor; and
quantizing the clipped tensor to a lowest bit-width producing a clipped quantized tensor, wherein quantizing a residual error of a quantized tensor to a scale of a next bit-width for a first iteration comprises quantizing a residual error of the clipped quantized tensor to the scale of the next bit-width, wherein the next bit-width is greater than the lowest bit-width.

### Claim 57 (depends on 56)

The method of claim 56, further comprising:
determining whether a stochastic gate of the quantization and pruning gates preceding the plurality of stochastic gates is selected for combination; and
outputting the clipped quantized tensor in response to determining that the stochastic gate preceding the plurality of stochastic gates is not selected for combination.

### Claim 58 (depends on 57)

The method of claim 57, wherein quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor, and adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor is further performed in response to determining that the stochastic gate preceding the plurality of stochastic gates is selected for combination.

### Claim 59 (depends on 56)

The method of claim 56, further comprising, in response to determining that the plurality of stochastic gates are selected for combination, iteratively for each successive stochastic gate of the plurality of stochastic gates selected for combination computing a residual error of a quantized tensor by subtracting the quantized tensor from the clipped tensor.

### Claim 60 (depends on 54)

The method of claim 54, further comprising:
receiving a weight tensor by the quantization and pruning gates;
determining whether a stochastic gate of the quantization and pruning gates preceding the plurality of stochastic gates is selected for combination; and
outputting a tensor configured to prune the weight tensor in response to determining that that the stochastic gate preceding the plurality of stochastic gates is not selected for combination.

### Claim 61 (depends on 54)

The method of claim 54, further comprising training the quantization and pruning gates using a regularizer configured to achieve a target balance of efficiency of a neural network and accuracy of the neural network by selecting fewer stochastic gates for combination producing a lower bit-width and more pruned quantized tensor favoring efficiency in comparison to selecting more stochastic gates for combination producing a higher bit-width and less pruned quantized tensor favoring accuracy.

### Claim 62 (depends on 61)

The method of claim 61, further comprising setting a hyperparameter for the regularizer such that the hyperparameter strengthens regularization on a basis of an efficiency factor of the neural network.

### Claim 63 (depends on 61)

The method of claim 61, further comprising setting a hyperparameter for the regularizer configured to control the target balance of efficiency of a neural network and accuracy of the neural network.

### Claim 64 (depends on 61)

The method of claim 61, wherein training the quantization and pruning gates using a regularizer comprises training the quantization and pruning gates to select no stochastic gates for combination for a group of weights of a weight tensor, wherein selecting no stochastic gates prunes the group of weights reducing a cost of an efficiency factor of the neural network.

### Claim 65 (depends on 54)

The method of claim 54, further comprising training the quantization and pruning gates for determining values of stochastic gates, including the plurality of stochastic gates, based on variational approximation of a posterior distribution of the quantization and pruning gates, wherein the values of the stochastic gates are configured to indicate to the quantization and pruning gates whether the stochastic gates are selected for combination.

### Claim 66 (depends on 54)

The method of claim 54, wherein quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor comprises quantizing the residual error of the quantized tensor to the scale of the next bit-width producing the residual error quantized tensor, wherein the next bit-width doubles for each successive iteration.
