Back to results
US 2026/0073662 A1Published application
Encoder neural networks with power constrained latent representations
AssigneeGoogle Inc.
InventorsWilliam R. Mark, Andrey Zhmoginov
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an encoder neural network to minimize the capacity of an encoded representation of an input observation subject to a per-observation distortion constraint.
Claims20
A method performed by one or more computers, the method comprising:
- receiving an input observation;
- processing the input observation using an encoder neural network to generate an encoder output that comprises:
- (i) an initial latent vector representing at least a portion of the input observation;
- and
- (ii) a power output that defines a noise power for the initial latent vector;
- determining a scaling factor from the power output;
- and
- applying the scaling factor to the initial latent vector to generate a final latent vector representing at least the portion of the input observation and having a constrained signal power.
5 more elements
A system comprising:
- one or more computers;
- and
- one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
- receiving an input observation;
- processing the input observation using an encoder neural network to generate an encoder output that comprises:
- (i) an initial latent vector representing at least a portion of the input observation;
- and
- (ii) a power output that defines a noise power for the initial latent vector;
- determining a scaling factor from the power output;
- and
- applying the scaling factor to the initial latent vector to generate a final latent vector representing at least the portion of the input observation and having a constrained signal power.
8 more elements
The system of 18, the operations further comprising:
- sampling a noise vector from a noise distribution;
- scaling the noise vector using a factor that is defined by the noise power to generate a scaled noise vector;
- adding the scaled noise vector to the final latent vector to generate a noisy latent vector;
- processing a decoder input comprising the noisy latent vector using a decoder neural network to generate a reconstruction of the input observation;
- and
- training the decoder neural network and the encoder neural network jointly on an objective that, for the input observation, minimizes a capacity of the input observation as defined by the noise power subject to a constraint on a per-observation distortion of the reconstruction of the input observation relative to the input observation.
3 more elements
One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
- receiving an input observation;
- processing the input observation using an encoder neural network to generate an encoder output that comprises:
- (i) an initial latent vector representing at least a portion of the input observation;
- and
- (ii) a power output that defines a noise power for the initial latent vector;
- determining a scaling factor from the power output;
- and
- applying the scaling factor to the initial latent vector to generate a final latent vector representing at least the portion of the input observation and having a constrained signal power.
5 more elements
Dependencies are inferred from the claim text.