Unsupervised pre-training of neural networks using generative models

A method for using a first model updated using a second model, comprising:

• receiving an image using a first model, the first model having one or more parameters updated using a second model;
• and
• generating at least one feature using the first model by applying the image as input,
• wherein the first model generates a first plurality of features using the image as an input, and the first model is updated using a loss corresponding to the first plurality of features determined with respect to a second plurality of features, the second plurality of features being generated using the second model and corresponding to an image.

A processor comprising:

• one or more circuits to:
• generate, using a first model, a plurality of first features corresponding to an image;
• generate, using a second model, a plurality of second features using the image as an input to the second model;
• determine a loss corresponding to the plurality of second features with respect to the plurality of first features;
• update the second model using the loss;
• and
• generate, using the second model, a response based on an input image.