US 2023/0156248 A1 · EPHEMERAL LEARNING OF MACHINE LEARNING MODEL(S)

Abstract

Implementations disclosed herein are directed to ephemeral learning of machine learning (“ML”) model(s) based on gradient(s) generated at a remote system (e.g., remote server(s)). Processor(s) of the remote system can receive stream(s) of audio data capturing spoken utterance(s) from a client device of a user. A fulfillment pipeline can process the stream(s) of audio data to cause certain fulfillment(s) of the spoken utterance(s) to be performed. Meanwhile, a training pipeline can process the stream(s) of audio data to generate gradient(s) using unsupervised learning techniques. Subsequent to the processing by the fulfillment pipeline and/or the training pipeline, the stream(s) of audio data are discarded by the remote system. Accordingly, the ML model(s) can be trained at the remote system without storing or logging of the stream(s) of audio data by non-transient memory thereof, thereby providing more efficient training mechanisms for training the ML model(s) and also increasing security of user data.

Claims20

Dependent claims

A method implemented by one or more processors at one or more remote servers, the method comprising:

receiving, from a client device of a user, a stream of audio data capturing a spoken utterance of the user, the stream of audio data being generated by one or more microphones of the client device;
processing the stream of audio data to cause a certain fulfillment of the spoken utterance to be performed;
while the stream of audio data is being processed to cause the certain fulfillment of the spoken utterance to be performed:
processing, using a global machine learning (ML) model stored remotely at one or more of the remote servers, the stream of audio data to generate predicted output;
generating, using unsupervised learning, a gradient based on the predicted output;
and
discarding the stream of audio data;
and
subsequent to generating the gradient based on the predicted output and using unsupervised learning:
updating, based on the gradient, global weights of the global ML model stored remotely at one or more of the remote servers.

7 more elements

A method implemented by one or more processors at a remote server, the method comprising:

receiving, from a client device of a third-party (3P) developer, a 3P automatic speech recognition (ASR) model;
receiving, from a client device of a user, a stream of audio data capturing a spoken utterance of the user, the stream of audio data being generated by one or more microphones of the client device;
processing the stream of audio data to cause a certain fulfillment of the spoken utterance to be performed, wherein processing the stream of audio data to cause the certain fulfillment of the spoken utterance to be performed comprises:
processing, using the 3P ASR model, the stream of audio data to generate a stream of ASR data;
processing, using a natural language understanding (NLU) model, the stream of ASR data to generate a stream of NLU data;
and
causing the certain fulfillment of the spoken utterance to be performed based on the stream of NLU data;
while the stream of audio data is being processed to cause the certain fulfillment of the spoken utterance to be performed:
generating, using unsupervised learning, a gradient based on the stream of ASR data;
and
discarding the stream of audio data;
and
subsequent to generating the gradient based on the stream of ASR data and using unsupervised learning:
updating, based on the gradient, 3P weights of the 3P ASR model that is received from the 3P developer to generate an updated 3P ASR model;
and
transmitting, to the client device of the 3P developer, the updated 3P ASR model.

13 more elements

Dependencies are inferred from the claim text.

Ephemeral learning of machine learning model(s)

Life of this patent

Ephemeral learning of machine learning model(s)

Life of this patent

Abstract

Claims20