Personalized language models for conversational AI systems and applications

A system comprising:

• one or more processing units to perform operations comprising:
• applying a plurality of first machine learning models to a first input prompt, wherein individual first machine learning models of the plurality of first machine learning models comprise a layer that generates a respective reward value of a first plurality of reward values for a respective attribute of a plurality of attributes;
• applying a second machine learning model to the first plurality of reward values to obtain first encoded reward values comprising vector embeddings, wherein the first encoded reward values are combined with an input of a language machine learning model to condition the language machine learning model to generate a first output response aligned with the plurality of attributes;
• applying the language machine learning model to the first encoded reward values and the first input prompt to obtain the first output response;
• calculating a first loss based at least on a comparison between the first output response and the first input prompt;
• and
• updating one or more parameters of the second machine learning model based at least on the first loss.

A non-transitory computer-readable storage medium storing instructions that, when executed by a processing device, cause the processing device to:

• apply a plurality of first machine learning models to a first input prompt, wherein individual first machine learning models of the plurality of first machine learning models comprise a layer that generates a respective reward value of a first plurality of reward values for a respective attribute of a plurality of attributes;
• apply a second machine learning model to the first plurality of reward values to obtain first encoded reward values comprising vector embeddings, wherein the first encoded reward values are combined with an input of a language machine learning model to condition the language machine learning model to generate a first output response aligned with the plurality of attributes;
• apply the language machine learning model to the first encoded reward values and the first input prompt to obtain the first output response;
• calculate a first loss based at least on a comparison between the first output response and the first input prompt;
• and
• update one or more parameters of the second machine learning model based at least on the first loss.