Diverse preference learning without reference model

A system comprising:

• one or more processors;
• and
• one or more memories storing processor-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
• receiving, from a preference dataset, a training sample that comprises a training context, a positive sample response for the training context, and a negative sample response for the training context;
• generating, using a positive loss function of a diverse preference training framework, a positive training signal based on the positive sample response to the training context;
• generating, using a contrastive loss function of the diverse preference training framework, a contrastive training signal based on the positive sample response and the negative sample response;
• generating, using a negative loss function of the diverse preference training framework, a negative training signal based on the negative sample response to the training context;
• generating an aggregated training signal based on the positive training signal, the contrastive training signal, and the negative training signal;
• training a domain-specific machine learned model based at least in part on altering one or more parameters of the domain-specific machine-learned model based on the aggregated training signal.

One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• receiving, from a preference dataset, a training sample that comprises a training context, a positive sample response for the training context, and a negative sample response for the training context;
• generating, using a positive loss function of a diverse preference training framework, a positive training signal based on the positive sample response to the training context;
• generating, using a contrastive loss function of the diverse preference training framework, a contrastive training signal based on the positive sample response and the negative sample response;
• generating, using a negative loss function of the diverse preference training framework, a negative training signal based on the negative sample response to the training context;
• generating an aggregated training signal based on the positive training signal, the contrastive training signal, and the negative training signal;
• training a domain-specific machine learned model based at least in part on altering one or more parameters of the domain-specific machine-learned model based on the aggregated training signal.