US 12,585,880 B2 · Scalable consistency ensemble for machine learning models

Abstract

Certain aspects of the disclosure provide a method for generating a final output response. The method may include receiving an input; generating a plurality of intermediate responses to the input using a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models generates at least one intermediate response based on the input; generating a semantic consistency metric for each respective intermediate response of the plurality of intermediate responses by performing a comparison between the respective intermediate response and other intermediate responses of the plurality of intermediate responses; selecting a subset of the plurality of intermediate responses based on the semantic consistency metric of each respective intermediate response of the plurality of intermediate responses; and generating a final output response based on the subset of the plurality of intermediate responses.

Claims20

Dependent claims

A method for generating an output response, comprising:

receiving an input;
generating a plurality of intermediate responses to the input using a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models generates at least one intermediate response based on the input;
generating a semantic consistency metric for each respective intermediate response of the plurality of intermediate responses by performing a comparison between the respective intermediate response and other intermediate responses of the plurality of intermediate responses;
selecting a subset of the plurality of intermediate responses based on the semantic consistency metric of each respective intermediate response of the plurality of intermediate responses;
and
generating an output response based on the subset of the plurality of intermediate responses.

3 more elements

A processing system, comprising:

a memory comprising computer-executable instructions;
and
a processor configured to execute the computer-executable instructions and cause the processing system to:
receive an input;
generate a plurality of intermediate responses to the input using a plurality of machine learning models, wherein each machine learning model of the plurality of machine learning models is configured to generate at least one intermediate response based on the input;
generate a semantic consistency metric for each respective intermediate response of the plurality of intermediate responses by performing a comparison between the respective intermediate response and other intermediate responses of the plurality of intermediate responses;
select a subset of the plurality of intermediate responses based on the semantic consistency metric of each respective intermediate response of the plurality of intermediate responses;
and
generate an output response based on the subset of the plurality of intermediate responses.

6 more elements

A method for evaluating consistency across machine learning model response outputs and synthesizing an output response, the method comprising:

receiving a plurality of response pairs, each response pair of the plurality of response pairs including an input and a generated output;
aggregating the plurality of response pairs into a response prompt;
providing the response prompt and an evaluation prompt template to a semantic evaluation consistency model;
receiving a semantic consistency metric for each respective response pair of the plurality of response pairs from the semantic evaluation consistency model;
selecting a subset of the plurality of response pairs based on the semantic consistency metric for each respective response pair of the plurality of response pairs;
providing a synthesizing prompt and the selected subset of the plurality of response pairs to a synthesizing model;
receiving a synthesized output response, based on the synthesizing prompt and the subset of the plurality of response pairs, from the synthesizing model;
and
providing the synthesized output response as the output response.

6 more elements

Dependencies are inferred from the claim text.

Scalable consistency ensemble for machine learning models

Life of this patent

Scalable consistency ensemble for machine learning models

Life of this patent

Abstract

Claims20