# US 2026/0093593 A1 · System for Automatically Evaluating the Output of Machine-Learned Models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2026/0093593 A1 - Document ID: US20260093593A1 - Status: Published application - Kind: A1 - Application number: 19/342,072 - Filed: 2025-09-26 - Published: 2026-04-02 - Earliest claimed: 2024-10-01 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Noam Velan; Doron Kukliansky; Alexander Goldshtein; Yehonatan Bitton; Itay Laish; Idan Szpektor; Ofir Roval - Web page: https://patentgrep.com/patent/US20260093593A1 ## CPC - G06F 11/3414: {Workload generation, e.g. scripts, playback} - G06F 11/0754: {by exceeding limits} - G06F 16/33295: {in dialogue systems} - G06F 40/30: Semantic analysis - G06F 40/56: Natural language generation - G06N 3/045: Combinations of networks - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/0475: Generative networks - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Provided is a system that automatically evaluates the output of machine-learned models. A computing system receives, from a user computing device, an input query. The computing system processes the input query with a generative model to generate a model output based on the input query. The computing system identifies one or more representative subsequences that correspond to a representation based on the textual response. The computing system generates a plurality of tuple pairs based on the one or more representative subsequences that correspond to a representation and the one or more media elements. For each of the relevant tuple pairs, the computing system processes the respective tuple pair with an entailment-scoring machine-learned model to generate an entailment score for the respective tuple pair. The computing system provides an entailment output for the model output based on the respective entailment scores generated for the one or more relevant tuple pairs. ## Claims ### Claim 1 A computer-implemented method for automatically evaluating a model output, the method comprising: receiving, by a computing system with one or more processors, an input query; processing, by the computing system, the input query with a generative model to generate a model output based on the input query, wherein the model output comprises a textual response, and wherein one or both of the input query and the output comprise one or more media elements; identifying, by the computing system, one or more representative subsequences that correspond to a representation based on the textual response; generating, by the computing system, a plurality of tuple pairs based on the one or more representative subsequences that correspond to a representation and the one or more media elements; for each of one or more relevant tuple pairs in the plurality of tuple pairs, processing, by the computing system, the respective tuple pair with an entailment-scoring machine-learned model to generate an entailment score for the respective tuple pair; and providing, by the computing system, an entailment output for the model output based on the respective entailment scores generated for the one or more relevant tuple pairs. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the method further comprises identifying the one or more relevant tuple pairs, wherein identifying the one or more relevant tuple pairs comprises: processing, by the computing system, the respective tuple pair with a relevance-scoring machine-learned model to generate a relevance score for the respective tuple pair; and generating, by the computing system, a list of relevant tuple pairs based on the relevance score for each tuple pair. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein generating, by the computing system, the list of relevant tuple pairs based on the relevance score for each tuple pair further comprises: for each respective tuple pair in the list of tuple pairs: determining, by the computing system, that the relevance score for the respective tuple pair satisfies a relevance threshold score; and in accordance with a determination that the relevance score for the respective tuple pair satisfies the relevance threshold score, adding, by the computing system, to the list of relevant tuple pairs. ### Claim 4 (depends on 2) The computer-implemented method of claim 2, wherein each tuple pair includes a representative subsequence that corresponds to a representation from the one or more representative subsequences that correspond to a representation and one media element from the one or more media elements. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the relevance score for the respective tuple pair represents a degree to which the representative subsequence that corresponds to a representation included in the respective tuple pair is relevant to the media element included in the tuple pair. ### Claim 6 (depends on 2) The computer-implemented method of claim 2, wherein the relevance-scoring machine-learned model is a multimodal required attribution detector model trained to generate a relevance score for a particular tuple pair that represents a degree to which a representative subsequence that corresponds to a representation included in the respective tuple pair is relevant to a media element included in the respective tuple pair. ### Claim 7 (depends on 2) The computer-implemented method of claim 2, wherein providing, by the computing system, the entailment output for the model output comprises: determining, by the computing system, that the entailment score for all the relevant tuple pairs in the list of relevant tuple pairs satisfy an entailment threshold score; and in accordance with a determination that the entailment score for all the relevant tuple pairs have associated entailment scores that satisfies the entailment threshold score, determining, by the computing system, that the output meets a quality measure for generated output. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, the method further comprising: in accordance with a determination that at least one respective tuple pair in the list of relevant tuple pairs has an associated entailment score that does not satisfy the entailment threshold score, determining, by the computing system, that the output does not meet a quality measure for generated output. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, wherein the entailment-scoring machine-learned model is a visual natural language inference model. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, the method further comprising: transmitting, by the computing system, the entailment output to a user computing device for display. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, wherein the one or more media elements comprise one or more of video content, image content, audio content, and interactive content. ### Claim 12 (depends on 1) The computer-implemented method of claim 1, wherein the input query is multimodal and includes at least one of the one or more media elements. ### Claim 13 (depends on 1) The computer-implemented method of claim 1, wherein the textual response includes a plurality of portions of text. ### Claim 14 (depends on 13) The computer-implemented method of claim 13, wherein identifying one or more representative subsequences that correspond to a representation based on the textual response further comprises: providing, by the computing system, a respective span in the plurality of portions of text to a representation detection machine-learned model; and receiving, by the computing system, an output from the representation detection machine-learned model, wherein the output includes one or more representative subsequences that correspond to a representation from the respective span. ### Claim 15 (depends on 1) The computer-implemented method of claim 1, wherein processing, by the computing system, the input query with the generative model to generate the model output based on the input query comprises: generating, by the computing system, a prompt as input for the generative model based on the input query. ### Claim 16 (depends on 15) The computer-implemented method of claim 15 wherein the prompt includes a natural language explanation of the input query. ### Claim 17 (depends on 1) The computer-implemented method of claim 1, wherein a number of tuple pairs in the plurality of tuple pairs are based on a number of representative subsequences that correspond to a representation in the one or more representative subsequences that correspond to a representation and a number of media elements in the one or more media elements. ### Claim 18 (depends on 1) The computer-implemented method of claim 1, the method further comprising: for each respective representative subsequence that corresponds to a representation in the one or more representative subsequences that correspond to a representation, executing, by the computing system, a web-based required attribution detector model trained to determine whether the respective representative subsequence that corresponds to a representation can be validated based on information available through a web search. ### Claim 19 (depends on 18) The computer-implemented method of claim 18, further comprising: in response to a determination that the respective representative subsequence that corresponds to a representation can be validated based on information available through a web search, executing, by the computing system, a web-based natural language inference model with web retrieval to determine a web entailment score for the respective representative subsequence that corresponds to a representation. ### Claim 20 A computing system, comprising: one or more processors; and one or more non-transitory computer-readable media that store instructions wherein, when executed by the one or more processors, the instructions cause the one or more processors to perform operations, the operations comprising: receiving an input query; processing the input query with a generative model to generate a model output based on the input query, wherein the input query comprises textual content, and wherein one or both of the input query and the output comprise one or more media elements; identifying, by the computing system, one or more representative subsequences that correspond to a representation based on the textual response; generating a plurality of tuple pairs based on the one or more representative subsequences that correspond to a representation and the one or more media elements; for each of one or more relevant tuple pairs in the plurality of tuple pairs, processing the respective tuple pair with an entailment-scoring machine-learned model to generate an entailment score for the respective tuple pair; and providing an entailment output for the model output based on the respective entailment scores generated for the one or more relevant tuple pairs.