US 2026/0065068 A1Published application

Eliciting black-box representations from machine learning models through self-queries

AssigneesRobert Bosch GmbH, Carnegie Mellon University

InventorsDylan Jiang SAM, Marc FINZI, Jeremy KOLTER, Devin T. WILLMOTT, Wan-Yi LIN

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods for determining black-box representations of machine learning models when information pertaining to internal states or parameters of the models are not accessible are disclosed. By using outputs of the model instead of internal states, the black-box representation is model-agnostic and provides a reliable and robust representation of the model using an external lens. The black-box representation is generated using responses from the model to a series of initialization and elicitation questions that quantify the confidence that the model has in answers it just returned. The black-box representation is then used as a training dataset for a linear classifier in order to learn performance metrics about the model.

Claims20

Dependent claims

A computer-implemented method for analyzing a large language model (LLM), comprising:

providing, via computing devices of a service provider network, a first text-based data sample to the LLM, wherein the first text-based data sample is formulated as an initialization question and is selected from a first dataset of text-based data samples;
receiving, from an Application Programming Interface (API) associated with the LLM, data indicating a response to the initialization question;
providing a second text-based data sample to the LLM, wherein the second text-based data sample is formulated as an elicitation question and is selected from a second dataset of text-based data samples;
receiving, from the API associated with the LLM, data indicating a response to the elicitation question, wherein the response to the elicitation question is one of two binary response options;
determining a black-box representation of the LLM based on the data indicating the responses to the initialization and elicitation questions and on data indicating subsequent responses of the LLM when provided with other text-based data samples of the first and second datasets;
and
providing the black-box representation as a training dataset to a linear classifier, wherein the linear classifier is trained to output performance data about the LLM.

4 more elements

A computer-implemented method for analyzing a large language model (LLM), comprising:

providing, via computing devices of a service provider network, text-based data samples of a first dataset and of a second dataset to the LLM, wherein:
text-based data samples of the first dataset are formulated as initialization questions;
and
text-based data samples of the second dataset are formulated as elicitation questions;
receiving, from an Application Programming Interface (API) associated with the LLM, data indicating responses to the initialization questions and to the elicitation questions, wherein the data indicating the responses to the elicitation questions are one of two binary response options;
determining a black-box representation of the LLM based on the data indicating the responses;
and
providing the black-box representation as a training dataset to a linear classifier, wherein the linear classifier is trained to output performance data about the LLM.

5 more elements

Dependencies are inferred from the claim text.