US 12,619,819 B1Granted

Identifying provenance information of a data item generated by a generative machine learning model

AssigneeAmazon Technologies, Inc.

InventorsJiangtao Zhang, Ramu Panayappan, Mark Fawaz, Vijay Dheeraj Reddy Mandadi, Sreenaath Vasudevan, Raviprasad V Mummidi

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Metadata may be identified for text generated by a generative machine learning model. A text is obtained and a weighting scheme determine for performing similarity analysis. Different similarity analysis techniques are performed that compare the text with representations of texts in the training data set for the generative machine learning model. Final similarity scores are generated that combine the different similarity analysis techniques according to the weighting scheme and are used to select metadata to provide that is relevant to the text.

Claims20

Dependent claims

A system, comprising:

at least one processor;
and
a memory, storing program instructions that when executed by the at least one processor, cause the at least one processor to implement a metadata identification system, configured to:
receive, via an interface, a request to search for metadata relevant to code generated by a language model trained using a machine learning technique applied to training data comprising a plurality of code;
identify a weighting scheme for performing similarity analysis with respect to the code generated by the language model;
cause performance of a plurality of different similarity analysis techniques that compare the code with one or more representations of the plurality of code of the training data set;
generate respective final similarity scores between the code and the one or more representations of the plurality of code according to the weighting scheme, wherein the weighting scheme indicates respective weights for combining individual similarity scores generated by the different similarity analysis techniques into the respective final similarity scores;
select one representation of the one or more representations according to the respective final similarity scores between the code and the one or more representations of the plurality of code;
and
return, via the interface, metadata corresponding to the selected one representation as descriptive of the code generated by the language model.

7 more elements

A method, comprising:

obtaining, at a metadata identification system, a data item generated by a generative machine learning model trained using a machine learning technique applied to training data comprising a plurality of data items;
determining, by the metadata identification system, a weighting scheme for performing similarity analysis with respect to the data item generated by the generative machine learning model;
performing, by the metadata identification system, a plurality of different similarity analysis techniques that compare the data item with one or more representations of the plurality of data items of the training data set;
generating, by the metadata identification system, respective final similarity scores between the data item and the one or more representations of the plurality of data items according to the weighting scheme, wherein the weighting scheme indicates respective weights for combining individual similarity scores generated by the different similarity analysis techniques into the respective final similarity scores;
selecting, by the metadata identification system, one representation of the one or more representations according to the respective final similarity scores between the data item and the one or more representations of the plurality of data items;
and
providing, by the metadata identification system, metadata corresponding to the selected one representation as descriptive of the data item generated by the generative machine learning model.

4 more elements

One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices cause the one or more computing devices to implement:

receiving a request to search for metadata relevant to a text generated by a large language model (LLM) trained using a machine learning technique applied to training data comprising a plurality of texts;
identifying a weighting scheme for performing similarity analysis with respect to the text generated by the LLM;
causing performance of a plurality of different similarity analysis techniques that compare the text with one or more representations of the plurality of texts of the training data set;
generating respective final similarity scores between the text and the one or more representations of the plurality of texts according to the weighting scheme, wherein the weighting scheme indicates respective weights for combining individual similarity scores generated by the different similarity analysis techniques into the respective final similarity scores;
selecting one representation of the one or more representations according to the respective final similarity scores between the text and the one or more representations of the plurality of texts;
and
returning metadata corresponding to the selected one representation as descriptive of the text generated by the LLM.

4 more elements

Dependencies are inferred from the claim text.