# US 2024/0403704 A1 · Machine Learning-Based Technique for Model Provenance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0403704 A1
- Document ID: US20240403704A1
- Status: Published application
- Kind: A1
- Application number: 18/462,351
- Filed: 2023-09-06
- Published: 2024-12-05
- Earliest claimed: 2023-06-01
- Applicants: HiddenLayer, Inc.
- Inventors: David Beveridge; Andrew Davis
- Web page: https://patentgrep.com/patent/US20240403704A1

## CPC

- G06F 21/554: {involving event detection and direct action}
- G06N 20/00: Machine learning
- G06F 2221/034: Test or assess a computer or a system
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Data is received that characterizes artefacts associated with each of a plurality of layers of a first machine learning model. Fingerprints are generated corresponding to each of the artefacts in each layer. The generated fingerprints can collectively form a model indicator for the first machine learning model. A second machine learning model then determines, based on the generated fingerprints, whether the first machine learning model is derived from another machine learning model. Data provided this characterization can be provided to a consuming application or process. This second machine learning model can be trained model with historical fingerprints having a known provenance classification. Related apparatus, systems, techniques and articles are also described.

## Claims

### Claim 1

A method comprising:
receiving data characterizing artefacts associated with each of a plurality of layers of a first machine learning model;
generating, for each layer of the first machine learning model, fingerprints corresponding to each of the artefacts in such layer, the generated fingerprints collectively forming a model indicator for the first machine learning model, at least a portion of the artefacts having two or more different fingerprints;
determining, using a second machine learning model and based on the generated fingerprints, whether the first machine learning model is derived from another machine learning model; and
providing data characterizing the determining to a consuming application or process.

### Claim 2 (depends on 1)

The method of claim 1, wherein the artefacts comprise: weights, biases, running mean, and running variance.

### Claim 3 (depends on 1)

The method of claim 1, wherein the determining is conducted on an fingerprint-by-fingerprint basis.

### Claim 4 (depends on 1)

The method of claim 1, wherein the determining is conducted on a model indicator-by-model indicator basis.

### Claim 5 (depends on 4)

The method of claim 4 further comprising: generating, by the second machine learning model, a score characterizing the level of similarity, wherein the score forms part of the provided data.

### Claim 6 (depends on 1)

The method of claim 1, wherein at least one of the artefacts only has a single corresponding generated fingerprint.

### Claim 7 (depends on 1)

The method of claim 1, wherein at least one of the artefacts has a plurality of corresponding generated fingerprints.

### Claim 8 (depends on 1)

The method of claim 1 further comprising:
initiating, by the consuming application or process, a remediation operation relative to the first machine learning model.

### Claim 9 (depends on 8)

The method of claim 8, wherein the remediation operation comprises:
isolating or blocking access to the first machine learning model.

### Claim 10 (depends on 8)

The method of claim 8, wherein the remediation operation comprises:
partially disabling functionality of the first machine learning model.

### Claim 11 (depends on 8)

The method of claim 8, wherein the remediation operation comprises:
logging inputs and/or outputs of the first machine learning model.

### Claim 12 (depends on 8)

The method of claim 8, wherein the remediation operation comprises:
poisoning the first machine learning model.

### Claim 13 (depends on 1)

The method of claim 1 further comprising:
training the provenance machine learning model with historical fingerprints having a known provenance classification.

### Claim 14 (depends on 1)

The method of claim 1, wherein the determining comprises one or more of: storing the data characterizing the determining in physical persistence, loading the data characterizing the determining into memory, transmitting the data characterizing the determining over a network to a remote computing device, or causing the data characterizing the determining to be displayed in a graphical user interface.

### Claim 15

A method comprising:
receiving data comprising a model indicator of a first machine learning model, the model indicator comprising a plurality of fingerprints corresponding to each of a plurality of artefacts forming part of the first machine learning model, at least a portion of the artefacts having two or more different fingerprints;
determining, using a second machine learning model and based on the signatures in the received model indicator, whether the first machine learning model is derived from another machine learning model; and
providing data characterizing the determining to a consuming application or process.

### Claim 16

A method comprising:
receiving data characterizing artefacts associated with a machine learning model;
generating fingerprints using two or more fingerprint generation techniques corresponding to each of the artefacts; and
providing the generated fingerprints to a consuming application or process.

### Claim 17 (depends on 16)

The method of claim 16, wherein the fingerprints are generated by determining a means distance of a manifold representing an artefact from a point in space.

### Claim 18 (depends on 16)

The method of claim 16, wherein the fingerprints are generated by measuring fluctuations of a manifold representing an artefact relative to a mean distance.

### Claim 19 (depends on 16)

The method of claim 16, wherein the fingerprints are generated by a combination of determining a means distance of a manifold representing an artefact from a point in space and measuring fluctuations of the manifold relative to a mean distance.

### Claim 20 (depends on 16)

The method of claim 16, wherein the fingerprints are generated by measuring frequency of values in an artefact relative to expected quantized values.
