# US 2025/0384663 A1 · Influential data selection for neural network training

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0384663 A1
- Document ID: US20250384663A1
- Status: Published application
- Kind: A1
- Application number: 19/238,113
- Filed: 2025-06-13
- Published: 2025-12-18
- Earliest claimed: 2024-06-14
- Applicants: GDM Holding LLC
- Inventors: Zhiwei Deng; Tao Li; Yang Li
- Web page: https://patentgrep.com/patent/US20250384663A1

## CPC

- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, methods, and apparatus, including computer programs encoded on computer storage media for selecting, from a training data set of training examples, a subset of training examples that will be used for training a neural network by selecting the subset of training examples whose combined gradients over time match a target gradient trajectory.

## Claims

### Claim 1

A method performed by one or more computers, the method comprising:
obtaining a training data set for training a neural network having a plurality of model parameters, the training data set comprising a plurality of training examples;
obtaining a target data set for evaluating the neural network on a particular set of one or more tasks, the target data set comprising a plurality of target examples;
obtaining a model parameter trajectory that comprises, for each time point in a sequence of time points during the training of the neural network, respective example values of the model parameters at the time point;
for each time point in the sequence of time points:
determining first gradients for the training examples in the training data set with respect to the model parameters at the time point;
determining a second gradient of the target data set with respect to the model parameters at the time point;
determining a respective first projection of each of the first gradients onto a respective subspace for the time point;
determining a second projection of the second gradient for the time point onto the respective subspace for the time point; and
selecting a subset of the training data set as a final training data set for training the neural network by performing an optimization to identify a subset of the training data set that minimizes an error between (i) a target set comprising the second projections for time points and (ii) a training set of weighted first projections for the time points, wherein the weighted first projection for each of the time points is a weighted combination of the respective first projections for the time point in accordance with respective weights for each of the training examples.

### Claim 2 (depends on 1)

The method of claim 1, wherein each of the target examples corresponds to one of the training examples.

### Claim 3 (depends on 1)

The method of claim 1, wherein the target examples are not included in the plurality of training examples.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
training the neural network on the selected subset of the training examples.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
after training the neural network on the selected subset of the training examples:
receiving a new input; and
processing the new input using the neural network to generate a respective output for each of one or more of the particular set of one or more tasks.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining a second gradient of the target data set with respect to the model parameters comprises:
determining, in accordance with the respective example values of the model parameters at the time point, a respective initial second gradient of each target example in the target data set with respect to the model parameters; and
combining the respective initial second gradients to generate the second gradient.

### Claim 7 (depends on 1)

The method of claim 1, wherein obtaining a model parameter trajectory that comprises, for each time point in a sequence of time points during the training of the neural network, respective example values of the model parameters at the time point comprises:
selecting an example subset of the training data set; and
training the neural network on the example subset of the training data set across a plurality of training iterations, wherein each time point corresponds to a respective training iteration and wherein the respective example values of the model parameters at the time point are values of the model parameters after the corresponding training iteration during the training of the neural network on the example subset of the training data set has been completed.

### Claim 8 (depends on 7)

The method of claim 7, wherein each time point corresponds to a respective training epoch during the training of the neural network on the example subset of the training data set.

### Claim 9 (depends on 7)

The method of claim 7, wherein selecting an example subset of the training data set comprises randomly selecting a fixed number of training examples from the training data set.

### Claim 10 (depends on 4, 7)

The method of claim 7, when dependent on claim 4, wherein the neural network is trained on the example subset of the training data for fewer training iterations than a number of training iterations performed during the training of the neural network on the selected subset.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
for each time point in the sequence of time points:
determining the respective subspace for the time point.

### Claim 12 (depends on 6, 11)

The method of claim 11, when dependent on claim 6, wherein determining the respective subspace for the time point comprises:
determining the respective subspace for the time point from at least some of the respective initial second gradients for the time point.

### Claim 13 (depends on 12)

The method of claim 12, wherein determining the respective subspace for the time point from the respective initial second gradients for the time point comprises:
determining principal components of a matrix of at least some of the initial second gradients; and
selecting, as the respective subspace for the time point, a top K orthonormal basis of the principal components.

### Claim 14 (depends on 1)

The method of claim 1, wherein the optimization identifies a subset of the training data set that minimizes an error between (i) the second set of the second projections for each of the time points and (ii) the first set of weighted first projections for each of the time points subject to a constraint on an L0 norm of the respective weights for the training examples.

### Claim 15 (depends on 14)

The method of claim 14, wherein performing the optimization comprises performing an orthogonal matching pursuit algorithm.

### Claim 16 (depends on 15)

The method of claim 15, wherein performing the optimization comprises performing a bounded version of the orthogonal matching pursuit algorithm.

### Claim 17 (depends on 1)

The method of claim 1, wherein, prior to selecting the subset, the neural network has been trained on one or more initial sets of training data.

### Claim 18

A method performed by one or more computers, the method comprising:
receiving a new network input; and
processing the new network input using a neural network to generate a new network output, wherein the neural network has been trained on a final training data set that has been generated by performing the operations comprising:
obtaining a training data set for training a neural network having a plurality of model parameters, the training data set comprising a plurality of training examples;
obtaining a target data set for evaluating the neural network on a particular set of one or more tasks, the target data set comprising a plurality of target examples;
obtaining a model parameter trajectory that comprises, for each time point in a sequence of time points during the training of the neural network, respective example values of the model parameters at the time point;
for each time point in the sequence of time points:
determining first gradients for the training examples in the training data set with respect to the model parameters at the time point;
determining a second gradient of the target data set with respect to the model parameters at the time point;
determining a respective first projection of each of the first gradients onto a respective subspace for the time point;
determining a second projection of the second gradient for the time point onto the respective subspace for the time point; and
selecting a subset of the training data set as a final training data set for training the neural network by performing an optimization to identify a subset of the training data set that minimizes an error between (i) a target set comprising the second projections for time points and (ii) a training set of weighted first projections for the time points, wherein the weighted first projection for each of the time points is a weighted combination of the respective first projections for the time point in accordance with respective weights for each of the training examples.

### Claim 19 (depends on 1)

The method of claim 1, wherein training the neural network comprises instruction tuning the neural network after the NN has been pre-trained.

### Claim 20 (depends on 19)

The method of claim 19, further comprising:
performing the instruction tuning of the neural network using the selected subset of the training examples.

### Claim 21 (depends on 1)

The method of claim 1, wherein the neural network is a generative neural network that generates an output token sequence from an input token sequence including the input prompt, and wherein the generative neural network is configured to process the input token sequence to generate for each position in the output token sequence, a respective score for each token in a vocabulary of output tokens.

### Claim 22 (depends on 1)

The method of claim 1, wherein the neural network is a generative neural network that processes an input prompt to generate, as output, a data item.

### Claim 23 (depends on 22)

The method of claim 22, wherein the data item comprises a language and/or image and/or audio response to the prompt.

### Claim 24 (depends on 22)

The method of claim 22, wherein the data item comprises an image or audio response to the prompt.

### Claim 25 (depends on 22)

The method of claim 22, wherein the input prompt comprises an input image and wherein the output data item is a classification data item that identifies a label for an object class to which the input belongs, and wherein the object class corresponds to a class of object depicted in the input image.

### Claim 26

A system comprising:
one or more computers; and
one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform the operations comprising:
obtaining a training data set for performing instruction tuning of a neural network having a plurality of model parameters, the training data set comprising a plurality of training examples;
obtaining a target data set for evaluating the neural network on a particular set of one or more instruction following tasks, the target data set comprising a plurality of target examples;
obtaining a model parameter trajectory that comprises, for each time point in a sequence of time points during the training of the neural network, respective example values of the model parameters at the time point;
for each time point in the sequence of time points:
determining first gradients for the training examples in the training data set with respect to the model parameters at the time point;
determining a second gradient of the target data set with respect to the model parameters at the time point;
determining a respective first projection of each of the first gradients onto a respective subspace for the time point;
determining a second projection of the second gradient for the time point onto the respective subspace for the time point; and
selecting a subset of the training data set as a final training data set for training the neural network by performing an optimization to identify a subset of the training data set that minimizes an error between (i) a target set comprising the second projections for time points and (ii) a training set of weighted first projections for the time points, wherein the weighted first projection for each of the time points is a weighted combination of the respective first projections for the time point in accordance with respective weights for each of the training examples.
