# US 2024/0256875 A1 · System and method for optimizer with enhanced neural estimation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0256875 A1
- Document ID: US20240256875A1
- Status: Published application
- Kind: A1
- Application number: 18/160,201
- Filed: 2023-01-26
- Published: 2024-08-01
- Earliest claimed: 2023-01-26
- Assignees: Goldman, Sachs & Co.
- Applicants: Goldman Sachs & Co. LLC
- Inventors: Dan Popescu; Giovanni Faonte; Arthur Maghakian
- Web page: https://patentgrep.com/patent/US20240256875A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method includes receiving a plurality of inputs including domain parameters and initial weights. The method also includes providing the plurality of inputs to an optimization model. The method also includes performing, using a first layer of the optimization model, a training and optimization process based on the plurality of inputs and based on a training objective, The method also includes performing, using a second layer of the optimization model, a differencing operation on an output of the first layer. The method also includes recording, using a third layer of the optimization model, a loss based on the training objective used by the optimization model. The method also includes calculating and storing, using a fourth layer of the optimization model, metrics regarding the training and optimization process. The method also includes outputting, using the optimization model, updated weights.

## Claims

### Claim 1

A method comprising:
receiving a plurality of inputs including domain parameters and initial weights;
providing the plurality of inputs to an optimization model;
performing, using a first layer of the optimization model, a training and optimization process based on the plurality of inputs and based on a training objective;
performing, using a second layer of the optimization model, a differencing operation on an output of the first layer;
recording, using a third layer of the optimization model, a loss based on the training objective used by the optimization model;
calculating and storing, using a fourth layer of the optimization model, metrics regarding the training and optimization process; and
outputting, using the optimization model, updated weights.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
setting one or more hyperparameters of the optimization model, wherein the one or more hyperparameters include at least one of:
whether the first layer of the optimization model is a dense layer or a recurrent layer;
a number of hidden layers of the first layer of the optimization model;
a number of units per hidden layer of the first layer of the optimization model; and
a maximum number of epochs for the training objective.

### Claim 3 (depends on 1)

The method of claim 1, wherein performing, using the first layer of the optimization model, the training and optimization process includes performing model fitting based on the plurality of inputs to fit a neural network with backpropagation.

### Claim 4 (depends on 3)

The method of claim 3, wherein performing the model fitting includes:
adapting parameter learning rates in real time based on an average of first and second moments by calculating an exponential moving average of a gradient and a moving average of a squared gradient;
controlling decay rates of the exponential moving average of the gradient and the moving average of the squared gradient;
bias correcting one or more weight parameters; and
updating the initial weights using the bias corrected one or more weight parameters.

### Claim 5 (depends on 1)

The method of claim 1, wherein the plurality of inputs further includes final weight parameters, and wherein performing the training and optimization process includes performing a rebalancing process in which a path to achieve the final weight parameters is determined by the optimization model.

### Claim 6 (depends on 1)

The method of claim 1, wherein performing the training and optimization process includes performing a multi-period portfolio optimization, wherein the plurality of inputs includes returns data, risks data, volumes data, and the initial weights from a feeder model, and wherein the optimization model outputs multi-period weights for a defined time period.

### Claim 7 (depends on 6)

The method of claim 6, wherein the multi-period portfolio optimization maximizes returns for a given forecasting based on market impact predictions.

### Claim 8 (depends on 6)

The method of claim 6, further comprising performing at least one data consistency check on at least one input of the plurality of inputs, wherein the at least one data consistency check includes one or more of:
checking that projected returns and volumes are in a certain range;
checking that a variance-covariance matrix associated with the risks data is positive semidefinite;
checking that the variance-covariance matrix associated with the risks data is positive definite;
checking a consistency of a number of securities;
checking validity of the volumes data;
checking a feasibility of one or more constraints; and
performing a boundless check on the plurality of inputs.

### Claim 9

An apparatus comprising:
at least one processor supporting optimization, wherein the at least one processor is configured to:
receive a plurality of inputs including domain parameters and initial weights;
provide the plurality of inputs to an optimization model;
perform, using a first layer of the optimization model, a training and optimization process based on the plurality of inputs and based on a training objective;
perform, using a second layer of the optimization model, a differencing operation on an output of the first layer;
record, using a third layer of the optimization model, a loss based on the training objective used by the optimization model;
calculate and store, using a fourth layer of the optimization model, metrics regarding the training and optimization process; and
output, using the optimization model, updated weights.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the at least one processor is further configured to:
set one or more hyperparameters of the optimization model, wherein the one or more hyperparameters include at least one of:
whether the first layer of the optimization model is a dense layer or a recurrent layer;
a number of hidden layers of the first layer of the optimization model;
a number of units per hidden layer of the first layer of the optimization model; and
a maximum number of epochs for the training objective.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein, to perform, using the first layer of the optimization model, the training and optimization process, the at least one processor is further configured to perform model fitting based on the plurality of inputs to fit a neural network with backpropagation.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein, to perform the model fitting, the at least one processor is further configured to:
adapt parameter learning rates in real time based on an average of first and second moments by calculating an exponential moving average of a gradient and a moving average of a squared gradient;
control decay rates of the exponential moving average of the gradient and the moving average of the squared gradient;
bias correct one or more weight parameters; and
update the initial weights using the bias corrected one or more weight parameters.

### Claim 13 (depends on 9)

The apparatus of claim 9, wherein the plurality of inputs further includes final weight parameters, and wherein, to perform the training and optimization process, the at least one processor is further configured to perform a rebalancing process in which a path to achieve the final weight parameters is determined by the optimization model.

### Claim 14 (depends on 9)

The apparatus of claim 9, wherein, to perform the training and optimization process, the at least one processor is further configured to perform a multi-period portfolio optimization, wherein the plurality of inputs includes returns data, risks data, volumes data, and the initial weights from a feeder model, and wherein the optimization model outputs multi-period weights for a defined time period.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the multi-period portfolio optimization maximizes returns for a given forecasting based on market impact predictions.

### Claim 16 (depends on 14)

The apparatus of claim 14, wherein the at least one processor is further configured to perform at least one data consistency check on at least one input of the plurality of inputs, wherein the at least one data consistency check includes one or more of:
a check that projected returns and volumes are in a certain range;
a check that a variance-covariance matrix associated with the risks data is positive semidefinite;
a check that the variance-covariance matrix associated with the risks data is positive definite;
a check of a consistency of a number of securities;
a check of validity of the volumes data;
a check of a feasibility of one or more constraints; and
a boundless check on the plurality of inputs.

### Claim 17

A non-transitory computer readable medium containing instructions that support optimization and that when executed cause at least one processor to:
receive a plurality of inputs including domain parameters and initial weights;
provide the plurality of inputs to an optimization model;
perform, using a first layer of the optimization model, a training and optimization process based on the plurality of inputs and based on a training objective;
perform, using a second layer of the optimization model, a differencing operation on an output of the first layer;
record, using a third layer of the optimization model, a loss based on the training objective used by the optimization model;
calculate and store, using a fourth layer of the optimization model, metrics regarding the training and optimization process; and
output, using the optimization model, updated weights.

### Claim 18 (depends on 17)

The non-transitory computer readable medium of claim 17, further containing instructions that when executed cause the at least one processor to:
set one or more hyperparameters of the optimization model, wherein the one or more hyperparameters include at least one of:
whether the first layer of the optimization model is a dense layer or a recurrent layer;
a number of hidden layers of the first layer of the optimization model;
a number of units per hidden layer of the first layer of the optimization model; and
a maximum number of epochs for the training objective.

### Claim 19 (depends on 17)

The non-transitory computer readable medium of claim 17, wherein, to perform, using the first layer of the optimization model, the training and optimization process, the non-transitory computer readable medium further contains instructions that when executed cause the at least one processor to perform model fitting based on the plurality of inputs to fit a neural network with backpropagation, wherein, to perform the model fitting, the instructions when executed further cause the at least one processor to:
adapt parameter learning rates in real time based on an average of first and second moments by calculating an exponential moving average of a gradient and a moving average of a squared gradient;
control decay rates of the exponential moving average of the gradient and the moving average of the squared gradient;
bias correct one or more weight parameters; and
update the initial weights using the bias corrected one or more weight parameters.

### Claim 20 (depends on 17)

The non-transitory computer readable medium of claim 17, wherein, to perform the training and optimization process, the non-transitory computer readable medium further contains instructions that when executed cause the at least one processor to perform a multi-period portfolio optimization, wherein the plurality of inputs includes returns data, risks data, volumes data, and the initial weights from a feeder model, and wherein the optimization model outputs multi-period weights for a defined time period, and wherein the multi-period portfolio optimization maximizes returns for a given forecasting based on market impact predictions.
