# US 12,222,849 B2 · Infrastructure refactoring via fuzzy upside down reinforcement learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,222,849 B2
- Document ID: US12222849B2
- Status: Granted
- Kind: B2
- Application number: 17/246,801
- Filed: 2021-05-03
- Published: 2022-11-03
- Granted: 2025-02-11
- Earliest claimed: 2021-05-03
- Assignees: Bank of America Corporation
- Applicants: Bank of America Corporation
- Inventors: Madhu Sudhanan Krishnamoorthy; Sreeram Raghavan; Rajarajan Pandiyan
- Web page: https://patentgrep.com/patent/US12222849B2

## CPC

- G06F 11/3688: {for test execution, e.g. scheduling of test suites}
- G06F 11/3457: {Performance evaluation by simulation}
- G06F 11/3616: {using software metrics}
- G06F 11/3698: Environments for analysis, debugging or testing of software
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06N 3/092: Reinforcement learning
- G06N 5/02: Knowledge representation; Symbolic representation
- G06N 20/00: Machine learning
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/096: Transfer learning
- G06N 5/048: Fuzzy inferencing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Apparatus and methods for refactoring infrastructure. The methods may include (a) defining parameters of an application landscape. The methods may include (b) stress-testing an application in a simulated environment based on: the parameters; and a simulated input to the application. The methods may include (c) identifying a state of stress of the application based on output of the stress-test. The methods may include (d) repeating (b)-(c) with a different simulated input until the state of stress satisfies a predetermined stochastic threshold. The methods may include (e) providing the state of stress to an upside down reinforcement learning (“UDRL”) engine. The methods may include (f) comparing a throughput corresponding to the state of stress to a benchmark throughput. The methods may include (g) redefining the parameters. The methods may include (h) repeating (a)-(f) until a threshold proximity to the benchmark throughput is reached.

## Claims

### Claim 1

A method for improving application throughput by refactoring infrastructure, the method comprising:
(a) defining a set of parameters of an application landscape;
(b) stress-testing an application in a simulated environment based on:
the parameters; and
a simulated input to the application;
the stress-testing comprising (1) generating a simulated hardware environment, (2) instantiating the application in the simulated hardware environment, (3) porting in the application landscape parameters, and (4) detecting an output comprising an application throughput;
(c) identifying an initial state of stress of the application based on output of the stress-test;
(d) generating an updated state of stress based on linear combinations of fuzzified basis vectors based on the initial state of stress, the fuzzified basis vectors based on the application landscape parameters;
(e) repeating, sequentially, (b) and (d), with a new application parameter in the set of parameters, and the initial state of stress replaced by the latest updated state of stress, until the updated state of stress satisfies a predetermined stochastic threshold;
(f) providing the updated state of stress to an upside down reinforcement learning (“UDRL”) engine;
(g) comparing a throughput corresponding to the updated state of stress to a benchmark throughput;
(h) re-weighting the parameters; and
(i) repeating (a)-(i) until a threshold proximity to the benchmark throughput is reached and outputting a set of parameters and weights associated with the benchmark throughput.

### Claim 2 (depends on 1)

The method of claim 1 wherein the application parameters include:
a cost metric;
a complexity gradient; and
an integration metric.

### Claim 3 (depends on 1)

The method of claim 1 wherein:
the UDRL engine is configured to evaluate a reward based on closeness of a parameter to a predetermined reward value; and
the reward is configured to accelerate convergence of a throughput optimization corresponding to the application.

### Claim 4 (depends on 3)

The method of claim 3 wherein the convergence includes matching a throughput metric to a bracket anchored to the benchmark throughput, the throughput metric corresponding to performance of the application in the updated state of stress.

### Claim 5 (depends on 1)

The method of claim 1 further comprising storing the updated state of stress in a database record.

### Claim 6 (depends on 5)

The method of claim 5 further comprising storing a feedback policy throughput in the database record.

### Claim 7 (depends on 6)

The method of claim 6 further comprising, when the application is a first application, and the benchmark throughput is a first benchmark throughput:
stress-testing a second application in the simulated environment; and
comparing a throughput of the second application to a benchmark throughput that is defined as feedback policy throughput.

### Claim 8

A method for improving application throughput by refactoring infrastructure, the method comprising:
defining a reward value corresponding to a first application landscape parameter and a reward value corresponding to a second application landscape parameter;
receiving from an application stress-test agent a first stress-state that:
corresponds to performance of an application under test, the test comprising generating a simulated hardware environment, instantiating the application in the simulated hardware environment, porting in the first application landscape parameter, and detecting an output comprising a first application throughput; and
includes the first application landscape parameter having a parameter value;
based on a comparison between the reward value and the parameter value, instructing the agent to generate a new stress-state that:
corresponds to performance of the application under test, the test comprising generating a simulated hardware environment, instantiating the application in the simulated hardware environment, porting in the second application landscape parameter, and detecting an output comprising a second application throughput; and
includes the second application landscape parameter;
the new stress-state based on linear combinations of fuzzified basis vectors, the fuzzified basis vectors based on the first application landscape parameter.

### Claim 9 (depends on 8)

The method of claim 8 wherein the application parameter includes:
a cost metric;
a complexity gradient; and
an integration metric.

### Claim 10 (depends on 8)

The method of claim 8 further comprising an upside down reinforcement learning (“UDRL”) engine that is configured to evaluate a reward based on closeness of a parameter to a predetermined reward value;
wherein the reward is configured to accelerate convergence of a throughput optimization corresponding to the application.

### Claim 11 (depends on 10)

The method of claim 10 wherein the convergence includes matching a throughput metric to a bracket anchored to a benchmark throughput, the throughput metric corresponding to performance of the application in the stress-state.

### Claim 12 (depends on 11)

The method of claim 11 further comprising storing the stress-state in a database record.

### Claim 13 (depends on 12)

The method of claim 12 further comprising storing a feedback policy throughput in the database record.

### Claim 14 (depends on 13)

The method of claim 13 further comprising, when the application is a first application, and the benchmark throughput is a first benchmark throughput:
stress-testing a second application in a simulated environment; and
comparing a throughput of the second application to a benchmark throughput that is defined as feedback policy throughput.

### Claim 15

A method for improving application throughput by refactoring infrastructure, the method comprising:
defining a reward value corresponding to an application landscape parameter;
receiving from an application stress-test agent a first stress-state that:
corresponds to performance of an application under test, the test comprising generating a simulated hardware environment, instantiating the application in the simulated hardware environment, porting in the application landscape parameter, and detecting an output comprising an application throughput; and
is based on fuzzified application landscape parameters, each having a fuzzified landscape parameter value;
based on a comparison between the reward value and a fuzzified parameter value, the comparison comprising linear combinations of fuzzified basis vectors, instructing the agent to generate a new stress-state that:
corresponds to performance of the application under test; and
includes a second application landscape parameter.

### Claim 16 (depends on 15)

The method of claim 15 wherein the application parameters include:
a cost metric;
a complexity gradient; and
an integration metric.

### Claim 17 (depends on 15)

The method of claim 15 further comprising an upside down reinforcement learning (“UDRL”) engine that is configured to evaluate a reward based on closeness of a parameter to a predetermined reward value;
wherein the reward is configured to accelerate convergence of a throughput optimization corresponding to the application.

### Claim 18 (depends on 17)

The method of claim 17 wherein the convergence includes matching a throughput metric to a bracket anchored to a benchmark throughput, the throughput metric corresponding to performance of the application in the new stress-state.

### Claim 19 (depends on 18)

The method of claim 18 further comprising storing the new stress-state in a database record.

### Claim 20 (depends on 19)

The method of claim 19 further comprising storing a feedback policy throughput in the database record.

### Claim 21 (depends on 20)

The method of claim 20 further comprising, when the application is a first application, and the benchmark throughput is a first benchmark throughput:
stress-testing a second application in a simulated environment; and
comparing a throughput of the second application to a benchmark throughput that is defined as feedback policy throughput.