# US 2019/0317734 A1 · Methods, systems, articles of manufacture and apparatus to improve code characteristics

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0317734 A1
- Document ID: US20190317734A1
- Status: Published application
- Kind: A1
- Application number: 16/456,984
- Filed: 2019-06-28
- Published: 2019-10-17
- Earliest claimed: 2019-06-28
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Li Chen; Justin Gottschlich; Alexander Heinecke; Zheng Zhang; Shengtian Zhou
- Web page: https://patentgrep.com/patent/US20190317734A1

## CPC

- G06F 8/33: Intelligent editors
- G06F 8/77: Software metrics
- G06F 8/316: {Aspect-oriented programming techniques}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/092: Reinforcement learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, apparatus, systems and articles of manufacture are disclosed to improve code characteristics. An example apparatus includes a weight manager to apply a first weight value to a first objective function, a state identifier to identify a first state corresponding to candidate code, and an action identifier to identify candidate actions corresponding to the identified first state. The example apparatus also includes a reward calculator to determine reward values corresponding to respective ones of (a) the identified first state, (b) one of the candidate actions and (c) the first weight value, and a quality function definer to determine a relative highest state and action pair reward value based on respective ones of the reward values

## Claims

### Claim 1

An apparatus to modify candidate code, the apparatus comprising:
a weight manager to apply a first weight value to a first objective function;
a state identifier to identify a first state corresponding to the candidate code;
an action identifier to identify candidate actions corresponding to the identified first state;
a reward calculator to determine reward values corresponding to respective ones of (a) the identified first state, (b) one of the candidate actions and (c) the first weight value; and
a quality function definer to determine a relative highest state and action pair reward value based on respective ones of the reward values.

### Claim 2 (depends on 1)

The apparatus as defined in claim 1, further including a machine learning engine to estimate a quality function by applying the respective ones of the reward values to a neural network.

### Claim 3 (depends on 2)

The apparatus as defined in claim 2, wherein the quality function definer is to define the quality function as a Bellman estimation.

### Claim 4 (depends on 1)

The apparatus as defined in claim 1, further including an objective function selector to:
select a second objective function; and
invoke the weight manager to apply a second weight value to the second objective function.

### Claim 5 (depends on 4)

The apparatus as defined in claim 4, wherein the reward calculator is to calculate an aggregate reward for the reward values based on the first and second objective functions.

### Claim 6 (depends on 1)

The apparatus as defined in claim 1, wherein the state identifier is to iteratively identify additional states corresponding to the candidate code, the action identifier to identify additional candidate actions corresponding to the respective additional states.

### Claim 7 (depends on 1)

The apparatus as defined in claim 1, wherein the weight manager is to determine the first weight value for the first objective function and a second weight value for a second objective function based on behavioral observation of a code developer associated with the candidate code.

### Claim 8

A non-transitory computer readable storage medium comprising computer readable instructions that, when executed, cause at least one processor to at least:
apply a first weight value to a first objective function;
identify a first state corresponding to candidate code;
identify candidate actions corresponding to the identified first state;
determine reward values corresponding to respective ones of (a) the identified first state, (b) one of the candidate actions and (c) the first weight value; and
determine a relative highest state and action pair reward value based on respective ones of the reward values.

### Claim 9 (depends on 8)

The non-transitory computer readable storage medium as defined in claim 8, wherein the instructions, when executed, cause the at least one processor to estimate a quality function by applying the respective ones of the reward values to a neural network.

### Claim 10 (depends on 9)

The non-transitory computer readable storage medium as defined in claim 9, wherein the instructions, when executed, cause the at least one processor to define the quality function as a Bellman estimation.

### Claim 11 (depends on 8)

The non-transitory computer readable storage medium as defined in claim 8, wherein the instructions, when executed, cause the at least one processor to:
select a second objective function; and
invoke the weight manager to apply a second weight value to the second objective function.

### Claim 12 (depends on 11)

The non-transitory computer readable storage medium as defined in claim 11, wherein the instructions, when executed, cause the at least one processor to calculate an aggregate reward for the reward values based on the first and second objective functions.

### Claim 13 (depends on 8)

The non-transitory computer readable storage medium as defined in claim 8, wherein the instructions, when executed, cause the at least one processor to iteratively identify additional states corresponding to the candidate code, the action identifier to identify additional candidate actions corresponding to the respective additional states.

### Claim 14 (depends on 8)

The non-transitory computer readable storage medium as defined in claim 8, wherein the instructions, when executed, cause the at least one processor to determine the first weight value for the first objective function and a second weight value for a second objective function based on behavioral observation of a code developer associated with the candidate code.

### Claim 15

A computer-implemented method to modify candidate code, the method comprising:
applying, by executing an instruction with at least one processor, a first weight value to a first objective function;
identifying, by executing an instruction with the at least one processor, a first state corresponding to candidate code;
identifying, by executing an instruction with the at least one processor, candidate actions corresponding to the identified first state;
determining, by executing an instruction with the at least one processor, reward values corresponding to respective ones of (a) the identified first state, (b) one of the candidate actions and (c) the first weight value; and
determining, by executing an instruction with the at least one processor, a relative highest state and action pair reward value based on respective ones of the reward values.

### Claim 16 (depends on 15)

The method as defined in claim 15, further including estimating a quality function by applying the respective ones of the reward values to a neural network.

### Claim 17 (depends on 16)

The method as defined in claim 16, further including defining the quality function as a Bellman estimation.

### Claim 18 (depends on 15)

The method as defined in claim 15, further including:
selecting a second objective function; and
invoking the weight manager to apply a second weight value to the second objective function.

### Claim 19 (depends on 18)

The method as defined in claim 18, further including calculating an aggregate reward for the reward values based on the first and second objective functions.

### Claim 20 (depends on 15)

The method as defined in claim 15, further including iteratively identifying additional states corresponding to the candidate code, the action identifier to identify additional candidate actions corresponding to the respective additional states.

### Claim 21 (depends on 15)

The method as defined in claim 15, further including determining the first weight value for the first objective function and a second weight value for a second objective function based on behavioral observation of a code developer associated with the candidate code.
