# US 2019/0317814 A1 · Resource management of resource-controlled system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0317814 A1
- Document ID: US20190317814A1
- Status: Published application
- Kind: A1
- Application number: 16/380,311
- Filed: 2019-04-10
- Published: 2019-10-17
- Earliest claimed: 2018-04-17
- Assignees: Nokia Solutions and Networks Oy
- Applicants: Nokia Solutions and Networks
- Inventors: Yue Jin; Dimitre Kostadinov; Makram Bouzid; Armen Aghasaryan
- Web page: https://patentgrep.com/patent/US20190317814A1

## CPC

- G06F 9/5011: {the resources being hardware resources other than CPUs, Servers and Terminals}
- G06F 9/5016: {the resource being the memory}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 20/00: Machine learning
- G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An apparatus (5) manages resources of a resource-controlled system (1), e.g. data processing resources in a data processing system. The resource-controlled system (1) has resource-elasticity, namely it may operate in several states involving different amounts of system resources. The resource management apparatus (5) may select a modification action having at least one resource variation parameter. The resource variation parameter represents a resource quantity to be added or removed in the resource-controlled system (1). The apparatus (5) may update a value function of an adaptive learning agent (7) as a function of performance metrics (4) measured after the resources of the resource-controlled system have been modified by effecting the modification action. Responsive to a safety criterion not being fulfilled, a finer resource variation parameter may be substituted.

## Claims

### Claim 1

An apparatus (5) comprising:
at least one processor; and
at least one memory including computer program code,
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:
select a modification action having at least one resource variation parameter,
wherein the resource variation parameter represents a resource quantity to be added or removed in a resource-controlled system,
update a value function of an adaptive learning agent as a function of one or more performance metrics measured after resources of the resource-controlled system have been modified by effecting the modification action,
wherein the one or more performance metrics relate to the resource-controlled system,
wherein the value function represents a desirability of the resource-controlled system as a function of one or more system states,
wherein selecting the modification action further comprises:
select a first resource variation parameter, and
assess whether the modification action having the first resource variation parameter fulfills at least one safety criterion, and
substitute the first resource variation parameter by a second resource variation parameter having a finer scale resource quantity than the first resource variation parameter responsive to the safety criterion not being fulfilled.

### Claim 2 (depends on 1)

The apparatus of claim 1, further configured to perform:
modify the resources of the resource-controlled system by effecting the modification action prior to updating the value function of the adaptive learning agent.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the value function is defined for one or more system states resulting from one or more macro-modification actions, wherein a macro-modification action has at least one resource variation parameter equal to an integer multiple of a predefined resource increment,
wherein the apparatus is further configured to perform:
determine a macro-modification action adjacent to the modification action by rounding the second resource variation parameter to an integer multiple of the predefined resource increment, wherein the value function of the adaptive learning agent is updated for a system state which would have resulted from the macro-modification action adjacent to the modification action, and
modify the resources of the resource-controlled system by reversing the modification action.

### Claim 4 (depends on 3)

The apparatus of claim 3, further configured to perform:
determine a reward function value for the modification action as a function of the performance metrics measured after the resources of the resource-controlled system have been modified by effecting the modification action, and
determine a reward function value for the macro-modification action by extrapolating a reward function.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the reward function is linearly extrapolated.

### Claim 6 (depends on 1)

The apparatus of claim 1, further configured to perform:
compare a variance of the value function to a predefined threshold value to assess whether the safety criterion is fulfilled.

### Claim 7 (depends on 1)

The apparatus of claim 1, further configured to perform:
compare the resource variation parameter to a predefined threshold value to assess whether the safety criterion is fulfilled.

### Claim 8 (depends on 1)

The method of claim 1, further configured to perform:
determine a number of prior occurrences of the modification action to assess whether the safety criterion is fulfilled.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the first resource variation parameter is randomly selected.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the first resource variation parameter is selected to maximize the value function in a current system state.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein selecting the modification action further comprises selecting a resource type in a set of resource types.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the resource-controlled system comprises a data processing system, wherein the at least one resource variation parameter relates to a resource selected in the group comprising:
RAM capacity, CPU processing power, CPU bandwidth, number of processor-cores, number of virtual machines, disk I/O bandwidth and network I/O bandwidth.

### Claim 13 (depends on 12)

The apparatus of claim 12, further configured to perform:
send one or more resource modification instructions to a virtualization layer of the data processing system to effect the modification action.

### Claim 14

A method comprising:
selecting a modification action having at least one resource variation parameter, wherein the resource variation parameter represents a resource quantity to be added or removed in the resource-controlled system,
updating a value function of an adaptive learning agent as a function of one or more performance metrics measured after the resources of the resource-controlled system have been modified by effecting the modification action, wherein the one or more performance metrics relate to the resource-controlled system, wherein the value function represents a desirability of the resource-controlled system as a function of one or more system states, wherein selecting the modification action further comprises:
selecting a first resource variation parameter, and
assessing whether the modification action having the first resource variation parameter fulfills at least one safety criterion, and
substituting the first resource variation parameter by a second resource variation parameter having a finer scale resource quantity than the first resource variation parameter responsive to the safety criterion not being fulfilled.

### Claim 15 (depends on 14)

The method of claim 14, further configured to:
modifying the resources of the resource-controlled system by effecting the modification action prior to updating the value function of the adaptive learning agent.

### Claim 16 (depends on 14)

The method of claim 14, further configured to:
comparing a variance of the value function to a predefined threshold value to assess whether the safety criterion is fulfilled.

### Claim 17 (depends on 14)

The method of claim 14, further configured to:
comparing the resource variation parameter to a predefined threshold value to assess whether the safety criterion is fulfilled.

### Claim 18 (depends on 14)

The method of claim 14, further configured to:
determining a number of prior occurrences of the modification action to assess whether the safety criterion is fulfilled.

### Claim 19 (depends on 14)

The method of claim 14, wherein the first resource variation parameter is selected to maximize the value function in a current system state.

### Claim 20

A non-transitory computer program comprising executable code for causing an apparatus to perform at least the following:
selecting a modification action having at least one resource variation parameter, wherein the resource variation parameter represents a resource quantity to be added or removed in a resource-controlled system,
updating a value function of an adaptive learning agent as a function of one or more performance metrics measured after the resources of the resource-controlled system have been modified by effecting the modification action,
wherein the one or more performance metrics relate to the resource-controlled system,
wherein the value function represents a desirability of the resource-controlled system as a function of one or more system states,
wherein selecting the modification action further comprises:
selecting a first resource variation parameter, and
assessing whether the modification action having the first resource variation parameter fulfills at least one safety criterion, and
substituting the first resource variation parameter by a second resource variation parameter having a finer scale resource quantity than the first resource variation parameter responsive to the safety criterion not being fulfilled.
