# US 2015/0178635 A1 · Deciding an optimal action in consideration of risk

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0178635 A1
- Document ID: US20150178635A1
- Status: Published application
- Kind: A1
- Application number: 14/635,316
- Filed: 2015-03-02
- Published: 2015-06-25
- Earliest claimed: 2011-02-15
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Mikael F. Onsjö; Takayuki Osogami
- Web page: https://patentgrep.com/patent/US20150178635A1

## CPC

- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06Q 40/00: Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

A method and system for deciding an optimal action in consideration of risk. The method includes the steps of: generating sequentially, by way of a Markov decision process based on a Monte Carlo method, a series of data objects having states on a memory of a computer; computing a risk measure of a data object by tracking generated data from opposite order to generation order, where the risk measure is calculated from a value at risk or an exceedance probability that is derived from risk measures of a plurality of states transitionable from a state of the data object; and executing the step of computing the risk measure while tracking back to starting data, where at least one of the steps is carried out using a computer device.

## Claims

### Claim 1

A method for computing an iterated risk measure, the method comprising the steps of:
generating sequentially, by way of a Markov decision process based on a Monte Carlo method, a series of data objects having states on a memory of a computer;
computing a risk measure of a data object by tracking generated data from opposite order to generation order, wherein said risk measure is calculated from a value at risk or an exceedance probability that is derived from risk measures of a plurality of states transitionable from a state of said data object, said computing a risk measure comprises computing:
Pr

(
X
>
x
)
=
∑
i

:


v
i
>
x


p
i
;
wherein:
Pr is a probability;
X is a random variable;
vi (i=1, . . . , n) is a value of each of said plurality of states transitionable from said state of a present object; and
pi (i=1, . . . , n) is a transition probability of each of said plurality of states transitionable from said state of said present object; and
executing said step of computing said risk measure while tracking back to starting data,
wherein at least one of the steps is carried out using a computer device.

### Claim 2 (depends on 1)

A computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions which when implemented, cause a computer to carry out the steps of claim 1.

### Claim 3

A method for computing an action that minimizes an iterated risk measure, the method comprising the steps of:
generating, during postdecision, data comprising combinations of a predetermined state and a possible action on a memory of the computer;
selecting a state-action combination data from generated data of said combinations of said state and said action, based on a value associated with each of said combinations;
generating, during predecision, a state from selected state-action combination data, by way of a Markov decision process based on a Monte Carlo method;
generating a state data sequence by iterating said step of generating a state and said step of generating data comprising combinations;
computing, based on risk measures of a plurality of states transitionable from a present predecision state, a risk measure of an immediately preceding postdecision state by tracking generated states in opposite order to order of the generation, wherein said risk measure is calculated from a value at risk or an exceedance probability, the computing of a risk measure comprises computing:
Pr

(
X
>
x
)
=
∑
i

:


v
i
>
x


p
i
wherein:
Pr is a probability;
X is a random variable;
vi (i=1, . . . , n) is a value of each of said plurality of states transitionable from said state of a present object; and
pi (i=1, . . . , n) is a transition probability of each of said plurality of states transitionable from said state of said present object; and
setting a value of a state having a minimum value in a present postdecision state to an immediately preceding predecision state, by tracking said generated states in the opposite order to the order of the generation,
wherein at least one of the steps is carried out using a computer device.

### Claim 4 (depends on 3)

The method according to claim 3, wherein said step of selecting a state-action combination data uses an evaluation function which is a monotonically decreasing function with respect to a frequency of visiting said state.

### Claim 5

A system for computing an iterated risk measure, the system comprising:
a generating module for generating sequentially, by way of a Markov decision process based on a Monte Carlo method, a series of data objects having states on a memory of a computer;
a risk measure module for computing a risk measure of a data object by tracking generated data objects from opposite order to generation order, wherein said risk measure is calculated from a value at risk or an exceedance probability that is derived from risk measures of a plurality of states transitionable from a state of said data object, said computing a risk measure comprises computing:
Pr

(
X
>
x
)
=
∑
i

:


v
i
>
x


p
i
;
wherein:
Pr is a probability;
X is a random variable;
vi (i=1, . . . , n) is a value of each of said plurality of states transitionable from said state of a present object; and
pi (i=1, . . . , n) is a transition probability of each of said plurality of states transitionable from said state of said present object; and
an executing module for executing said risk measure module while tracking back to starting object.

### Claim 6 (depends on 5)

A computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions which when implemented, cause a computer to carry out the steps of claim 5.

### Claim 7

A system for computing an action that minimizes an iterated risk measure, the system comprising:
a postdecision module for generating, during postdecision, data comprising combinations of a predetermined state and a possible action on a memory of the computer;
a selecting module for selecting a state-action combination data from generated data of said combinations of said state and said action, based on a value associated with each of said combinations;
a predecision module for generating, during predecision, a state from selected state-action combination data, by way of a Markov decision process based on a Monte Carlo method;
a state data sequence module for generating a state data sequence by iterating said step of generating a state and said step of generating data comprising combinations;
a risk measure module for computing, based on risk measures of a plurality of states transitionable from a present predecision state, a risk measure of an immediately preceding postdecision state by tracking generated states in opposite order to order of the generation, wherein said risk measure is calculated from a value at risk or an exceedance probability, the computing of a risk measure comprises computing:
Pr

(
X
>
x
)
=
∑
i

:


v
i
>
x


p
i
wherein:
Pr is a probability;
X is a random variable;
vi (i=1, . . . , n) is a value of each of said plurality of states transitionable from said state of a present object; and
pi (i=1, . . . , n) is a transition probability of each of said plurality of states transitionable from said state of said present object; and
a value module for setting a value of a state having a minimum value in a present postdecision state to an immediately preceding predecision state, by tracking said generated states in the opposite order to the order of the generation.

### Claim 8 (depends on 7)

The system according to claim 7, wherein said selecting module uses an evaluation function which is a monotonically decreasing function with respect to a frequency of visiting said state.
