# US 2023/0054692 A1 · Reinforcement machine learning framework for dynamic demand forecasting

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0054692 A1
- Document ID: US20230054692A1
- Status: Published application
- Kind: A1
- Application number: 17/740,770
- Filed: 2022-05-10
- Published: 2023-02-23
- Earliest claimed: 2021-08-20
- Assignees: Amadeus S.A.S.
- Applicants: Amadeus S.A.S.
- Inventors: Michael WITTMAN; Thomas FIIG; Riccardo JADANZA; Giovanni GATTI PINHEIRO; Michael DEFOIN PLATEL
- Web page: https://patentgrep.com/patent/US20230054692A1

## CPC

- G06N 20/00: Machine learning
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06Q 10/02: Reservations, e.g. for tickets, services or events
- G06Q 30/0206: {Price or cost determination based on market factors}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Systems and methods for implementing a reinforcement machine learning framework for dynamic demand forecasting. A method includes generating estimated booking data for an initial time with a demand model trained using a training set of historical booking data. A variance is detected between the estimated booking data and transient booking data observed at the initial time that exceeds a defined threshold. In response to detecting the variance, a reinforcement learning service is activated. An updated training set including enhanced booking data observed at a subsequent time is created after activating the reinforcement learning service. A parameter of the demand model is updated by training the demand model using the updated training set.

## Claims

### Claim 1

A method comprising:
generating estimated booking data for an initial time with a demand model trained using a training set of historical booking data;
detecting a variance between the estimated booking data and transient booking data observed at the initial time that exceeds a defined threshold;
activating a reinforcement learning service in response to detecting the variance;
creating an updated training set comprising enhanced booking data observed at a subsequent time after activating the reinforcement learning service; and
updating a parameter of the demand model by training the demand model using the updated training set.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the estimated booking data comprises:
computing a probability that a travel service or a flight will receive a given number of bookings by a given day-to-departure (“DTD”) based on a demand forecast obtained using the demand model.

### Claim 3 (depends on 1)

The method of claim 1, wherein the reinforcement learning service is configured to increase price disparity within an updated training set at a subsequent time based on increasing a diversity of observations of price-demand pairs.

### Claim 4 (depends on 1)

The method of claim 1, wherein the reinforcement learning service is configured to compute an optimal probability of transitioning to an exploration mode to maximize a net value of a gain of information minus a cost of exploration, wherein the gain of information comprises an expected improvement in demand model accuracy as a result of increased price diversity in an updated training set based on the exploration mode, and the cost of exploration comprises a difference in expected revenue between offering an exploratory fare class selected by the exploration mode and offering the fare class that would have been selected by an exploitation mode.

### Claim 5 (depends on 1)

The method of claim 1, wherein the updated training set is created using an offer threshold process configured to effectuate an exploration mode by modifying offer frequencies of multiple fare classes based a respective number of offers for each fare class identified within the historical booking data.

### Claim 6 (depends on 5)

The method of claim 5, wherein modifying the offer frequencies of the multiple fare classes comprises:
reducing an offer frequency of a given fare class with a highest number of offers among the multiple fare classes.

### Claim 7 (depends on 5)

The method of claim 5, wherein modifying the offer frequencies of the multiple fare classes comprises:
increasing an offer frequency of a given fare class with less than a pre-defined minimum number of offers.

### Claim 8 (depends on 1)

The method of claim 1, wherein activating the reinforcement learning service comprises:
selecting, by an optimization service, one or more inventory control attributes; and
intercepting and processing, by the reinforcement learning service, the one or more inventory control attributes based on an exploration-exploitation strategy.

### Claim 9 (depends on 8)

The method of claim 8, wherein the exploration-exploitation strategy comprises an operating mode of transitioning between an exploration mode and an exploitation mode.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
generating a confidence cone by aggregating a set of probabilities computed for multiple flights with each probability estimating a likelihood that a given flight among the multiple flights will receive a given number of bookings by a given day-to-departure (“DTD”) based on a demand forecast obtained using the demand model.

### Claim 11 (depends on 1)

The method of claim 1, wherein the updated training set contains an increased diversity of observations of price-demand pairs as a result of the activation of an exploration mode of the reinforcement learning service.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
updating the training set by:
computing an entropy of the training set to determine an uncertainty of the demand model;
identifying fare classes to explore to minimize the entropy in the updated training set;
estimating a cost of exploring at each fare class, comprised of a difference in expected revenue between offering an exploratory fare class and offering the fare class that would have been selected by an exploitation mode; and
selecting a fare class to explore that maximizes an expected improvement in demand model accuracy as a result of increased price diversity in the updated training set based on a maximum exploration cost threshold.

### Claim 13 (depends on 1)

The method of claim 1, wherein the parameter comprises a price elasticity parameter, a product attractiveness parameter, a demand volume parameter, or a temporal adjustment parameter.

### Claim 14

A system comprising:
one or more processors;
at least one memory device coupled with the one or more processors; and
a data communications interface operably associated with the one or more processors, wherein the at least one memory device contains a plurality of program instructions that, when executed by the one or more processors, cause the system to:
generate estimated booking data for an initial time with a demand model trained using a training set of historical booking data;
detect a variance between the estimated booking data and transient booking data observed at the initial time that exceeds a defined threshold;
activate a reinforcement learning service in response to detecting the variance;
create an updated training set comprising enhanced booking data observed at a subsequent time after activating the reinforcement learning service; and
update a parameter of the demand model by training the demand model using the updated training set.

### Claim 15 (depends on 14)

The system of claim 14, wherein the plurality of program instructions cause the system to generate the estimated booking data by causing the system to compute a probability that a travel service or flight will receive a given number of bookings by a given day-to-departure (“DTD”) based on a demand forecast obtained using the demand model.

### Claim 16 (depends on 14)

The system of claim 14, wherein the reinforcement learning service is configured to increase price disparity within an updated training set based on increasing a diversity of observations of price-demand pairs.

### Claim 17 (depends on 14)

The system of claim 14, wherein the reinforcement learning service is configured to compute an optimal probability of transitioning to an exploration mode to maximize a net value of a gain of information minus a cost of exploration.

### Claim 18 (depends on 14)

The system of claim 14, wherein the plurality of program instructions cause the system to create the updated training set with an offer threshold process that is configured to effectuate an exploration mode by modifying offer frequencies of multiple fare classes based a respective number of offers for each fare class identified within the historical booking data.

### Claim 19 (depends on 14)

The system of claim 14, wherein the program instructions, when executed by the one or more processors, further cause the system to update the training set by causing the system to:
compute an entropy of the training set to determine an uncertainty of the demand model;
identify fare classes to explore to minimize the entropy in the updated training set;
estimate a cost of exploring at each fare class, comprised of a difference in expected revenue between offering an exploratory fare class and offering the fare class that would have been selected by an exploitation mode; and
select a fare class to explore that maximizes an expected improvement in demand model accuracy as a result of increased price diversity in the updated training set based on a maximum exploration cost threshold.

### Claim 20

A computer program product comprising:
a non-transitory computer-readable storage medium; and
program code stored on the non-transitory computer-readable storage medium that, when executed by one or more processors, causes the one or more processors to:
generate estimated booking data for an initial time with a demand model trained using a training set of historical booking data;
detect a variance between the estimated booking data and transient booking data observed at the initial time that exceeds a defined threshold;
activate a reinforcement learning service in response to detecting the variance;
create an updated training set comprising enhanced booking data observed at a subsequent time after activating the reinforcement learning service; and
update a parameter of the demand model by training the demand model using the updated training set.
