US 10,375,585 B2Granted

System and method for deep learning and wireless network optimization using deep learning

AssigneeFuturwei Technologies, Inc.

InventorsYongxi Tan, Jin Yang, Qitao Song, Yunjun Chen, Zhangxiang Ye

Life of this patent

Estimating the term…

Loading patent…

Abstract

A neural network is trained using deep reinforcement learning (DRL) techniques for adjusting cell parameters of a wireless network by generating a plurality of experience tuples, and updating the neural network based on the generated experience tuples. The trained neural network may be used to select actions to adjust the cell parameters. Each experience tuple includes a cell identifier, a first state, a second state, an action applied to the cell that moves the cell from the first state to the second state, a local reward, and a global reward. The neural network is updated based on whether or not each action is acceptable, which is determined based on the global reward and the local reward associated with each action.

Claims32

Dependent claims

A method comprising:

initializing a neural network with a set of weight values, the neural network being used to determine actions that adjust one or more settings of cells associated with base stations in a wireless network, each base station providing communication services to user equipments (UEs) within one or more cells;
and
training the neural network by using a deep reinforcement learning (DRL) process, the DRL processing comprising:
generating a first plurality of experience tuples for a plurality of cells in the wireless network, each experience tuple comprising a cell identifier that identifies a cell, a first state of the cell, a second state of the cell, an action that causes the cell to transit from the first state to the second state, and a reward value for taking the action, wherein a state of a cell comprises a setting of a base station providing a coverage area of the cell, and a reward value is calculated using a cost function based on measurement reports received from UEs in the wireless network, wherein each experience tuple can be a DRL-generated experience tuple in which a respective action is selected by a DRL agent based on the neural network according to a DRL technique or an expert-generated experience tuple in which the respective action is provided based on expert experience, and wherein whether an action is selected by the DRL agent based on the neural network or provided based on the expert experience is determined based on a first criterion;
and
selecting a second plurality of experience tuples from the first plurality of experience tuples;
and
updating the set of weight values of the neural network according to reward values in the second plurality of experience tuples.

5 more elements

A method comprising:

training a neural network for a plurality of cells in a wireless network using a deep reinforcement learning (DRL) process for adjusting one or more cell parameters of cells associated with base stations in the wireless network, each base station providing communication services to user equipments (UEs) within a coverage area of one or more cells, and the neural network being trained to determine actions that can be performed on the base stations, wherein the DRL process comprises:
generating a plurality of experience tuples for a plurality of cells in the wireless network, comprising:
generating a state tensor for each of the plurality of cells, each state tensor indicating a state of a respective cell, wherein a state of a cell comprises a setting of a base station associated with the cell, the base station providing a coverage area of the cell;
selecting an action for each of the plurality of cells, the action moving the respective cell from one state to another state, wherein an action comprises information for adjusting a setting of a base station associated with a cell;
applying respective actions selected for the plurality of cells to the respective cells to adjust one or more cell parameters;
and
generating an experience tuple for each of the plurality of cells based on the respective action applied, the experience tuple comprising a cell identifier identifying the respective cell, a first state of the respective cell that is indicated by a respective state tensor, a second state of the respective cell, the action applied to the respective cell that moves the respective cell from the first state to the second state, a local reward calculated for applying the action to the respective cell, and a global reward calculated for applying the action to the respective cell, the local reward being calculated based on a local cost function and the global reward being calculated based on a global cost function;
and
determining whether or not an action of a first experience tuple in the plurality of experience tuples corresponding to a first cell is acceptable based on the local reward and the global reward of the first experience tuple;
and
updating weights of the neural network based on whether or not the action is acceptable;
selecting an action for adjusting a cell parameter of a cell in the wireless network based on the trained neural network;
and
instructing to adjust the cell parameter of the cell in the wireless network according to the selected action.

11 more elements

An apparatus comprising:

a non-transitory memory storage comprising instructions;
and
one or more processors in communication with the memory storage, wherein the one or more processors execute the instructions to:
train a neural network for a plurality of cells in a wireless network using a deep reinforcement learning (DRL) process for adjusting one or more cell parameters of cells associated with base stations in the wireless network, each base station providing communication services to user equipments (UEs) within a coverage area of one or more cells, and the neural network being trained to determine actions that can be performed on the base stations, wherein the DRL process comprises:
generating a plurality of experience tuples for a plurality of cells in the wireless network, comprising:
generating a state tensor for each of the plurality of cells, each state tensor indicating a state of the respective cell, wherein a state of a cell comprises a setting of a base station associated with the cell, the base station providing a coverage area of the cell;
selecting an action for each of the plurality of cells, the action moving the respective cell from one state to another state, wherein an action comprises information for adjusting a setting of a base station associated with the cell;
applying respective actions selected for the plurality of cells to the respective cells to adjust one or more cell parameters;
and
generating an experience tuple for each of the plurality of cells based on the respective action applied, the experience tuple comprising a cell identifier identifying the respective cell, a first state of the respective cell that is indicated by a respective state tensor, a second state of the respective cell, the action applied to the respective cell that moves the respective cell from the first state to the second state, a local reward calculated for applying the action to the respective cell, and a global reward calculated for applying the action to the respective cell, the local reward being calculated based on a local cost function and the global reward being calculated based on a global cost function;
and
determining whether or not an action of a first experience tuple in the plurality of experience tuples corresponding to a first cell is acceptable based on the local reward and the global reward of the first experience tuple;
and
updating weights of the neural network based on whether or not the action is acceptable;
and
select an action for adjusting a cell parameter of a cell in the wireless network based on the trained neural network;
and
instruct to adjust the cell parameter of the cell in the wireless network according to the selected action.

15 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/643,266Filed 6 Jul 2017
Published: US 2019/0014488 A110 Jan 2019
Granted: 6 Aug 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification10 codes

H04W 24/02
Supervisory, monitoring or testing arrangements: arrangements for optimising operational condition
G06N 3/044
Architecture, e.g. interconnection topology: recurrent networks, e.g. Hopfield networks
G06N 3/045
Architecture, e.g. interconnection topology: combinations of networks
G06N 3/0464
Architecture, e.g. interconnection topology: convolutional networks [CNN, ConvNet]

Family

US 2019/0014488 A1Pre-grant publication · published 10 Jan 2019
US 10,375,585 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key