# US 12,015,937 B2 · Operation of sectorized communications from aerospace platforms using reinforcement learning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,015,937 B2 - Document ID: US12015937B2 - Status: Granted - Kind: B2 - Application number: 18/355,125 - Filed: 2023-07-19 - Published: 2023-11-16 - Granted: 2024-06-18 - Earliest claimed: 2018-12-17 - Assignees: Aalyria Technologies, Inc. - Applicants: Aalyria Technologies, Inc. - Inventors: Sharath Ananth; Brian Barritt; Jin Zhang - Web page: https://patentgrep.com/patent/US12015937B2 ## CPC - H04W 24/02: Arrangements for optimising operational condition - G06N 3/08: Learning methods - G06N 3/092: Reinforcement learning - H04W 84/06: Airborne or Satellite Networks (space-based or airborne stations H04B7/185) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The disclosure provides a method of operating a communication network. The method includes receiving input data related to a state of the communication network and operation of the communication network. The method then includes determining a policy for the communication network based on the input data. The policy is a set of features for forming a plurality of communication links in the communication network over a time interval. The plurality of communication links provides one or more paths through the communication network. Determining the policy is based at least in part on utility values of previous policies. The utility values of previous policies are derived using simulation and/or real-world implementation of the previous policies. The communication network is then operated to implement the policy in the time interval. ## Claims ### Claim 1 A method of operating a communication network that includes a plurality of nodes, the method comprising: receiving, by one or more processors, input data related to a state of the communication network and input data related to operation of the communication network for a first time interval; determining, by the one or more processors, a trend in features of a first policy for the communication network based on the input data; determining, by the one or more processors, a second policy for the communication network for a second time interval based at least in part on the trend in features of the first policy; and operating, by the one or more processors, the communication network to implement the second policy in the second time interval. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining, by the one or more processors, a performance metric associated with the first policy. ### Claim 3 (depends on 1) The method of claim 1, wherein the plurality of nodes includes one or more non-geostationary satellites. ### Claim 4 (depends on 1) The method of claim 1, wherein the input data related to the operation of the communication network includes weather conditions or forecasts for atmospheric levels. ### Claim 5 (depends on 1) The method of claim 1, wherein the one or more processors form a neural network. ### Claim 6 (depends on 5) The method of claim 5, further comprising training the neural network by: receiving, by the one or more processors, training input data related to state information of the communication network; determining, by the one or more processors, a training policy based on the training input data; simulating, by the one or more processors, the training policy based on internal and external influences of the communication network; and determining, by the one or more processors, a utility value of the training policy according to the simulation; and wherein determining the first policy is further based on the training policy and the utility value of the training policy. ### Claim 7 (depends on 1) The method of claim 1, wherein determining the trend in features of the first policy includes: selecting features for the first policy that increase the performance metric of the communication network according to the trend. ### Claim 8 A system for operating a communication network that includes a plurality of nodes, the system comprising one or more processors configured to: receive input data related to a state of the communication network and input data related to operation of the communication network for a first time interval; determine a trend in features of a first policy for the communication network based on the input data; determine a second policy for the communication network for a second time interval based at least in part on the trend in features of the first policy; and operate the communication network to implement the second policy in the second time interval. ### Claim 9 (depends on 8) The method of claim 8, further comprising: determining, by the one or more processors, a performance metric associated with the first policy. ### Claim 10 (depends on 8) The system of claim 8, further comprising one or more nodes of the plurality of nodes. ### Claim 11 (depends on 10) The system of claim 10, wherein the one or more nodes includes a non-geostationary satellite. ### Claim 12 (depends on 8) The system of claim 8, wherein the input data related to the operation of the communication network includes weather conditions or forecasts for atmospheric levels. ### Claim 13 (depends on 8) The system of claim 8, wherein the one or more processors form a neural network. ### Claim 14 (depends on 13) The system of claim 13, wherein the one or more processors are further configured to train the neural network by: receive training input data related to state information of the communication network; determine a training policy based on the training input data; simulate the training policy based on internal and external influences of the communication network; and determine a utility value of the training policy according to the simulation; and wherein the first policy is determined further based on the training policy and the utility value of the training policy. ### Claim 15 (depends on 8) The system of claim 8, wherein the first policy is determined to includes one or more features that increase the performance metric of the communication network. ### Claim 16 A non-transitory, tangible computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising: receiving input data related to a state of a communication network and input data related to operation of the communication network for a first time interval; determining a trend in features of a first policy for the communication network based on the input data; determining a second policy for the communication network for a second time interval based at least in part on the trend in features of the first policy; and operating the communication network to implement the second policy in the second time interval. ### Claim 17 (depends on 16) The storage medium of claim 16, wherein the communication network includes one or more non-geostationary nodes. ### Claim 18 (depends on 16) The storage medium of claim 16, wherein the input data related to the operation of the communication network includes weather conditions or forecasts for atmospheric levels. ### Claim 19 (depends on 18) The storage medium of claim 18, wherein determining the trend in features of the first policy includes: selecting features for the first policy that increase the performance metric of the communication network according to the trend.