# US 2022/0046447 A1 · Machine learning based antenna panel switching > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0046447 A1 - Document ID: US20220046447A1 - Status: Published application - Kind: A1 - Application number: 17/392,877 - Filed: 2021-08-03 - Published: 2022-02-10 - Earliest claimed: 2020-08-04 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Klaus Ingemann PEDERSEN; István Zsolt KOVÁCS; Ali ESSWIE - Web page: https://patentgrep.com/patent/US20220046447A1 ## CPC - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - H04W 24/08: Testing, {supervising or monitoring} using real traffic - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/092: Reinforcement learning - G06N 20/00: Machine learning - H04B 7/0691: {using subgroups of transmit antennas} - H04B 17/336: Signal-to-interference ratio [SIR] or carrier-to-interference ratio [CIR] - G06N 3/045: Combinations of networks - H04B 7/0608: {Antenna selection according to transmission parameters} - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - H04B: TRANSMISSION ## Abstract Disclosed is a method comprising using a machine learning algorithm to select an antenna panel from a plurality of antenna panels. A first long-term reward value associated with the selected antenna panel is determined based at least partly on one or more first signals received on the selected antenna panel. A second signal is then transmitted or received via the selected antenna panel, if the first long-term reward value exceeds one or more second long-term reward values associated with at least a subset of the plurality of antenna panels. ## Claims ### Claim 1 An apparatus, comprising: at least one processor; and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to: use a machine learning algorithm to select an antenna panel from a plurality of antenna panels; determine a first long-term reward value associated with the selected antenna panel based at least partly on one or more first signals received on the selected antenna panel, wherein the first long-term reward value is an average or a sum of consecutive immediate reward values over a first time window, wherein the consecutive immediate reward values are based at least partly on a plurality of key performance indicators comprising at least a serving cell quality metric and a beam failure rate; transmit or receive a second signal via the selected antenna panel, if the first long-term reward value exceeds one or more second long-term reward values associated with at least a subset of the plurality of antenna panels. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein the machine learning algorithm is based on reinforcement learning. ### Claim 3 (depends on 1) The apparatus according to claim 1, wherein the machine learning algorithm is based on a state space comprising at least two dimensions, wherein a first dimension indicates the serving cell quality metric, and a second dimension indicates a neighbour cell quality metric. ### Claim 4 (depends on 1) The apparatus according to claim 1, wherein the antenna panel is selected by using a policy to select an action corresponding with the selected antenna panel based at least partly on a current state and a reward function, wherein the reward function is based on a plurality of weights. ### Claim 5 (depends on 4) The apparatus according to claim 4, wherein the plurality of weights are scalar values indicating a priority for maximizing the serving cell quality metric, minimizing the beam failure rate, minimizing a radio link failure rate and/or minimizing a handover failure rate. ### Claim 6 (depends on 4) The apparatus according to claim 4, wherein the apparatus is further caused to obtain the plurality of weights from a base station. ### Claim 7 (depends on 4) The apparatus according to claim 4, wherein the apparatus is further caused to: measure a quality metric associated with the one or more first signals received on the selected antenna panel; update the state, the reward function and the policy based at least partly on the measured quality metric. ### Claim 8 (depends on 7) The apparatus according to claim 7, wherein the policy is updated by using Q-learning to update a Q-value for one or more state-action pairs. ### Claim 9 (depends on 7) The apparatus according to claim 7, wherein the apparatus is further caused to: transmit the measured quality metric to the base station; receive an updated plurality of weights from the base station. ### Claim 10 (depends on 7) The apparatus according to claim 7, wherein the quality metric comprises a reference signal received power value or a signal-to-interference-plus-noise ratio value. ### Claim 11 (depends on 1) The apparatus according to claim 1, wherein the one or more second long-term reward values are obtained from a database. ### Claim 12 (depends on 1) The apparatus according to claim 1, wherein the one or more first signals comprise at least a serving cell signal and a neighbour cell signal. ### Claim 13 A system, comprising: a terminal device; and a base station; wherein the terminal device is configured to: use a machine learning algorithm to select an antenna panel from a plurality of antenna panels; receive one or more first signals from the base station via the selected antenna panel; determine a first long-term reward value associated with the selected antenna panel based at least partly on the one or more first signals received on the selected antenna panel, wherein the first long-term reward value is an average or a sum of consecutive immediate reward values over a first time window, wherein the consecutive immediate reward values are based at least partly on a plurality of key performance indicators comprising at least a serving cell quality metric and a beam failure rate; transmit or receive a second signal via the selected antenna panel, if the first long-term reward value exceeds one or more second long-term reward values associated with at least a subset of the plurality of antenna panels; and wherein the base station is configured to: transmit the one or more first signals to the terminal device; receive or transmit the second signal from or to the terminal device. ### Claim 14 A method, comprising: using a machine learning algorithm to select an antenna panel from a plurality of antenna panels; determining a first long-term reward value associated with the selected antenna panel based at least partly on one or more first signals received on the selected antenna panel, wherein the first long-term reward value is an average or a sum of consecutive immediate reward values over a first time window, wherein the consecutive immediate reward values are based at least partly on a plurality of key performance indicators comprising at least a serving cell quality metric and a beam failure rate; transmitting or receiving a second signal via the selected antenna panel, if the first long-term reward value exceeds one or more second long-term reward values associated with at least a subset of the plurality of antenna panels. ### Claim 15 (depends on 14) The method according to claim 14, wherein the machine learning algorithm is based on reinforcement learning. ### Claim 16 (depends on 14) The method according to claim 14, wherein the machine learning algorithm is based on a state space comprising at least two dimensions, wherein a first dimension indicates the serving cell quality metric, and a second dimension indicates a neighbor cell quality metric. ### Claim 17 (depends on 14) The method according to claim 14, wherein the antenna panel is selected by using a policy to select an action corresponding with the selected antenna panel based at least partly on a current state and a reward function, wherein the reward function is based on a plurality of weights. ### Claim 18 (depends on 17) The method according to claim 17, wherein the plurality of weights are scalar values indicating a priority for maximizing the serving cell quality metric, minimizing the beam failure rate, minimizing a radio link failure rate or minimizing a handover failure rate. ### Claim 19 (depends on 17) The method according to claim 17, wherein the plurality of weights are obtained from a base station. ### Claim 20 (depends on 17) The method according to claim 17, further comprising: measuring a quality metric associated with the one or more first signals received on the selected antenna panel; and updating the state, the reward function and the policy based at least partly on the measured quality metric.