# US 2025/0158688 A1 · Multi-modal beam prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0158688 A1
- Document ID: US20250158688A1
- Status: Published application
- Kind: A1
- Application number: 18/505,966
- Filed: 2023-11-09
- Published: 2025-05-15
- Earliest claimed: 2023-11-09
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Maximilian Wolfgang Martin ARNOLD; Bence MAJOR; Arash BEHBOODI; Hanno ACKERMANN; Fabio Valerio MASSOLI; Joseph Binamira SORIAGA; Fatih Murat PORIKLI
- Web page: https://patentgrep.com/patent/US20250158688A1

## CPC

- H04B 7/0695: {using beam selection}
- H04B 7/06952: {Selecting one or more beams from a plurality of beams, e.g. beam training, management or sweeping}
- G06N 3/02: Neural networks
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N 20/20: Ensemble learning
- H04B 7/0626: {Channel coefficients, e.g. channel state information [CSI]}
- H04B: TRANSMISSION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A processor-implemented method for multimodal beam management implemented by a network device includes receiving, by the network device, a stream of inputs from one or more sensors. The network device generates a digital twin modeling an environment of a region observed by the one or more sensors. The digital twin includes one or more objects detected based on the stream of inputs. The network device manages a wireless communication signal beam for communicating with at least one user equipment (UE) in the region observed by the one or more sensors based at least in part on the digital twin.

## Claims

### Claim 1

An apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory, the at least one processor configured to:
receive, by a network device, a stream of inputs from one or more sensors;
generate, by the network device, a digital twin modeling an environment of a region observed by the one or more sensors, the digital twin including one or more objects detected based on the stream of inputs; and
manage, by the network device, a wireless communication signal beam for communicating with at least one user equipment (UE) in the region observed by the one or more sensors based at least in part on the digital twin.

### Claim 2 (depends on 1)

The apparatus of claim 1, in which the at least one processor is further configured to generate, by the network device, a predicted beam for conducting wireless communication with the at least one UE using the digital twin based on the stream of inputs from the one or more sensors.

### Claim 3 (depends on 1)

The apparatus of claim 1, in which the at least one processor is further configured to:
generate, by the network device, a second predicted beam for the at least one UE to conduct uplink communications with the network device; and
transmit, by the network device, an indication of the second predicted beam to the at least one UE.

### Claim 4 (depends on 1)

The apparatus of claim 1, in which the one or more sensors includes one or more of a camera, an inertial measurement unit (IMU) sensor, a light detection and ranging (LiDAR) sensor, and a global positioning satellite (GPS) sensor.

### Claim 5 (depends on 1)

The apparatus of claim 1, in which the network device manages the wireless communication signal beam based on a fusion of information from the stream of inputs.

### Claim 6 (depends on 1)

The apparatus of claim 1, in which the at least one processor is further configured to:
track, by the network device, a location of an object of the one or more objects detected based on the stream of inputs, based on an object type;
update, by the network device, the digital twin based on the tracking; and
adapt, by the network device, the wireless communication signal beam based on the updated digital twin.

### Claim 7 (depends on 1)

The apparatus of claim 1, in which the at least one processor is further configured to predict, by the network device, a blocker of a wireless communication signal to a first UE of the at least one UE based on the digital twin and a first location of an object relative to a second location of an antenna of the first UE.

### Claim 8 (depends on 7)

The apparatus of claim 7, in which one or more of the first UE or the object are moving.

### Claim 9 (depends on 7)

The apparatus of claim 7, in which the at least one processor is further configured to:
determine, by the network device, link degradation due to the object based on a degree of overlap of the first location and the second location; and
adapt, by the network device, wireless communication with the at least one UE based on a link degradation type, the wireless communication being adapted by one of: adapting the wireless communication signal beam, changing a frequency for the wireless communication, or initiating a handover.

### Claim 10 (depends on 1)

The apparatus of claim 1, in which the at least one processor is further configured to:
determine a channel state associated with the wireless communication signal beam; and
supply the channel state as feedback to the digital twin.

### Claim 11

An apparatus, comprising:
at least one memory; and
at least one processor coupled to the at least one memory, the at least one processor configured to:
initiate, by a user equipment (UE), wireless communication with a network device; and
receive, by the UE, a wireless communication signal beam from the network device, the wireless communication signal beam being managed based on a digital twin and a stream of inputs from one or more sensors observing the UE and one or more objects in an environment modeled in the digital twin.

### Claim 12 (depends on 11)

The apparatus of claim 11, in which the wireless communication signal beam is predicted based on the stream of inputs from the one or more sensors.

### Claim 13 (depends on 11)

The apparatus of claim 11, in which the at least one processor is further configured to:
receive, from the network device, an indication of a predicted beam for conducting uplink communication with the network device, the predicted beam being determined using the digital twin based on the stream of inputs from the one or more sensors; and
generate, by the UE, a wireless communication signal based on the predicted beam to communicate with the network device.

### Claim 14 (depends on 11)

The apparatus of claim 11, in which the one or more sensors includes one or more of a camera, an inertial measurement unit (IMU) sensor, a light detection and ranging (LiDAR) sensor, and a global positioning satellite (GPS) sensor.

### Claim 15 (depends on 11)

The apparatus of claim 11, in which the wireless communication signal beam is managed based on a fusion of information from the stream of inputs.

### Claim 16 (depends on 11)

The apparatus of claim 11, in which the wireless communication signal beam is selected from a set of multiple candidate beams determined by the digital twin.

### Claim 17

A processor-implemented method, implemented by a network device, the processor-implemented method comprising:
receiving, by the network device, a stream of inputs from one or more sensors;
generating, by the network device, a digital twin modeling an environment of a region observed by the one or more sensors, the digital twin including one or more objects detected based on the stream of inputs; and
managing, by the network device, a wireless communication signal beam for communicating with at least one user equipment (UE) in the region observed by the one or more sensors based at least in part on the digital twin.

### Claim 18 (depends on 17)

The processor-implemented method of claim 17, further comprising:
generating, by the network device, a predicted beam for conducting wireless communication with the at least one UE using the digital twin based on the stream of inputs from the one or more sensors.

### Claim 19 (depends on 17)

The processor-implemented method of claim 17, further comprising:
generating, by the network device, a second predicted beam for the at least one UE to conduct uplink communications with the network device; and
transmitting, by the network device, an indication of the second predicted beam to the at least one UE.

### Claim 20 (depends on 17)

The processor-implemented method of claim 17, in which the network device manages the wireless communication signal beam based on a fusion of information from the stream of inputs.

### Claim 21 (depends on 17)

The processor-implemented method of claim 17, further comprising:
tracking, by the network device, a location of an object of the one or more objects detected based on the stream of inputs, based on an object type;
updating, by the network device, the digital twin based on the tracking; and
adapting, by the network device, the wireless communication signal beam based on the updated digital twin.

### Claim 22 (depends on 17)

The processor-implemented method of claim 17, further comprising predicting, by the network device, a blocker of a wireless communication signal to a first UE of the at least one UE based on the digital twin and a first location of an object relative to a second location of an antenna of the first UE.

### Claim 23 (depends on 22)

The processor-implemented method of claim 22, in which one or more of the first UE or the object are moving.

### Claim 24 (depends on 22)

The processor-implemented method of claim 22, further comprising:
determining, by the network device, link degradation due to the object based on a degree of overlap of the first location and the second location; and
adapting, by the network device, wireless communication with the at least one UE based on a link degradation type, the wireless communication being adapted by one of adapting the wireless communication signal beam, changing a frequency for the wireless communication, or initiating a handover.

### Claim 25 (depends on 17)

The processor-implemented method of claim 17, further comprising:
determining a channel state associated with the wireless communication signal beam; and
supplying the channel state as feedback to the digital twin.

### Claim 26

A processor-implemented method, implemented by a user equipment (UE), the processor-implemented method comprising:
initiating, by a user equipment (UE), wireless communication with a network device; and
receiving, by the UE, a wireless communication signal beam from the network device, the wireless communication signal beam being managed based on a digital twin and a stream of inputs from one or more sensors observing the UE and one or more objects in an environment modeled in the digital twin.

### Claim 27 (depends on 26)

The processor-implemented method of claim 26, in which the wireless communication signal beam is predicted based on the stream of inputs from the one or more sensors.

### Claim 28 (depends on 26)

The processor-implemented method of claim 26, further comprising:
receiving, from the network device, an indication of a predicted beam for conducting uplink communication with the network device, the predicted beam being determined using the digital twin based on the stream of inputs from the one or more sensors; and
generating, by the UE, a wireless communication signal based on the predicted beam to communicate with the network device.

### Claim 29 (depends on 26)

The processor-implemented method of claim 26, in which the wireless communication signal beam is managed based on a fusion of information from the stream of inputs.

### Claim 30 (depends on 26)

The processor-implemented method of claim 26, in which the wireless communication signal beam is selected from a set of multiple candidate beams determined by the digital twin.
