# US 12,603,972 B2 · Wireless transmitter identification in visual scenes > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,603,972 B2 - Document ID: US12603972B2 - Status: Granted - Kind: B2 - Application number: 17/863,567 - Filed: 2022-07-13 - Published: 2023-02-02 - Granted: 2026-04-14 - Earliest claimed: 2021-07-13 - Assignees: Arizona Board of Regents on behalf of Arizona State University - Applicants: Arizona Board of Regents On Behalf of Arizona State University - Inventors: Ahmed Alkhateeb; Muhammad Alrabeiah; Gouranga Charan - Web page: https://patentgrep.com/patent/US12603972B2 ## CPC - H04N 7/181: {for receiving images from a plurality of remote sources} - G06N 3/0499: Feedforward networks - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods} - H04W 64/003: {locating network equipment} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract Wireless transmitter identification in visual scenes is provided. This technology enables important wireless communications and sensing applications such as (i) fast beam/blockage prediction in fifth generation (5G)/sixth generation (6G) systems using camera data, (ii) identifying cars and people in a surveillance camera feed using joint visual and wireless data processing, and (iii) enabling efficient autonomous vehicle communication relying on both the camera and wireless data. This is done by developing multimodal machine learning based frameworks that use the sensory data obtained by visual and wireless sensors. More specifically, given some visual data, an algorithm needs to perform the following: (i) predict whether an object responsible for a received radio signal is present or not, (ii) if it is present, detect which object it is out of the candidate transmitters, and (iii) predict what type of signal the detected object is transmitting. ## Claims ### Claim 1 A method for providing wireless transmitter identification, the method comprising: receiving a signal from a transmitter; receiving image data of an environment; extracting vision features from the image data; extracting wireless features from the signal; and analyzing, using cross-modal analysis, the vision features and the wireless features to identify the transmitter in the environment and a type of the signal received from the transmitter, wherein the cross-modal analysis learns a new set of cross-modality features, and uses the new set of cross-modality features to predict whether there are transmitters in the environment, and predict which objects represent the transmitters. ### Claim 2 (depends on 1) The method of claim 1, further comprising adapting wireless communications with a wireless device associated with the transmitter based on the identification of the transmitter. ### Claim 3 (depends on 2) The method of claim 2, wherein adapting the wireless communications with the wireless device comprises adjusting an allocation of network resources based on the analyzed image data. ### Claim 4 (depends on 2) The method of claim 2, further comprising: receiving additional environmental data of the environment; and adapting the wireless communications with the wireless device further based on the additional environmental data. ### Claim 5 (depends on 4) The method of claim 4, wherein the additional environmental data comprises at least one of global positioning system (GPS) data received from the wireless device, light detection and ranging (LIDAR) data received from a LIDAR device, radar data received from a radar system, or a wireless signal received at a transceiver. ### Claim 6 (depends on 1) The method of claim 1, further comprising locating the transmitter in the environment. ### Claim 7 (depends on 1) The method of claim 1, further comprising processing the image data to track a location of a wireless device associated with the transmitter in the environment. ### Claim 8 (depends on 7) The method of claim 7, further comprising adapting wireless communications with the wireless device by beam selecting one or more beamforming vectors for the wireless communications based on the tracked location of the wireless device in the environment. ### Claim 9 (depends on 1) The method of claim 1, wherein receiving the image data comprises receiving image data from a plurality of imaging devices distributed in the environment. ### Claim 10 (depends on 1) The method of claim 1, wherein extracting the vision features comprises: detecting objects from the image data; and limiting overlapping bounding boxes for the detected objects. ### Claim 11 (depends on 1) The method of claim 1, wherein the wireless features comprise: wireless data; channels; beams; and received power. ### Claim 12 A network node, comprising: communication circuitry configured to establish communications with a wireless device in an environment; and a processing system configured to: receive image data of the environment; receive a signal from the wireless device; extract vision features from the image data; extract wireless features from the signal; perform an analysis, using cross-modal analysis, of the vision features and the wireless features to identify the wireless device and a type of the signal received from the transmitter; and adapt the communications with the wireless device in accordance with the analysis, wherein the cross-modal analysis learns a new set of cross-modality features, and uses the new set of cross-modality features to predict whether there are transmitters in the environment, and predict which objects represent the transmitters. ### Claim 13 (depends on 12) The network node of claim 12, wherein the communication circuitry comprises a multi-band radio transceiver. ### Claim 14 (depends on 13) The network node of claim 13, wherein the multi-band radio transceiver is configured to communicate via at least one sub-6 gigahertz (GHz) band and one millimeter wave (mmWave) band. ### Claim 15 (depends on 13) The network node of claim 13, wherein the multi-band radio transceiver is configured to communicate via at least one terahertz (THz) band. ### Claim 16 (depends on 12) The network node of claim 12, wherein the processing system is configured to perform the analysis of the environment using a machine learning framework. ### Claim 17 (depends on 12) The network node of claim 12, wherein the image data is received from the wireless device. ### Claim 18 (depends on 12) The network node of claim 12, further comprising a red-green-blue (RGB) camera configured to capture the image data. ### Claim 19 (depends on 12) The network node of claim 12, wherein the processing system is configured to: detect objects from the image data; and limit overlapping bounding boxes for the detected objects. ### Claim 20 A neural network for wireless transmitter identification, the neural network comprising: a vision feature extractor; a wireless feature extractor; and a bounding box selection engine configured to provide transmitter identification and identification of a type of the signal received from the transmitter from a cross-modal analysis of vision features from the vision feature extractor and wireless features from the wireless feature extractor, wherein the cross-modal analysis learns a new set of cross-modality features, and uses the new set of cross-modality features to predict whether there are transmitters in the environment, and predict which objects represent the transmitters.