# US 2021/0084352 A1 · Automatic generation of augmented reality media

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0084352 A1
- Document ID: US20210084352A1
- Status: Published application
- Kind: A1
- Application number: 16/570,454
- Filed: 2019-09-13
- Published: 2021-03-18
- Earliest claimed: 2019-09-13
- Assignees: AT&T Intellectual Property I, L.P.
- Applicants: AT&T Intellectual Property I, L.P.
- Inventors: David Crawford Gibbon; Zhu Liu; Lee Begeja; Behzad Shahraray; Eric Zavesky
- Web page: https://patentgrep.com/patent/US20210084352A1

## CPC

- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 10/7784: {based on feedback from supervisors}
- G06V 20/42: {of sport video content}
- H04N 5/44504: {Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits}
- H04N 21/2187: Live feed
- H04N 21/222: Secondary servers, e.g. proxy server, cable television Head-end {(provisioning of proxy services in data packet switching networks H04L67/56)}
- H04N 21/251: {Learning process for intelligent management, e.g. learning user preferences for recommending movies (details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08)}
- H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)}
- H04N 21/4307: {Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen}
- H04N 21/8106: {involving special audio data, e.g. different tracks for different languages}
- G06Q 30/0241: Advertisements
- G06T 2207/30221: Sports video; Sports image
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

In one example, a method performed by a processing system in a telecommunications network includes acquiring live footage of a event, acquiring sensor data related to the event, wherein the sensor data is collected by a sensor positioned in a location at which the event occurs, extracting an analytical statistic related to a target participating in the event, wherein the extracting is based on content analysis of the live footage and the sensor data, filtering data relating to the target based on the analytical statistic to identify content of interest in the data, wherein the data comprises the live footage, the sensor data, and data relating to historical events that are similar to the event, and generating computer-generated content to present the content of interest, wherein when the computer-generated content is synchronized with the live footage on an immersive display, an augmented reality media is produced.

## Claims

### Claim 1

A method comprising:
acquiring, by a processing system in a telecommunications network, live footage of an event;
acquiring, by the processing system, sensor data related to the event, wherein the sensor data is collected by a sensor positioned in a location at which the event occurs;
extracting, by the processing system, an analytical statistic related to a target participating in the event, wherein the extracting is based on content analysis of the live footage and the sensor data;
filtering, by the processing system, data relating to the target based on the analytical statistic to identify content of interest in the data, wherein the data comprises the live footage, the sensor data, and data relating to historical events that are similar to the event;
generating, by the processing system, computer-generated content to present the content of interest, wherein the computer-generated content includes metadata that indicates how to synchronize presentation of the computer-generated content with presentation of the live footage on an immersive display to produce an augmented reality media; and
delivering, by the processing system, the computer-generated content to a user endpoint device including the immersive display, wherein the user endpoint device is currently acquiring the live footage from a source other than the processing system.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
prior to the acquiring the live footage and the acquiring the sensor data, discovering, by the processing system, an occurrence of the event.

### Claim 3 (depends on 2)

The method of claim 2, wherein the discovering comprises reviewing a public schedule on which the event appears.

### Claim 4 (depends on 2)

The method of claim 2, wherein the discovering comprises detecting a social media posting about the event.

### Claim 5 (depends on 4)

The method of claim 4, wherein the social media posting is one of a plurality of social media postings posted by a plurality of social media users, and wherein the plurality of social media postings includes user-generated video of the event.

### Claim 6 (depends on 1)

The method of claim 1, wherein the sensor comprises at least one selected from a group of: a radar sensor, a lidar sensor, an infrared depth measurement sensor, and an audio sensor.

### Claim 7

(canceled)

### Claim 8 (depends on 1)

The method of claim 1, wherein the sensor data comprises a depth-sensitive image.

### Claim at position 9

-10. (canceled)

### Claim 11 (depends on 1)

The method of claim 1, wherein the extracting comprises:
identifying, by the processing system, the target;
tracking, by the processing system, the target through the live footage and the sensor data; and
quantifying, by the processing system, a change in a trajectory of the target based on the tracking.

### Claim 12 (depends on 11)

The method of claim 11, wherein the identifying comprises:
receiving, by the processing system, a request from a user to track the target.

### Claim 13 (depends on 11)

The method of claim 11, wherein the identifying comprises:
learning, by the processing system, that the target is of interest to a viewer, based on a review by the processing system of historical event that is similar to the event.

### Claim 14 (depends on 11)

The method of claim 11, wherein the change in the trajectory is defined relative to a reference point in the live footage.

### Claim 15 (depends on 1)

The method of claim 1, wherein the filtering comprises:
performing, by the processing system, a search of the live footage and the sensor data, wherein the search uses the analytical statistic as part of a numerical range query.

### Claim 16 (depends on 1)

The method of claim 1, wherein the filtering comprises:
performing, by the processing system, a search of the live footage and the sensor data, wherein the search uses the analytical statistic as part of a direct identity query.

### Claim 17 (depends on 1)

The method of claim 1, wherein the computer-generated content comprises a visual overlay that is superimposed over the live footage to produce the augmented reality media.

### Claim 18 (depends on 1)

The method of claim 1, wherein the computer-generated content comprises a synthesized audio track generated as the live footage and sensor data are acquired using text-to-speech technology that is synchronized with the live footage to produce the augmented reality media.

### Claim 19

A non-transitory computer-readable medium storing instructions which, when executed by a processing system in a telecommunications network, cause the processing system to perform operations, the operations comprising:
acquiring live footage of an event;
acquiring sensor data related to the event, wherein the sensor data is collected by a sensor positioned in a location at which the event occurs;
extracting an analytical statistic related to a target participating in the event, wherein the extracting is based on content analysis of the live footage and the sensor data;
filtering data relating to the target based on the analytical statistic to identify content of interest in the data, wherein the data comprises the live footage, the sensor data, and data relating to historical events that are similar to the event;
generating computer-generated content to present the content of interest, wherein the computer-generated content includes metadata that indicates how to synchronize presentation of the computer-generated content with presentation of the live footage on an immersive display to produce an augmented reality media; and
delivering the computer-generated content to a user endpoint device including the immersive display, wherein the user endpoint device is currently acquiring the live footage from a source other than the processing system.

### Claim 20

A device comprising:
a processor; and
a computer-readable medium storing instructions which, when executed by the processor, cause the processor to perform operations, the operations comprising:
acquiring live footage of an event;
acquiring sensor data related to the event, wherein the sensor data is collected by a sensor positioned in a location at which the event occurs;
extracting an analytical statistic related to a target participating in the event, wherein the extracting is based on content analysis of the live footage and the sensor data;
filtering data relating to the target based on the analytical statistic to identify content of interest in the data, wherein the data comprises the live footage, the sensor data, and data relating to historical events that are similar to the event;
generating computer-generated content to present the content of interest, wherein the computer-generated content includes metadata that indicates how to synchronize presentation of the computer-generated content with presentation of the live footage on an immersive display to produce an augmented reality media; and
delivering the computer-generated content to a user endpoint device including the immersive display, wherein the user endpoint device is currently acquiring the live footage from a source other than the processing system.

### Claim 21 (depends on 1)

The method of claim 1, wherein the filtering is further based on a current viewpoint of a user of the user endpoint device.

### Claim 22 (depends on 1)

The method of claim 1, wherein the sensor data is tagged with metadata to indicate a relative priority of the sensor data.

### Claim 23 (depends on 1)

The method of claim 1, wherein the sensor data is tagged with metadata to indicate a reference location related to the sensor data.
