# US 2023/0156245 A1 · Systems and methods for processing and presenting media data to allow virtual engagement in events

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0156245 A1
- Document ID: US20230156245A1
- Status: Published application
- Kind: A1
- Application number: 17/919,317
- Filed: 2021-04-17
- Published: 2023-05-18
- Earliest claimed: 2020-04-17
- Assignees: 15 Seconds of Fame, Inc.
- Applicants: 15 Seconds of Fame, Inc.
- Inventors: Adam RESNICK; Gregg DONNENFELD
- Web page: https://patentgrep.com/patent/US20230156245A1

## CPC

- H04N 21/21805: {enabling multiple viewpoints, e.g. using a plurality of cameras}
- H04N 21/2187: Live feed
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/25866: {Management of end-user data (customer care in data networks H04L41/5077)}
- H04N 21/2743: Video hosting of uploaded data from client
- H04N 21/41415: {involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/431: Generation of visual interfaces {for content selection or interaction}; Content or additional data rendering
- H04N 21/4415: using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning {(cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861)}
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/44227: {Monitoring of local network, e.g. connection or bandwidth variations; Detecting new devices in the local network (configuring of peripheral devices in general G06F9/4411; monitoring connectivity in data switched networks H04L43/0811)}
- H04N 21/472: End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content {(end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612)}
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 21/632: {using a connection between clients on a wide area network, e.g. setting up a peer-to-peer communication via Internet for retrieving video segments from the hard-disk of other client devices (broadcast-related systems characterised by transmission among terminal devices H04H60/80; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 21/8146: {involving graphical data, e.g. 3D object, 2D graphics}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An illustrative example method of hosting a virtual audience during an event at a venue includes distributing an observable representation of the event to be received by a plurality of user devices located remote from the venue; receiving a media stream from each of a plurality of virtual attendees located remote from the venue, each received media stream including a visual representation of at least one of the plurality of virtual attendees; and displaying, on a display at the venue, the visual representation of at least some of the virtual attendees such that the virtual attendees appear to be attending the event at the venue.

## Claims

### Claim at position 1

1-25. (canceled)

### Claim 26

A method of hosting a virtual audience during a live event at a physical venue, the method comprising:
distributing an observable representation of the live event to be received by a plurality of user devices located remote from the physical venue;
receiving a live video stream from each of a plurality of persons located remote from the physical venue, each received live video stream including a visual representation of at least one of the plurality of persons; and
displaying, on a physical display at the physical venue, the live video stream of at least some of the persons such that the persons appear to be attending the live event as virtual attendees at the physical venue.

### Claim 27 (depends on 26)

The method of claim 26, wherein
the received live video streams include audio representing sounds made by the virtual attendees, and
the method includes reproducing the sounds within the physical venue so the sounds made by the virtual attendees are audible at the physical venue.

### Claim 28 (depends on 26)

The method of claim 26, comprising
determining contextual information corresponding to each received live video stream, and
selecting the at least some of the virtual attendees for the displaying based on the contextual information.

### Claim 29 (depends on 28)

The method of claim 28, comprising
using at least one of facial recognition or voice recognition for recognizing at least one individual in each received live video stream,
including a result of the facial recognition or voice recognition in the contextual information, and
selecting the at least some of the virtual attendees based on the included result of the facial recognition or voice recognition.

### Claim 30 (depends on 29)

The method of claim 29, comprising selecting a position of the visual representation of the recognized individual within the physical venue based on the result of the facial recognition or voice recognition.

### Claim 31 (depends on 30)

The method of claim 30, comprising grouping the visual representation of some of the plurality of virtual attendees within the physical venue based on the result of the facial recognition or voice recognition.

### Claim 32 (depends on 29)

The method of claim 29, comprising
determining at least one other characteristic of the live video stream including a recognized individual, and
selecting a position of the visual representation of the recognized individual within the physical venue based on the at least one other characteristic.

### Claim 33 (depends on 32)

The method of claim 32, comprising grouping the visual representation of some of the plurality of virtual attendees within the physical venue based on a similarity between the determined at least one other characteristic of the respective live video streams of the some of the plurality of virtual attendees.

### Claim 34 (depends on 28)

The method of claim 28, wherein
the contextual information comprises user profile data regarding a corresponding one of the received live video streams, and
the method includes determining, based on the user profile data, whether the visual representation of the corresponding one of the received live video streams should be included among the displayed virtual attendees.

### Claim 35 (depends on 34)

The method of claim 34, comprising establishing a peer networking session between some of the virtual attendees during the event based on at least one of
a choice or selection made by one of the virtual attendees to be in the peer networking session with at least one other of the virtual attendees, or
the user profile data of each of some of the plurality of virtual attendees indicating an association between the some of the virtual attendees.

### Claim 36 (depends on 26)

The method of claim 26, comprising
determining that at least one of the persons appears in the distributed observable representation of the live event or appears on a dedicated display at the physical venue during the event; and
sending a media file to the at least one of the persons during or after the live event, wherein the sent media file includes the appearance of the at least one of the persons.

### Claim 37 (depends on 26)

The method of claim 26, comprising selecting at least one of the virtual attendees and displaying the visual representation of the selected at least one of the virtual attendees differently than others of the visual representations of the virtual attendees for at least a portion of the event.

### Claim 38 (depends on 39)

The method of claim 39, comprising facilitating an interaction between an individual at the physical venue participating in the event and the selected at least one of the virtual attendees while displaying the visual representation of the selected at least one of the virtual attendees differently than others of the visual representations of the virtual attendees.

### Claim 39 (depends on 26)

The method of claim 26, comprising removing the visual representation of one of the virtual attendees from the display based on at least one characteristic of the received live video stream from the at least one of the virtual attendees, wherein the at least one characteristic is
a quality below a minimum quality threshold,
a connection rate below a minimum threshold,
a loss of data packets,
an absence of the visual representation of the one of the virtual attendees, or
inappropriate content.

### Claim 40

A system for hosting a virtual audience during a live event at a physical venue, the system comprising:
a camera arrangement situated at the physical venue, the camera arrangement being configured to capture an observable representation of the live event;
a distribution device that is configured to distribute the observable representation of the live event to be received by a plurality of user devices located remote from the physical venue;
a host device including
a communication interface configured to receive a live video stream from each of a plurality of virtual attendee user devices located remote from the physical venue, each received live video stream including a visual representation of at least one of a plurality of persons,
at least one processor that is configured to analyze the received live video streams and to select at least some of the visual representations of corresponding ones of the plurality of persons; and
at least one display situated at the physical venue, the host device causing the at least one display to include the visual representation of the selected at least some of the visual representations such that the persons corresponding to the selected visual representations appear to be attending the live event as virtual attendees at the physical venue.

### Claim 41 (depends on 40)

The system of claim 40, comprising at least one speaker, and wherein
the received live video streams include audio representing sounds made by the persons;
the host device causes the at least one speaker to reproduce the sounds within the physical venue so the sounds made by the persons are audible at the physical venue; and
the at least one display comprises
a display panel that is configured to include multiple visual representations of virtual attendees, or
a plurality of display panels each configured to include a single visual representation of a corresponding virtual attendee.

### Claim 42 (depends on 40)

The system of claim 40, wherein the at least one processor is configured to
use at least one of facial recognition or voice recognition for recognizing at least one of the persons in each received live video stream,
select the at least some of the visual representations for displaying the virtual attendees based on the facial recognition or voice recognition.

### Claim 43 (depends on 42)

The system of claim 42, wherein the at least one processor is configured to select a position of the visual representation of the recognized one of the persons on the at least one display based on the result of the facial recognition or voice recognition.

### Claim 44 (depends on 43)

The system of claim 43, wherein the at least one processor is configured to
determine at least one other characteristic of the live video stream including the recognized one of the persons, and
select a position of the visual representation of the recognized one of the persons on the at least one display based on the at least one other characteristic.

### Claim 45 (depends on 40)

The system of claim 40, wherein
the at least one processor is configured to determine user profile data regarding a corresponding one of the received live video streams, and
the at least one processor is configured to determine, based on the user profile data, whether the visual representation of the corresponding one of the received live video streams should be included among the displayed virtual attendees.
