# US 2019/0246165 A1 · Messaging and commenting for videos

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0246165 A1
- Document ID: US20190246165A1
- Status: Published application
- Kind: A1
- Application number: 16/341,895
- Filed: 2017-10-18
- Published: 2019-08-08
- Earliest claimed: 2016-10-18
- Inventors: Robert Brouwer; Ahmed Abdulwahab
- Web page: https://patentgrep.com/patent/US20190246165A1

## CPC

- H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
- G06F 16/7867: {using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings}
- H04N 21/44: Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4725: using interactive regions of the image, e.g. hot spots {(details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94)}
- H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
- H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)}
- H04N 21/845: Structuring of content, e.g. decomposing content into time segments
- H04N 21/8583: {by creating hot-spots}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods, processes and systems for contextually augmenting and annotating moving pictures or images with tags using region tracking on computing devices with screen displays, including mobile devices and virtual reality headsets. The present invention enables both content authors and viewers to directly tag and link supplementary content, such as text and video messages, to locations representative of objects in a moving picture or image and share these tags with other authorized users, thereby facilitating messaging.

## Claims

### Claim 1

A method to facilitate communications, the method comprising:
receiving a selection of a location in a starting video frame from a user;
identifying a first group of pixels in proximity to the selected location;
determining whether the first group of pixels can be tracked through subsequent video frames for a predetermined period of time;
permitting the user to attach a tag to the selected location if the first group of pixels can be tracked for the predetermined period of time; and
enabling the user to associate a message with the tag.

### Claim 2 (depends on 1)

The method of claim 1 further comprising playing the video while displaying the tag attached to the first group of pixels beginning at the starting video frame and finishing after the predetermined period of time.

### Claim 3 (depends on 2)

The method of claim 2 wherein the tag is displayed to a first user and is not displayed to a second user.

### Claim 4 (depends on 1)

The method of claim 1 wherein the predetermined period of time is approximately four seconds.

### Claim 5 (depends on 2)

The method of claim 2 further enabling a second user to associate a second message with the message already associated with the tag.

### Claim 6 (depends on 2)

The method of claim 2 further comprising displaying the associated message upon interaction with the displayed tag.

### Claim 7 (depends on 2)

The method of claim 2 further comprising disabling the display of the tag during subsequent plays of the video.

### Claim 8 (depends on 1)

The method of claim 1 wherein the attached tag is stored in a transparent overlay separate from the video.

### Claim 9 (depends on 1)

The method of claim 1 wherein information concerning the attached tag is stored in a database.

### Claim 10 (depends on 1)

The method of claim 1 further comprising selecting a second, larger, group of pixels in proximity to the selected location when the first group of pixels cannot be tracked for the predetermined period of time.

### Claim 11 (depends on 10)

The method of claim 10 further comprising determining whether the second group of pixels can be tracked through subsequent video frames for a predetermined period of time.

### Claim 12 (depends on 11)

The method of claim 11 wherein the predetermined period of time is four seconds.

### Claim 13

A system to facilitate communications, the system comprising:
a source of video content;
a database of tags, each tag being associated with an element in a video content for a predetermined period of time;
a database of messages, each message being associated with a tag.

### Claim 14 (depends on 13)

The system of claim 13 further comprising a player to display video content from the source of video content and at least one tag from the database of tags in proximity to the element in the video with which it is associated.

### Claim 15 (depends on 14)

The system of claim 14 wherein the player displays the at least one tag in a transparent layer overlaid on the displayed video content.

### Claim 16 (depends on 13)

The system of claim 13 further comprising an editor to receive a selection of a location in a video content from a user.

### Claim 17 (depends on 16)

The system of claim 16 further comprising a pixel tracker to track a collection of pixels near the selected location through subsequent frames of the video content.

### Claim 18 (depends on 17)

The system of claim 17 wherein the pixel tracker checks the presence of the pixel collection in a plurality of keyframes.

### Claim 19 (depends on 16)

The system of claim 16 further comprising an object tracker to track an object near the selected location through subsequent frames of the video content.

### Claim 20 (depends on 19)

The system of claim 19 wherein the object tracker tracks the object through the next four seconds of video content.
