# US 2023/0276102 A1 · Object-based video commenting

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0276102 A1
- Document ID: US20230276102A1
- Status: Published application
- Kind: A1
- Application number: 17/437,592
- Filed: 2020-11-16
- Published: 2023-08-31
- Earliest claimed: 2020-11-16
- Assignees: ARRIS Enterprises LLC
- Applicants: ARRIS Enterprises LLC
- Inventors: Ji CHEN; Yu'an CHEN; Kunming REN; Xiang SHEN; Yeyi CUI
- Web page: https://patentgrep.com/patent/US20230276102A1

## CPC

- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/234318: {by decomposing into objects, e.g. MPEG-4 objects}
- H04N 21/431: Generation of visual interfaces {for content selection or interaction}; Content or additional data rendering
- H04N 21/4333: {Processing operations in response to a pause request}
- H04N 21/47202: {for requesting content on demand, e.g. video on demand}
- H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)}
- H04N 21/475: End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
- G06T 2207/20084: Artificial neural networks [ANN]
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method, system, and computer program product for object-based commenting in an on-demand video includes a processor to receive an on-demand video file selection from a first user for display on a first user device. The processor can receive a first user input pausing the video file at a scene from the first via a first graphical user interface. The processor can receive a second user input from the first user via a first graphical user interface. The second user input can include an object identification and a user comment associated with the object. The processor can identify the object in the scene of the video file based on the object identification and display the second input from the first user to one or more second users via one or more second graphical user interfaces with the identified object over the scene of the video file.

## Claims

### Claim 1

A method for object-based commenting in an on-demand video, the method comprising:
receiving a video file selection from a first user for display on a first user device, wherein the video file is an on-demand video file;
receiving a first user input from the first user on the first user device via a first graphical user interface, the first user input pausing the video file at a scene;
receiving a second user input from the first user on the first user device via a first graphical user interface, the second user input including an object identification and a user comment associated with the object
identifying the object in the scene of the video file based on the object identification; and
displaying the second input from the first user to one or more second users on one or more second user devices via one or more second graphical user interfaces, the second user input being displayed with the identified object over the scene of the video file.

### Claim 2 (depends on 1)

A method according to claim 1, wherein identifying the object comprises:
analyzing the user object identification using natural language processing; and
analyzing the scene of the video using and object detection model, the object detection model using a neural network.

### Claim 3 (depends on 1)

A method as in claim 1, wherein the object identification is one or more words describing the object.

### Claim 4 (depends on 1)

A method as in claim 1, wherein the object is at least one of the group consisting of: a person, an animal, an object, and a building.

### Claim 5 (depends on 1)

A method as in claim 1, wherein the second user comment is at least one of the group consisting of: a textual input, an image file input, an audio input, an emoji, an emoticon, and a .gif.

### Claim 6 (depends on 1)

A method as in claim 1, wherein the first graphical user interface and the one or more second graphical user interfaces are bullet screen interfaces.

### Claim 7

A system for object-based commenting in an on-demand video, the system comprising:
one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and instructions stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, the instructions comprising:
instructions to receive a video file selection from a first user for display on a first user device, wherein the video file is an on-demand video file;
instructions to receive a first user input from the first user on the first user device via a first graphical user interface, the first user input pausing the video file at a scene;
instructions to receive a second user input from the first user on the first user device via a first graphical user interface, the second user input including an object identification and a user comment associated with the object
instructions to identify the object in the scene of the video file based on the object identification; and
instructions to display the second input from the first user to one or more second users on one or more second user devices via one or more second graphical user interfaces, the second user input being displayed with the identified object over the scene of the video file.

### Claim 8 (depends on 7)

A system according to claim 7, wherein the instructions to identify the object comprises:
instructions to analyze the user object identification using natural language processing; and
instructions to analyze the scene of the video using and object detection model, the object detection model using a neural network.

### Claim 9 (depends on 7)

A system as in claim 7, wherein the object identification is one or more words describing the object.

### Claim 10 (depends on 7)

A system as in claim 7, wherein the object is at least one of the group consisting of:
a person, an animal, an object, and a building.

### Claim 11 (depends on 7)

A system as in claim 7, wherein the second user comment is at least one of the group consisting of: a textual input, an image file input, an audio input, an emoji, an emoticon, and a .gif.

### Claim 12 (depends on 7)

A system as in claim 7, wherein the first graphical user interface and the one or more second graphical user interfaces are bullet screen interfaces.

### Claim 13

A computer program product for object-based commenting in an on-demand video, the computer program product comprising:
a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method, comprising:
receiving a video file selection from a first user for display on a first user device, wherein the video file is an on-demand video file;
receiving a first user input from the first user on the first user device via a first graphical user interface, the first user input pausing the video file at a scene;
receiving a second user input from the first user on the first user device via a first graphical user interface, the second user input including an object identification and a user comment associated with the object
identifying the object in the scene of the video file based on the object identification; and
displaying the second input from the first user to one or more second users on one or more second user devices via one or more second graphical user interfaces, the second user input being displayed with the identified object over the scene of the video file.

### Claim 14 (depends on 13)

A computer program product according to claim 13, wherein identifying the object comprises:
analyzing the user object identification using natural language processing; and
analyzing the scene of the video using and object detection model, the object detection model using a neural network.

### Claim 15 (depends on 13)

A computer program product as in claim 13, wherein the object identification is one or more words describing the object.

### Claim 16 (depends on 13)

A computer program product as in claim 13, wherein the object is at least one of the group consisting of: a person, an animal, an object, and a building.

### Claim 17 (depends on 13)

A computer program product as in claim 13, wherein the second user comment is at least one of the group consisting of: a textual input, an image file input, an audio input, an emoji, an emoticon, and a .gif.

### Claim 18 (depends on 13)

A computer program product as in claim 13, wherein the first graphical user interface and the one or more second graphical user interfaces are bullet screen interfaces.

### Claim 19

A method for object-based commenting in an on-demand video, the method comprising:
receiving a video file selection from a first user for display on a first user device, wherein the video file is an on-demand video file;
receiving a first user input from the first user on the first user device via a first graphical user interface, the first user input pausing the video file at a scene;
receiving a user selection of an area of the scene, the area including an object;
receiving a second user input from the first user on the first user device via a first graphical user interface, the second user input being associated with the object in the selected area; and
displaying the second input from the first user to one or more second users on one or more second user devices via one or more second graphical user interfaces, the second user input being displayed over the scene of the video file in the selected area.

### Claim 20

A method for object-based commenting in an on-demand video, the method comprising:
receiving a video file selection from a first user for display on a first user device, wherein the video file is an on-demand video file;
receiving a first user input from the first user on the first user device via a first graphical user interface, the first user input pausing the video file at a scene;
Identifying one or more user selectable objects in the scene using object detection;
presenting the one or more user selectable objects associated with the scene to the first user via the first graphical user interface;
receiving a user selection of one of the one or more user selectable objects via the first graphical user interface;
receiving a second user input from the first user on the first user device via a first graphical user interface, the second user input being associated with the selected object; and
displaying the second input from the first user to one or more second users on one or more second user devices via one or more second graphical user interfaces, the second user input being displayed over the scene of the video file.
