# US 2025/0029380 A1 · Comment generation device, comment generation method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0029380 A1
- Document ID: US20250029380A1
- Status: Published application
- Kind: A1
- Application number: 18/709,093
- Filed: 2022-10-20
- Published: 2025-01-23
- Earliest claimed: 2021-11-18
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: FUMINORI HOMMA
- Web page: https://patentgrep.com/patent/US20250029380A1

## CPC

- G06V 20/42: {of sport video content}
- G06F 16/783: using metadata automatically derived from the content
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 10/764: using classification, e.g. of video objects
- H04N 21/234: Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N 21/235: Processing of additional data, e.g. scrambling of additional data or processing content descriptors
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Provided is a technique advantageous for providing a live commentary corresponding to a state of an event recorded in target video data together with a video at an adaptive timing. A comment generation device includes: a history meta information acquisition unit that analyzes target video data and acquires history meta information associated with a target event recorded in the target video data; and a live determination unit that acquires target live classification meta information on the basis of the history meta information and determines a target live commentary corresponding to the target live classification meta information. The history meta information includes past event meta information and live classification meta information associated with an event state before a time point at which the target live commentary is associated in the target event. The event meta information is meta information related to the state of the target event, and the live classification meta information is meta information related to the classification of the live commentary.

## Claims

### Claim 1

A comment generation device comprising:
a history meta information acquisition unit configured to analyze target video data and acquire history meta information associated with a target event recorded in the target video data; and
a live determination unit configured to acquire target live classification meta information on a basis of the history meta information, and determine a target live commentary corresponding to the target live classification meta information, wherein
the history meta information includes past event meta information and live classification meta information associated with an event state before a time point with which the target live commentary is associated in the target event, and
the event meta information is meta information associated with a state of the target event, and
the live classification meta information is meta information associated with classification of the live commentary.

### Claim 2 (depends on 1)

The comment generation device according to claim 1, wherein the live determination unit acquires the target live classification meta information by inputting the history meta information associated with the target event to a learned live issuance model learned to output a live classification meta information on a basis of the history meta information.

### Claim 3 (depends on 2)

The comment generation device according to claim 2, wherein
the learned live issuance model is obtained on a basis of
learning live classification meta information corresponding to a learning target live commentary included in the learning event recorded in the learning video data; and
the live classification meta information acquired by inputting, to a live issuance model, learning history meta information associated with an event state before a time point at which the learning target live commentary is associated in the learning event.

### Claim 4 (depends on 2)

The comment generation device according to claim 2, further comprising:
a live classification unit configured to analyze learning video data to acquire learning live classification meta information corresponding to a learning target live commentary included in a learning event recorded in the learning video data; and
a learning unit configured to learn the live issuance model on a basis of the learning live classification meta information used as teacher data and the live classification meta information acquired by inputting, to the live issuance model, learning history meta information associated with an event state before a time point at which the learning target live commentary is associated in the learning event.

### Claim 5 (depends on 1)

The comment generation device according to claim 1, wherein the live determination unit determines the target live commentary on a basis of live template data selected from among a plurality of pieces of live template data stored in a repository unit in accordance with the target live classification meta information.

### Claim 6 (depends on 5)

The comment generation device according to claim 5, wherein the plurality of pieces of live template data is acquired by inputting the event meta information to a learned live generation model learned to output a plurality of pieces of live template data on a basis of the event meta information.

### Claim 7 (depends on 6)

The comment generation device according to claim 6, further comprising a learning unit that performs learning of the live generation model by using, as teacher data, learning live template data extracted from information disclosed on a network according to the event meta information.

### Claim 8 (depends on 1)

The comment generation device according to claim 1, wherein the event meta information includes information associated with a person.

### Claim 9 (depends on 8)

The comment generation device according to claim 8, wherein
the information associated with the person includes situation meta information estimated on a basis of motion information indicating a motion of the person obtained by analyzing the target video data.

### Claim 10 (depends on 9)

The comment generation device according to claim 9, wherein
the situation meta information includes at least one of scene information indicating a scene content of an event and play information indicating a play content of the event.

### Claim 11 (depends on 9)

The comment generation device according to claim 9, wherein the motion information is based on information on a body part of the person obtained by analyzing the target video data.

### Claim 12 (depends on 9)

The comment generation device according to claim 9, wherein the motion information is based on information on a moving position of the person obtained by analyzing the target video data.

### Claim 13 (depends on 8)

The comment generation device according to claim 8, wherein the information associated with the person includes information for identifying the person derived from at least one of an image of appearance of the person and an image of attachment of the person.

### Claim 14 (depends on 1)

The comment generation device according to claim 1, wherein the event meta information includes information not associated with the person.

### Claim 15 (depends on 1)

The comment generation device according to claim 1, wherein
the target event is a sports game, and
the event meta information includes at least one of scene information regarding a scene content of the game, play information regarding a play content of the game, person identification information regarding a participant of the game, score information regarding a score of the game, and time information regarding a time of the game.

### Claim 16 (depends on 3)

The comment generation device according to claim 3, wherein
a genre of the learning event is different from a genre of the target event.

### Claim 17 (depends on 3)

The comment generation device according to claim 3, wherein
one of the target video data and the learning video data is live-action video data, and the other is generated video data.

### Claim 18

A comment generation method comprising
the steps of:
analyzing target video data and acquiring history meta information associated with a target event recorded in the target video data; and
acquiring target live classification meta information on a basis of the history meta information, and determining a target live commentary corresponding to the target live classification meta information, wherein
the history meta information includes past event meta information and live classification meta information associated with an event state before a time point with which the target live commentary is associated in the target event, and
the event meta information is meta information associated with a state of the target event, and
the live classification meta information is meta information associated with classification of the live commentary.

### Claim 19

A program for
causing a computer to implement:
analyzing target video data and acquiring history meta information associated with a target event recorded in the target video data; and
acquiring target live classification meta information on a basis of the history meta information, and determining a target live commentary corresponding to the target live classification meta information, wherein
the history meta information includes past event meta information and live classification meta information associated with an event state before a time point with which the target live commentary is associated in the target event, and
the event meta information is meta information associated with a state of the target event, and
the live classification meta information is meta information associated with classification of the live commentary.
