# US 12,010,393 B2 · Automatic appending of subtitles based on media context

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,010,393 B2
- Document ID: US12010393B2
- Status: Granted
- Kind: B2
- Application number: 17/387,316
- Filed: 2021-07-28
- Published: 2023-02-02
- Granted: 2024-06-11
- Earliest claimed: 2021-07-28
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Jian Dong Yin; Wen Wang; Zhuo Cai; Rong Fu; Hao Sheng; Kang Zhang
- Web page: https://patentgrep.com/patent/US12010393B2

## CPC

- H04N 21/4884: {for displaying subtitles}
- G06V 10/993: {Evaluation of the quality of the acquired pattern}
- G06V 20/20: in augmented reality scenes
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/8126: {involving additional data, e.g. news, sports, stocks, weather forecasts}
- H04N 21/854: Content authoring
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A processor may automatically generate one or more transcripts based on a media context. The processor may append at least one of the one or more transcripts to the media. The processor may modify the at least one of the one or more transcripts based on an adjustment to a weight factor.

## Claims

### Claim 1

A system for automatically appending additional transcription in a media, the system comprising:
a memory; and
a processor in communication with the memory, the processor being configured to perform operations comprising:
extract dialog from a video fragment, wherein dialog is human voices;
determine one or more quality features from the extracted dialog;
identify a tone of the dialog based on the one or more quality features;
generate, automatically, one or more transcripts based on a media context, wherein the one or more transcripts are not a part of an existing transcript of the media;
append at least one of the one or more transcripts to the media, wherein appending the at least one of the one or more transcripts to the media includes relaying a mood associated with the identified tone of the dialog; and
modify the at least one of the one or more transcripts based on an adjustment to a weight factor.

### Claim 2 (depends on 1)

The system of claim 1, wherein the media context includes one or more background features of the media and a quality of the media.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one of the one or more transcripts is appended based on the weight factor, and wherein the processor is further configured to perform operations comprising:
generate the weight factor.

### Claim 4 (depends on 3)

The system of claim 3, wherein generating the weight factor includes operations comprising:
analyze the media for an occupation ratio;
determine whether the occupation ratio is above an occupation threshold; and
adjust the weight factor based on the determination.

### Claim 5 (depends on 4)

The system of claim 4, wherein determining whether the occupation ratio is above the occupation threshold includes operations comprising:
identify that the occupation ratio is above the occupation threshold; and
prioritize the quality of the media before the one or more background features of the media.

### Claim 6 (depends on 4)

The system of claim 4, wherein determining whether the occupation ratio is above the occupation threshold includes operations comprising:
identify that the occupation ratio is below the occupation threshold; and
prioritize the one or more background features of the media before the quality of the media.

### Claim 7 (depends on 4)

The system of claim 4, wherein the adjustment in the weight factor determines a classification of modification to the at least one of the one or more transcripts.

### Claim 8

A computer-implemented method for automatically appending additional transcription in a media, the method comprising:
extracting dialog from a video fragment, wherein dialog is human voices;
determining one or more quality features from the extracted dialog;
identifying a tone of the dialog based on the one or more quality features;
generating, automatically, one or more transcripts based on a media context, wherein the one or more transcripts are not a part of an existing transcript of the media;
appending at least one of the one or more transcripts to the media, wherein appending the at least one of the one or more transcripts to the media includes relaying a mood associated with the identified tone of the dialog; and
modifying the at least one of the one or more transcripts based on an adjustment to a weight factor.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the media context includes one or more background features of the media and a quality of the media.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein the at least one of the one or more transcripts is appended based on the weight factor, and wherein the method further comprises:
generating the weight factor.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein generating the weight factor includes:
analyzing the media for an occupation ratio;
determining whether the occupation ratio is above an occupation threshold; and
adjusting the weight factor based on the determination.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, wherein determining whether the occupation ratio is above the occupation threshold includes:
identifying that the occupation ratio is above the occupation threshold; and
prioritizing the quality of the media before the one or more background features of the media.

### Claim 13 (depends on 11)

The computer-implemented method of claim 11, wherein determining whether the occupation ratio is above the occupation threshold includes:
identifying that the occupation ratio is below the occupation threshold; and
prioritizing the one or more background features of the media before the quality of the media.

### Claim 14 (depends on 11)

The computer-implemented method of claim 11, wherein the adjustment in the weight factor determines a classification of modification to the at least one of the one or more transcripts.

### Claim 15

A computer program product for automatically appending additional transcription in a media comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform operations, the operations comprising:
extract dialog from a video fragment, wherein dialog is human voices;
determine one or more quality features from the extracted dialog;
identify a tone of the dialog based on the one or more quality features;
generate, automatically, one or more transcripts based on a media context, wherein the one or more transcripts are not a part of an existing transcript of the media;
append at least one of the one or more transcripts to the media, wherein appending the at least one of the one or more transcripts to the media includes relaying a mood associated with the identified tone of the dialog; and
modify the at least one of the one or more transcripts based on an adjustment to a weight factor.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the media context includes one or more background features of the media and a quality of the media.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein the at least one of the one or more transcripts is appended based on the weight factor, and wherein the processor is further configured to perform operations comprising:
generate the weight factor.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein generating the weight factor includes operations comprising:
analyze the media for an occupation ratio;
determine whether the occupation ratio is above an occupation threshold; and
adjust the weight factor based on the determination.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein determining whether the occupation ratio is above the occupation threshold includes operations comprising:
identify that the occupation ratio is above the occupation threshold; and
prioritize the quality of the media before the one or more background features of the media.

### Claim 20 (depends on 18)

The computer program product of claim 18, wherein determining whether the occupation ratio is above the occupation threshold includes operations comprising:
identify that the occupation ratio is below the occupation threshold; and
prioritize the one or more background features of the media before the quality of the media.