# US 10,321,204 B2 · Intelligent closed captioning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,321,204 B2
- Document ID: US10321204B2
- Status: Granted
- Kind: B2
- Application number: 14/329,374
- Filed: 2014-07-11
- Published: 2016-01-14
- Granted: 2019-06-11
- Earliest claimed: 2014-07-11
- Assignees: Lenovo (Singapore) Pte. Ltd.
- Applicants: Lenovo (Singapore) Pte. Ltd.
- Inventors: Neal Robert Caliendo, Jr.; Russell Speight VanBlon; Arnold S. Weksler
- Web page: https://patentgrep.com/patent/US10321204B2

## CPC

- H04N 21/8133: {specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme}
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- H04N 21/4396: {by muting the audio signal}
- H04N 21/4884: {for displaying subtitles}
- H04N 21/42201: {biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user (input arrangements for interaction with the human body based on nervous system activity detection G06F3/015)}
- H04N 21/42206: {characterized by hardware details}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An aspect provides a method, including: playing, on a display device, video content; providing, using at least one speaker, audio content associated with the video content; obtaining, from an external source, data relating to playback context; determining, using a processor, that the data relating to playback context is associated with a reduced audibility context; and providing, on the display device, textual data associated with dialog of the video content. Other aspects are described and claimed.

## Claims

### Claim 1

A method, comprising:
playing, on a display device, video content;
providing, using at least one speaker, audio content associated with the video content;
obtaining, from an external source, playback context data, wherein the playback context data comprises contextual data associated with user comprehension of the audio content and wherein the playback context data comprises ambient audio data;
determining, using a processor, that the playback context data is associated with a reduced audibility context, wherein the determining comprises:
analyzing, using a speech recognition engine, the ambient audio data;
identifying a predetermined trigger expression in the ambient audio data;
accessing, responsive to identifying the predetermined trigger expression, closed caption data; and
providing, on the display device, the closed caption data associated with the dialogue and the ambient audio data comprising the predetermined trigger expression.

### Claim 2 (depends on 1)

The method of claim 1, further comprising associating, using a processor, the playback context data associated with a reduced audibility context with the dialogue of the audio content.

### Claim 3 (depends on 1)

The method of claim 1, wherein the closed caption data includes a reference to previously displayed video data.

### Claim 4 (depends on 3)

The method of claim 3, wherein the reference indicates a particular speaker within the video data.

### Claim 5 (depends on 1)

The method of claim 1, wherein the predetermined trigger expression is associated using natural language processing to a user provided indication of inadequate audio.

### Claim 6

An electronic device, comprising:
a speech recognition engine;
a display device;
an input component;
at least one speaker;
one or more processors; and
a memory operatively coupled to the one or more processors, the display device, the at least one speaker, and the input component, the memory storing instructions executable by the one or more processors, the instructions being executable by the one or more processors to:
play, on the display device, video content;
provide, using the at least one speaker, audio content associated with the video content;
obtain, from an external source, playback context data, wherein the playback context data comprises data associated with user comprehension of the audio content and wherein the playback context data comprises ambient audio data;
determine that the playback context data is associated with a reduced audibility context, wherein the instructions executable by the processor to determine comprise instructions executable by the processor to:
analyze the ambient audio data;
identify a predetermined trigger expression in the ambient audio data;
access, responsive to identifying the predetermined trigger expression, closed caption data; and
provide, on the display device, the closed caption data associated with the dialogue and the ambient audio data comprising the predetermined trigger expression.

### Claim 7 (depends on 6)

The electronic device of claim 6, wherein the instructions are further executable by the one or more processors to associate the playback context data associated with a reduced audibility context with the dialogue of the audio content.

### Claim 8 (depends on 6)

The electronic device of claim 6, wherein the closed caption data includes a reference to previously displayed video data.

### Claim 9 (depends on 8)

The electronic device of claim 8, wherein the reference indicates a particular speaker within the video data.

### Claim 10

A computer program product, comprising:
a storage device that stores instructions executable by a processor, the instructions comprising:
instructions that play, on a display device, video content;
instructions that provide, using at least one speaker, audio content associated with the video content;
instructions that obtain, from an external source, playback context data, wherein the playback context data comprises contextual data associated with user comprehension of the audio content and wherein the playback context data comprises ambient audio data;
instructions that determine, using a processor, that the playback context data is associated with a reduced audibility context, wherein the instructions that determine comprise instructions that:
analyze the ambient audio data;
identify a predetermined trigger expression in the ambient audio data;
access, responsive to identifying the predetermined trigger expression, closed caption data; and
provide, on the display device, the closed caption data associated with the dialogue and the ambient audio data comprising the predetermined trigger expression.
