US 2019/0198022 A1 · DEVICE AND METHOD FOR GENERATING TEXT REPRESENTATIVE OF LIP MOVEMENT

Abstract

A device and method for generating text representative of lip movement is provided. One or more portions of video data are determined that include: audio with an intelligibility rating below a threshold intelligibility rating; and lips of a human face. A lip-reading algorithm is applied to the one or more portions of the video data to determine text representative of detected lip movement in the one or more portions of the video data. The text representative of the detected lip movement is stored in a memory. A transcript that includes the text representative of the detected lip movement may be generated. Captioned video data may be generated from the video data and the text representative of detected lip movement.

Claims20

Dependent claims

A device comprising:

a controller and a memory, the controller configured to:
determine one or more portions of video data that include:
audio with an intelligibility rating below a threshold intelligibility rating;
and lips of a human face;
apply a lip-reading algorithm to the one or more portions of the video data to determine text representative of detected lip movement in the one or more portions of the video data;
and
store, in the memory, the text representative of the detected lip movement.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application

15/850,814Filed 21 Dec 2017

Published

27 Jun 2019

Term ends

Not estimatedThe source records don’t support an estimate.

Classification8 codes

G10L 15/25
Speech recognition using non-acoustical features: using position of the lips, movement of the lips or face analysis
G06V 20/40
Scenes; Scene-specific elements: in video content
G06V 40/20
Recognition of biometric, human-related or animal-related patterns in image or video data: movements or behaviour, e.g. gesture recognition
G10L 15/26
Speech recognition: speech to text systems

Device and method for generating text representative of lip movement

Life of this patent

Device and method for generating text representative of lip movement

Life of this patent

Abstract

Claims20