US 12,737,563 B2Granted

Regional sign language translation

AssigneeDell Products L.P.

InventorsDhilip Kumar, Ajay Maikhuri

Life of this patent

Estimating the term…

Loading patent…

Abstract

An example methodology implementing the disclosed techniques includes, by a computing device, receiving a first video stream captured by a camera, analyzing images within the first video stream to recognize a first regional sign language, and determining a caption that portrays a meaning conveyed by the recognized first regional sign language. The method also includes, by the computing device, translating the first caption to a neutral language to generate a neutral language caption, translating the neutral language caption to generate a regional language caption, the regional language associated with an intended recipient of the first video stream, and generating a second video stream composed of a second regional sign language images representing the regional language caption. The method may also include sending the second video stream to the intended recipient of the first video stream. The first video stream may be received during a sign language conversation session.

Claims20

Dependent claims

A method comprising:

receiving, by a computing device, a first video stream captured by a camera;
de-noising, by the computing device, images within the first video stream to reduce artifacts other than sign language content from the image;
analyzing, by the computing device, the de-noised images to recognize sign language content appearing within the first video stream using a generative adversarial text-to-image synthesis model configured to distinguish between real sign language images and fake sign language images in a first regional sign language, wherein the generative adversarial text-to-image synthesis model is trained using first training images with corresponding text captions that portray a meaning conveyed by the first training images and second training images with arbitrary non corresponding text captions, the sign language content being in the first regional sign language;
determining, by the computing device, a first caption in a first caption language that portrays a meaning conveyed by the content in the first regional sign language, wherein the first caption language is associated with the first regional sign language;
translating, by the computing device, the first caption from the first caption language to a neutral language to generate a neutral language caption;
translating, by the computing device, the neutral language caption to generate a first regional language caption in a first regional language, the first regional language associated with a first intended recipient of the first video stream, the first regional language being different from the neutral language;
and
generating, by the computing device, a second video stream composed of second regional sign language images representing the first regional language caption.

5 more elements

A computing device comprising:

one or more non-transitory machine-readable mediums configured to store instructions;
and
one or more processors configured to execute the instructions stored on the one or more non-transitory machine-readable mediums, wherein execution of the instructions causes the one or more processors to carry out a process comprising:
receiving a first video stream captured by a camera;
analyzing images within the first video stream to reduce noise comprising artifacts other than content in a first regional sign language from the images using a generative adversarial text-to-image synthesis model configured to distinguish between real sign language images and fake sign language images in the first regional sign language, wherein the generative adversarial text-to-image synthesis model is trained using first training images with corresponding text captions that portray a meaning conveyed by the first training images and second training images with arbitrary non-corresponding text captions;
determining a first caption in a first caption language that portrays a meaning conveyed by the content in the first regional sign language, wherein the first caption language is associated with the first regional sign language;
translating the first caption from the first caption language to a neutral language to generate a neutral language caption in a first regional language;
translating the neutral language caption to generate a first regional language caption in the first regional language, the first regional language associated with a first intended recipient of the first video stream, the first regional language being different from the neutral language;
and
generating a second video stream composed of second regional sign language images representing the first regional language caption.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/189,308Filed 24 Mar 2023
Published: US 2024/0320449 A126 Sep 2024
Granted: 15 Sep 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification7 codes

G06F 40/58
Processing or translation of natural language: use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for…
G06V 20/41
In video content: higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian…
G06V 40/28
Movements or behaviour, e.g. gesture recognition: recognition of hand or arm movements, e.g. recognition of deaf sign language
H04L 65/1083
Session management: in-session procedures