Back to results
US 12,705,931 B2Granted
Sign-language translation
AssigneeSorenson IP Holdings, LLC
InventorsAbolfazl Zargari Khuzani, Naveen Kulkarni, Mariam Rahmani, Shashank Bujimalla Venkata Sesha, Abolfazl Ravanshad
Life of this patent
Estimating the term…
Abstract
System and techniques to facilitate the translation of a sign language into another language are described herein. A modular architecture may be used in which the output of different classifiers may be used to produce intermediate representations, or final translations, of the sign language. These classifiers may be trained on different types of signs to enhance accuracy while reduce training time and complexity.
Claims20
An apparatus for sign-language translation, the apparatus comprising:
- a memory including instructions;
- and
- processing circuitry that, when in operation, is configured by the instructions to:
- obtain a video segment that includes one or more signs in sign language;
- invoke a first classifier trained to identify individual signs and produce a single symbol based on individual signs;
- in response to the first classifier indicating that the video segment does not include a single sign, the processing circuitry is further configured to:
- invoke a second classifier to divide the video segment into parts that correspond to discrete signs in a sequence of signs in the video segment;
- in response to the second classifier dividing the video segment into parts that correspond to the discrete signs, the processing circuitry is further configured to:
- invoke a third classifier on the parts of the video segment to produce first labels for the sequence of signs, the third classifier trained on regular continuous signs that are non-fingerspelled signs, each first label indicating either detection of a regular continuous sign or no detection of a regular continuous sign;
- and
- invoke a fourth classifier on the parts of the video segment to produce second labels for the sequence of signs, the fourth classifier trained on fingerspelled signs, the fingerspelled signs corresponding to hand movements representing one or more individual letters, each second label indicating either detection of a fingerspelled sign or no detection of a fingerspelled sign;
- and
- produce a sequence of symbols based on the sequence of signs by merging the first labels and the second labels.
10 more elements
At least one non-transitory machine readable medium including instructions for sign-language translation, the instructions, when executed by processing circuitry, cause the processing circuitry to perform operations comprising:
- obtaining a video segment that includes one or more signs in sign language;
- invoking a first classifier trained to identify individual signs and produce a single symbol based on individual signs;
- in response to the first classifier indicating that the video segment does not include a single sign, the operations further comprise:
- invoking a second classifier to divide the video segment into parts that correspond to discrete signs in a sequence of signs in the video segment;
- in response to the second classifier dividing the video segment into parts that correspond to the discrete signs, the operations further comprise:
- invoking a third classifier on the parts of the video segment to produce first labels for the sequence of signs, the third classifier trained on regular continuous signs that are non-fingerspelled signs, each first label indicating either detection of a regular continuous sign or no detection of a regular continuous sign;
- and
- invoking a fourth classifier on the parts of the video segment to produce second labels for the sequence of signs, the fourth classifier trained on fingerspelled signs that correspond to hand movements representing one or more individual letters, each second label indicating either detection of a fingerspelled sign or no detection of a fingerspelled sign;
- and
- producing a sequence of symbols based on the sequence of signs by merging the first labels and the second labels.
7 more elements
A system for sign-language translation, the system comprising:
- means for obtaining a video segment that includes one or more signs in sign language;
- means for invoking a first classifier trained to identify individual signs and produce a single symbol based on individual signs;
- means for invoking a second classifier to divide the video segment into parts that correspond to discrete signs in a sequence of signs in the video segment in response to the first classifier indicating that the video segment does not include a single sign;
- means for invoking a third classifier on the parts to produce first labels for the sequence of signs in response to the second classifier dividing the video segment into parts that correspond to the discrete signs, the third classifier trained on regular continuous signs that are non-fingerspelled signs, each first label indicating either detection of a regular continuous sign or no detection of a regular continuous sign;
- means for invoking a fourth classifier on the parts to produce second labels for the sequence of signs in response to the second classifier dividing the video segment into parts that correspond to the discrete signs, the fourth classifier trained on fingerspelled signs corresponding to hand movements representing one or more individual letters, each second label indicating either a detection of a fingerspelled sign or no detection of a fingerspelled sign;
- and
- means for producing a sequence of symbols based on the sequence of signs by merging the first labels and the second labels.
4 more elements
Dependencies are inferred from the claim text.