# US 2025/0165725 A1 · Training models for sign language translation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165725 A1
- Document ID: US20250165725A1
- Status: Published application
- Kind: A1
- Application number: 18/957,202
- Filed: 2024-11-22
- Published: 2025-05-22
- Earliest claimed: 2023-11-22
- Assignees: Sorenson IP Holdings, LLC
- Applicants: Sorenson IP Holdings, LLC
- Inventors: David Thomson
- Web page: https://patentgrep.com/patent/US20250165725A1

## CPC

- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06F 40/42: Data-driven translation
- G06F 40/47: Machine-assisted translation, e.g. using translation memory
- G06V 10/778: Active pattern-learning, e.g. online learning of image or video features
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G09B 21/009: {Teaching or communicating with deaf persons (G09B21/04 takes precedence)}
- G10L 15/063: {Training}
- G10L 15/16: using artificial neural networks
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G10L 21/10: Transforming into visible information
- G11B 27/02: Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)

## Abstract

A method may include providing first training data to a translation system configured to translate between sign language and language data. In some embodiments, the translation system may include multiple stages and each of the stages including one or more machine learning models. The method may further include obtaining a first hypothesis output from the translation system based on the first training data and modifying one or more of the machine learning models based on the first hypothesis output. The method may also include providing second training data to a first set of the stages without providing the second training data to other of the stages. The method may further include obtaining a second hypothesis output from the first set of the stages based on the second training data and modifying the machine learning models of the first set of the stages based on the second hypothesis output.

## Claims

### Claim 1

A method comprising:
providing first training data to a translation system configured to translate between sign language and language data, the translation system includes a plurality of stages and each of the plurality of stages including one or more machine learning models;
obtaining a first hypothesis output from the translation system based on the first training data;
modifying one or more of the machine learning models based on the first hypothesis output;
providing second training data to a first set of the plurality of stages without providing the second training data to other of the plurality of stages not included in the first set of the plurality of stages;
obtaining a second hypothesis output from the first set of the plurality of stages based on the second training data; and
modifying one or more of the machine learning models of the first set of the plurality of stages based on the second hypothesis output.

### Claim 2 (depends on 1)

The method of claim 1, wherein the translation system is configured for sign language recognition or sign language generation.

### Claim 3 (depends on 1)

The method of claim 1, wherein the one or more of the machine learning models of the first set of the plurality of stages modified based on the second hypothesis output is the same one or more of the machine learning models modified based on the first hypothesis output.

### Claim 4 (depends on 1)

The method of claim 1, wherein the modifying the one or more of the machine learning models based on the first hypothesis output includes modifying all the machine learning models in the translation system based on the first hypothesis output.

### Claim 5 (depends on 1)

The method of claim 1, wherein the second training data is a subset of the first training data.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first training data is obtained from a communication session between devices and deleted before the communication session ends and the second training data is stored before, during, and after the communication session.

### Claim 7 (depends on 1)

The method of claim 1, wherein the second training data is obtained from a communication session between devices and deleted substantially at an end of the communication session and the first training data is stored before, during, and after the communication session.

### Claim 8 (depends on 1)

The method of claim 1, wherein the steps of providing first training data, obtaining the first hypothesis output, and modifying based on the first hypothesis output comprises end-to-end training and is iteratively repeated and the steps of providing the second training data, obtaining the second hypothesis output, and modifying based on the second hypothesis output comprises sub-training and is iteratively repeated.

### Claim 9 (depends on 8)

The method of claim 8, wherein a number of iterations for the sub-training are different than a number of iterations for the end-to-end training.

### Claim 10 (depends on 8)

The method of claim 8, wherein the iterations for the sub-training are intermixed between iterations for the end-to-end training.

### Claim 11 (depends on 1)

At least one non-transitory computer-readable media configured to store one or more instructions that, in response to being executed by a system, cause or direct the system to perform the method of claim 1.

### Claim 12

A system comprising:
one or more computer readable mediums including instructions;
one or more computing systems coupled to the one or more computer readable mediums and configured to execute the instructions to cause or direct the system to perform operations, the operations comprising:
providing first training data to a translation system configured to translate between sign language and language data, the translation system includes a plurality of stages and each of the plurality of stages including one or more machine learning models;
obtaining a first hypothesis output from the translation system based on the first training data;
modifying one or more of the machine learning models based on the first hypothesis output;
providing second training data to a first set of the plurality of stages without providing the second training data to other of the plurality of stages not included in the first set of the plurality of stages;
obtaining a second hypothesis output from the first set of the plurality of stages based on the second training data; and
modifying one or more of the machine learning models of the first set of the plurality of stages based on the second hypothesis output.

### Claim 13 (depends on 12)

The system of claim 12, wherein the translation system is configured for sign language recognition or sign language generation.

### Claim 14 (depends on 12)

The system of claim 12, wherein the one or more of the machine learning models of the first set of the plurality of stages modified based on the second hypothesis output is the same one or more of the machine learning models modified based on the first hypothesis output.

### Claim 15 (depends on 12)

The system of claim 12, wherein the modifying the one or more of the machine learning models based on the first hypothesis output includes modifying all the machine learning models in the translation system based on the first hypothesis output.

### Claim 16 (depends on 12)

The system of claim 12, wherein the second training data is a subset of the first training data.

### Claim 17 (depends on 12)

The system of claim 12, wherein the first training data is obtained from a communication session between devices and deleted before the communication session ends and the second training data is stored before, during, and after the communication session.

### Claim 18 (depends on 12)

The system of claim 12, wherein the second training data is obtained from a communication session between devices and deleted substantially at an end of the communication session and the first training data is stored before, during, and after the communication session.

### Claim 19 (depends on 12)

The system of claim 12, wherein the steps of providing first training data, obtaining the first hypothesis output, and modifying based on the first hypothesis output comprises end-to-end training and is iteratively repeated and the steps of providing second training data, obtaining the second hypothesis output, and modifying based on the second hypothesis output comprises sub-training and is iteratively repeated.

### Claim 20 (depends on 19)

The system of claim 19, wherein a number of iterations for the sub-training are different than a number of iterations for the end-to-end training.
