# US 2026/0154515 A1 · Speech-to-speech translation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0154515 A1
- Document ID: US20260154515A1
- Status: Published application
- Kind: A1
- Application number: 19/403,777
- Filed: 2025-11-28
- Published: 2026-06-04
- Earliest claimed: 2024-11-29
- Assignees: Mabel AI AB
- Applicants: Mabel AI AB
- Inventors: Shahid JABBAR; Karolina SJÖBERG JABBAR; Sarab YOUSSEF; Yana POPOVA
- Web page: https://patentgrep.com/patent/US20260154515A1

## CPC

- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06F 40/42: Data-driven translation
- G06F 40/45: Example-based machine translation; Alignment
- G06F 40/51: Translation evaluation
- G10L 13/033: Voice editing, e.g. manipulating the voice of the synthesiser
- G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management
- G10L 13/08: Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
- G10L 15/183: using context dependencies, e.g. language models
- G10L 2013/083: {Special characters, e.g. punctuation marks}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A speech-to-speech translation method comprises transcribing speech spoken in a source language into transcribed text data in the source language using an on-premises speech recognition model. The transcribed text data is translated into translated text data in a target language using a first on-premises machine translation model. The translated text data is reverse translated into retranslated text data in the source language using a second, different on-premises machine translation model. The transcribed text and the retranslated text are displayed on a screen. The method also involves synthesizing, using an on-premises speech synthesis model, translated speech data in the target language based on the translated text data and play back, in response to a user confirmation, translated speech in the target language based on the translated speech data in the target language.

## Claims

### Claim 1

A computer-implemented speech-to-speech translation method performed by a processor, the method comprising:
transcribing speech spoken in a source language into transcribed text data in the source language using an on-premises speech recognition model stored in a memory connected to the processor;
translating the transcribed text data into translated text data in a target language using a first on-premises machine translation model stored in the memory;
reverse translating the translated text data into retranslated text data in the source language using a second on-premises machine translation model stored in the memory, wherein the second on-premises machine translation model is different than the first on-premises machine translation model;
displaying, on a screen of a user device, a transcribed text based on the transcribed text data and a retranslated text based on the retranslated text data;
synthesizing, using an on-premises speech synthesis model stored in the memory, translated speech data in the target language based on the translated text data; and
playing back, in response to a user confirmation and based on the translated speech data in the target language, translated speech in the target language

### Claim 2 (depends on 1)

The method according to claim 1, wherein displaying comprises displaying, on the screen, the transcribed text, the retranslated text and a translated text based on the translated text data.

### Claim 3 (depends on 1)

The method according to claim 1, wherein synthesizing comprises synthesizing, in response to the user confirmation and using the on-premises speech synthesis model, the translated speech data in the target language based on the translated text data.

### Claim 4 (depends on 1)

The method according to claim 1, further comprising recording the speech spoken in the source language.

### Claim 5 (depends on 1)

The method according to claim 1, further comprising:
receiving information of the target language;
selecting the first on-premises machine translation model from a set of multiple first on-premises machine translation models stored in the memory based on the information of the target language; and
selecting the second on-premises machine translation model from a set of multiple second on-premises machine translation models stored in the memory based on the information of the target language.

### Claim 6 (depends on 5)

The method according to claim 5, further comprising:
receiving information of the source language; and
selecting the on-premises speech recognition model from a set of multiple on-premises speech recognition models stored in the memory based on the information of the source language, wherein selecting the first on-premises machine translation model comprises selecting the first on-premises machine learning model from the set of multiple first on-premises machine translation models stored in the memory based on the information of the target language and the information of the source language; and
selecting the second on-premises machine translation model comprises selecting the second on-premises machine learning model from the set of multiple second on-premises machine translation models stored in the memory based on the information of the target language and the information of the source language.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising:
receiving gender information of a gender of a listener and/or of a speaker; and
on-premises adapting the translated text data into gender-specific translated text data based on gender-specific linguistic rules selected based on the information of the gender of the listener and/or of the speaker, wherein synthesizing comprises synthesizing, using the on-premises speech synthesis model, the translated speech data in the target language based on the gender-specific translated text data.

### Claim 8 (depends on 1)

The method according to claim 1, further comprising:
receiving gender information of a gender of a speaker; and
selecting a male voice or a female voice based on the gender information of the speaker, wherein
playing back comprises playing back, in response to the user confirmation and in the selected male voice or female voice, the translated speech in the target language based on the translated speech data in the target language.

### Claim 9 (depends on 1)

The method according to claim 1, further comprising:
analyzing the transcribed text data using an on-premises rule engine to identify words or phrases that are labelled as hard to translate; and
replacing any identified words or phrases are labelled as hard to translate in the transcribed text data with pre-defined substitutions using an on-premises rule engine to obtain adjusted transcribed text data in the source language, wherein translating the transcribed text data comprises translating the adjusted transcribed text data into the translated text data in the target language using the first on-premises machine translation model.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the words or phrases that are labelled as hard to translate are selected from the group consisting of idiomatic expressions, domain-specific terms, and words and phrases not part of the training data used to train the first on-premises machine translation model.

### Claim 11 (depends on 1)

The method according to claim 1, further comprising converting the transcribed text data in the source language into standard transcribed text data in a standard version of the source language, wherein translating the transcribed text data comprises translating the standard transcribed text data into the translated text data in the target language using the first on-premises machine translation model.

### Claim 12 (depends on 11)

The method according to claim 11, wherein converting the transcribed text data comprises converting (S40) the transcribed text data in the source language into the standard transcribed text data in the standard version of the source language by replacing dialect-specific words or expressions with standard words or expressions in the source language.

### Claim 13 (depends on 1)

The method according to claim 1, further comprising:
removing diacritics from the translated text data in the target language to obtain diacritics-free translated text data in the target language; and
adding diacritics to the diacritics-free translated text data in the target language to obtain translated text data with diacritics in the target language, wherein
synthesizing speech comprises synthesizing, using the on-premises speech synthesis model, the translated speech data in the target language based on the translated text data with diacritics.

### Claim 14

A user device comprising:
a microphone configured to record speech;
a speaker configured to play back speech;
a screen;
a memory comprising a speech recognition model, a first machine translation model, a second machine translation model and a speech synthesis model; and
a processor configured to:
transcribe speech spoken in a source language and recorded by the microphone into transcribed text data in the source language using the speech recognition model;
translate the transcribed text data into translated text data in a target language using the first machine translation model;
reverse translate the translated text data into retranslated text data in the source language using the second machine translation model that is different than the first machine translation model;
display, on the screen, a transcribed text based on the transcribed text data and a retranslated text based on the retranslated text data;
synthesize, using the speech synthesis model, speech data in the target language based on the translated text data; and
control, in response to a user confirmation, the speaker to play back speech in the target language based on the speech data in the target language.

### Claim 15

A computer program comprising instructions, which when executed by at least one processor, cause the at least one processor to:
transcribe speech spoken in a source language into transcribed text data in the source language using an on-premises speech recognition model;
translate the transcribed text data into translated text data in a target language using a first on-premises machine translation model;
reverse translate the translated text data into retranslated text data in the source language using a second on-premises machine translation model that is different than the first on-premises machine translation model;
output the transcribed text data and the retranslated text data for display of a transcribed text and a retranslated text;
synthesize, using an on-premises speech synthesis model, translated speech data in the target language based on the translated text data; and
output, in response to a user confirmation, the translated speech data in the target language for playback.

### Claim 16 (depends on 15)

A non-volatile computer-readable storage medium comprising a computer program according to claim 15.

### Claim 17

A system comprising:
a user device; and
one or more computing devices implemented in a private or local network or cloud, wherein
the user device comprises:
a microphone configured to record speech;
a speaker configured to play back speech;
a screen; and
a communication unit configured to communicate with the one or more computing devices in the private or local network or cloud; and
the one or more computing devices comprises:
one or more memories comprising an on-premises speech recognition model, a first on-premises machine translation model, a second on-premises machine translation model and an on-premises speech synthesis model; and
one or more processors configured to:
transcribe speech spoken in a source language and recorded by the microphone into transcribed text data in the source language using the on-premises speech recognition model;
translate the transcribed text data into translated text data in a target language using the first on-premises machine translation model;
reverse translate the translated text data into retranslated text data in the source language using the second on-premises machine translation model that is different than the first on-premises machine translation model;
transmit the transcribed text data and the retranslated text data to the user device, wherein the user device is configured to display, on the screen, a transcribed text based on the transcribed text data and a retranslated text based on the retranslated text data;
synthesize, using the on-premises speech synthesis model, speech data in the target language based on the translated text data; and
transmit, in response to a user confirmation from the user device, the speech data in the target language, wherein the user device is configured to play back speech in the target language on the speaker based on the speech data in the target language.
