US 12,737,565 B2 · Translating speech in a gender-aware manner

Abstract

A technique translates speech in a first language to text in a second language manner in a manner that is appropriate for the gender of the speaker. In some implementations, the technique receives an input setting that specifies one of three translation modes: masculine mode, feminine mode, and auto mode. The first two modes produce translations in masculine and feminine modes by default, respectively, while the auto mode produces translations in forms that are based on the detected characteristics of audio signals. According to some implementations, the technique uses a training framework that automatically converts a corpus of training examples that exhibit gender bias (e.g., a male gender bias) to training examples having a reducing incidence of gender bias. In some implementations, the training framework updates weights of the machine-trained model based on a combination of two loss components: translation loss and gender loss.

Claims20

Dependent claims

A method for training a machine-trained model for translating speech comprising:

receiving original training examples that include audio signals containing speech in a first language by speakers, first-language textual transcripts of the audio signals, and second-language textual transcripts of translations of the audio signals in a second language that is different than the first language;
producing converted training examples by detecting and then correcting instances of gender bias in the second-language textual transcripts, the instances of gender bias having second-language transcripts having gender forms that do not match audio characteristics of associated audio signals and genders of the speakers;
and
training parameters of the machine-trained model, based on the converted training examples, by minimizing loss that is a combination of translation loss that expresses accuracy at which the machine-trained model produces translations, and gender loss that expresses accuracy at which the machine-trained model detects a gender.

1 more element

A computing system for translating speech, comprising:

an instruction data store for storing computer-readable instructions;
and
a processing system for executing the computer-readable instructions in the data store, to perform operations including:
receiving an audio signal from an audio signal source that captures speech in a first language;
receiving a gender preference signal that identifies a translation mode selected from among a set of translation modes;
producing feature information based on the audio signal and the gender preference signal;
converting the feature information into a translation of the speech in the audio signal in a second language that is different than the first language using a machine-trained model;
and
generating output information that includes the translation,
the machine-trained model including machine-trained parameters that are capable of producing translations in the different translation modes, including:
a first translation mode for producing first translations in masculine form irrespective of characteristics of audio signals;
a second translation mode for producing second translations in feminine form irrespective of characteristics of the audio signals, and a third translation mode for producing a mix of masculine and feminine forms based on the characteristics of the audio signals.

9 more elements

Dependencies are inferred from the claim text.

Translating speech in a gender-aware manner

Life of this patent

Translating speech in a gender-aware manner

Life of this patent

Abstract

Claims20