US 11,636,858 B2 · Audio analysis system for automatic language proficiency assessment

Abstract

A language proficiency analyzer automatically evaluates a person's language proficiency by analyzing that person's oral communications with another person. The analyzer first enhances the quality of an audio recording of a conversation between the two people using a neural network that automatically detects loss features in the audio and adds those loss features back into the audio. The analyzer then performs a textual and audio analysis on the improved audio. Through textual analysis, the analyzer uses a multi-attention network to determine how focused one person is on the other and/or how pleased one person is with the other. Through audio analysis, the analyzer uses a neural network to determine how well one person pronounced words during the conversation.

Claims20

Dependent claims

An apparatus comprising:

a memory;
and
a hardware processor communicatively coupled to the memory, the hardware processor configured to:
receive an audio signal of a conversation between a first person and a second person;
receive a plurality of vectors corresponding to a plurality of words spoken during the conversation, each vector of the plurality of vectors indicative of a meaning of a word of the plurality of words;
compare the audio signal to the plurality of vectors using a neural network to produce a singular value decomposition;
generate, based on the singular value decomposition, a first score indicative of a pronunciation of the plurality of words during the conversation;
receive a second score indicative of how attentive the first person was to the second person during the conversation;
and
combine the first and second scores to produce a third score indicative of a performance of the first person during the conversation.

7 more elements

A method comprising:

receiving, by a hardware processor communicatively coupled to a memory, an audio signal of a conversation between a first person and a second person;
receiving, by the hardware processor, a plurality of vectors corresponding to a plurality of words spoken during the conversation, each vector of the plurality of vectors indicative of a meaning of a word of the plurality of words;
comparing, by the hardware processor, the audio signal to the plurality of vectors using a neural network to produce a singular value decomposition;
generating, by the hardware processor and based on the singular value decomposition, a first score indicative of a pronunciation of the plurality of words during the conversation;
receiving, by the hardware processor, a second score indicative of how attentive the first person was to the second person during the conversation;
and
combining, by the hardware processor, the first and second scores to produce a third score indicative of a performance of the first person during the conversation.

4 more elements

A system comprising:

a first device;
a second device;
and
a language proficiency analyzer comprising a hardware processor communicatively coupled to a memory, the hardware processor configured to:
receive an audio signal of a conversation between a first person using the first device and a second person using the second device;
receive a plurality of vectors corresponding to a plurality of words spoken during the conversation, each vector of the plurality of vectors indicative of a meaning of a word of the plurality of words;
compare the audio signal to the plurality of vectors using a neural network to produce a singular value decomposition;
generate, based on the singular value decomposition, a first score indicative of a pronunciation of the plurality of words during the conversation;
receive a second score indicative of how attentive the first person was to the second person during the conversation;
and
combine the first and second scores to produce a third score indicative of a performance of the first person during the conversation.

8 more elements

Dependencies are inferred from the claim text.

Audio analysis system for automatic language proficiency assessment

Life of this patent

Audio analysis system for automatic language proficiency assessment

Life of this patent

Abstract

Claims20