Systems and methods for automated fine-grained speech scoring
AssigneeEducational Testing Service
InventorsSeongjin Park, Rutuja Ubale
Life of this patent
Estimating the term…
Abstract
A speech in response to a prompt is accessed. The speech is provided to a speech recognition module that is configured to generate a text transcript of the speech. Speech features are extracted from the speech. Similarly, text features are extracted from the text transcript. Both speech features and text features are vector representations of the speech. The two features are concatenated into one vector representation that captures both perceptual and linguistic components of the speech. The concatenated vector is provided to a speech scoring model. The speech scoring model simultaneously provides a holistic score as well as fine-grained scores to the speech based on the concatenated features.
Claims21
A computer-implemented method comprising:
- accessing a speech in response to a prompt;
- extracting, with a speech representation module that comprises a first transformer model, speech features from the speech as vector representations;
- providing the speech to a speech recognition module;
- generating, with the speech recognition module, a text transcript of the speech;
- extracting, with a text representation module that comprises a second transformer model, text features from the text transcript as vector representations;
- concatenating, with a concatenation module, the speech features and the text features into a combined vector representation;
- and
- simultaneously providing, with a speech scoring module, a holistic score and fine-grained scores as scalar values based on the concatenated features.
A system comprising:
- one or more data processors;
- and
- a computer-readable medium encoded with instructions for commanding the one or more data processors to execute steps of a process, the steps comprising:
- accessing a speech in response to a prompt;
- extracting, with a speech representation module that comprises a first transformer model, speech features from the speech;
- providing the speech to a speech recognition module;
- generating, with the speech recognition module, a text transcript of the speech;
- extracting, with a text representation module that comprises a second transformer model, text features from the text transcript;
- concatenating, with a concatenation module, the speech features and the text features;
- and
- simultaneously providing, with a speech scoring module, a holistic score and fine-grained scores based on the concatenated features.
A computer-implemented method comprising:
- accessing a speech in response to a prompt;
- extracting speech features from the speech as vector representations;
- generating a text transcript of the speech;
- extracting text features from the text transcript as vector representations;
- concatenating the speech features and the text features into a combined vector representation;
- and
- simultaneously providing a holistic score and fine-grained scores as scalar values based on the concatenated features,
- wherein the extracting steps and the scoring step are implemented by neural network language models.
Dependencies are inferred from the claim text.