# US 12,190,870 B2 · Learning device, learning method, and learning program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,190,870 B2
- Document ID: US12190870B2
- Status: Granted
- Kind: B2
- Application number: 16/966,056
- Filed: 2019-02-01
- Published: 2020-11-19
- Granted: 2025-01-07
- Earliest claimed: 2018-02-02
- Assignees: Nippon Telegraph and Telephone Corporation
- Applicants: Nippon Telegraph and Telephone Corporation
- Inventors: Atsunori Ogawa; Marc Delcroix; Shigeki Karita; Tomohiro Nakatani
- Web page: https://patentgrep.com/patent/US12190870B2

## CPC

- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G06F 18/2113: by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G10L 15/063: {Training}
- G10L 15/16: using artificial neural networks
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A learning device includes a memory, and processing circuitry coupled to the memory and configured to receive an input of a plurality of series for learning having known accuracy, and learn a model represented by a neural network, the model being capable of determining accuracy levels of two series when given feature amounts of the two series among the plurality of series.

## Claims

### Claim 1

A learning device comprising:
a memory; and
processing circuitry coupled to the memory and configured to:
receive an input of a plurality of series for learning having known accuracy, and
learn a model represented by a neural network, the model determining accuracy levels of two series of the plurality of series based on a one-to-one comparison between the two series when given feature amounts of the two series among the plurality of series, one of the two series of the plurality of series being a series estimated to have a highest accuracy of the plurality of series after repeated one-to-one comparisons among the plurality of series.

### Claim 2 (depends on 1)

The learning device according to claim 1, wherein the model converts two hypotheses into hidden state vectors using a recurrent neural network, and outputs a first posterior probability indicating that sequence of the accuracy levels of the two series is correct and a second posterior probability indicating that the sequence of the accuracy levels of the two series is incorrect on a basis of the hidden state vectors using a neural network.

### Claim 3 (depends on 1)

The learning device according to claim 1, wherein the processing circuitry is further configured to assign a correct label to the model to be learned when a series having higher accuracy among the two series is placed in a higher rank than another series, and assign an error label to the model to be learned when the series having the higher accuracy among the two series is placed in a lower rank than an other series.

### Claim 4 (depends on 3)

The learning device according to claim 3, wherein the processing circuitry is further configured to:
receive an input of N-best hypotheses for learning having known speech recognition accuracy, and
assign a correct label to a model to be learned when a hypothesis having higher speech recognition accuracy among two hypotheses of the N-best hypotheses is placed in a higher rank than another hypothesis, and assign the error label to the model to be learned when the hypothesis having the higher speech recognition accuracy among the two hypotheses is placed in a lower rank than an other hypotheses.

### Claim 5 (depends on 4)

The learning device according to claim 4, wherein one hypothesis among the two hypotheses is an Oracle hypothesis having highest speech recognition accuracy.

### Claim 6 (depends on 5)

The learning device according to claim 5, wherein another hypothesis among the two hypotheses includes at least any of a first hypothesis having second-highest speech recognition accuracy after the Oracle hypothesis, a second hypothesis having a highest speech recognition score in the N-best hypotheses, a third hypothesis having lowest speech recognition accuracy, and a fourth hypothesis having a lowest speech recognition score in the N-best hypotheses.

### Claim 7 (depends on 6)

The learning device according to claim 6, wherein the other hypothesis among the two hypotheses includes a prescribed number of hypotheses and the first to the fourth hypotheses, the prescribed number of hypotheses being extracted according to a prescribed rule from hypotheses excluding the Oracle hypothesis, the first hypothesis, the second hypothesis, the third hypothesis, and the fourth hypothesis from the N-best hypotheses.

### Claim 8

A learning method that is performed by a learning device, the learning method comprising:
receiving an input of a plurality of series for learning having known accuracy; and
learning a model represented by a neural network, a model determining accuracy levels of two series of the plurality of series based on a one-to-one comparison between the two series when given feature amounts of the two series among the plurality of series, one of the two series of the plurality of series being a series estimated to have a highest accuracy of the plurality of series after repeated one-to-one comparisons among the plurality of series.

### Claim 9

A non-transitory computer-readable recording medium storing therein a learning program that causes a computer to execute a process comprising:
receiving an input of a plurality of series for learning having known accuracy; and
learning a model represented by a neural network, the model determining accuracy levels of two series of the plurality of series based on a one-to-one comparison between two series when given feature amounts of the two series among the plurality of series, one of the two series of the plurality of series being a series estimated to have a highest accuracy of the plurality of series after repeated one-to-one comparisons among the plurality of series.

### Claim 10 (depends on 1)

The learning device according to claim 1, wherein the model is a recurrent neural network.

### Claim 11 (depends on 10)

The learning device according to claim 10, wherein the processing circuitry
maintains a series of the two of series having a highest accuracy of the two series and selects another series from a plurality of series to form a pair of another of series, and
determines an accuracy level of each series in a pair of an other two.

### Claim 12 (depends on 8)

The method according to claim 8, wherein the model is a recurrent neural network.

### Claim 13 (depends on 12)

The method according to claim 12, further comprising:
maintaining a series of the two of series having a highest accuracy of the two series and selects another series from a plurality of series to form a pair of another of series; and
determining an accuracy level of each series in the pair of an other two series.

### Claim 14 (depends on 9)

The non-transitory computer-readable medium according to claim 9, wherein the model is a recurrent neural network.

### Claim 15 (depends on 14)

The non-transitory computer-readable medium according to claim 14, further comprising:
maintaining a series of the two of series having a highest accuracy of the two series and selects another series from a plurality of series to form a pair of another of series; and
determining an accuracy level of each series in the pair of an other two series.