# US 2014/0195238 A1 · Method and apparatus of confidence measure calculation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0195238 A1 - Document ID: US20140195238A1 - Status: Published application - Kind: A1 - Application number: 14/130,335 - Filed: 2012-03-01 - Published: 2014-07-10 - Earliest claimed: 2011-07-01 - Assignees: University of Washington through its Center for Commercialization; NEC Corporation - Inventors: Makoto Terao; Mari Ostendorf - Web page: https://patentgrep.com/patent/US20140195238A1 ## CPC - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 15/063: {Training} - G10L 15/10: using distance or distortion measures between unknown speech and reference templates - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract An apparatus that calculates a confidence measure of a target word string specified in a recognition result includes: an alternative candidate generator which generates an alternative candidate word string in the position of the target word string; a classifier training unit which trains a classifier which is configured to discriminate between the target word string and the alternative candidate word string; a feature extractor which extracts a feature value representing an adjacent context in the position of the target word string; and a confidence measure calculator which determining whether the true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value and calculates a confidence measure of the target word string on the basis of the determination result. ## Claims ### Claim 1 A confidence measure calculation method of calculating a confidence measure of a target word string specified in a recognition result, the method comprising: receiving the recognition result and the target word string; generating an alternative candidate word string in a position of the target word string; training a classifier which is configured to discriminate between the target word string and the alternative candidate word string; extracting a feature value representing an adjacent context in the position of the target word string; determining whether a true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value; and calculating a confidence measure of the target word string on the basis of the determination result. ### Claim 2 (depends on 1) The method according to claim 1, wherein a plurality of the alternative candidate word strings are generated, the classifier is configured to discriminate between the target word string and any of the plurality of alternative candidate word strings, and whether the true word string in the position of the target word string is the target word string or any of the plurality of alternative candidate word strings is determined by using the classifier. ### Claim 3 (depends on 1) The method according to claim 1, wherein the classifier is configured to output a posterior probability of classification result, and the posterior probability that the target word string is correct is used as the confidence measure of the target word string. ### Claim 4 (depends on 1) The method according to claim 1, wherein, when the true word string in the position of the target word string is classified as the target word string, a first constant is used as the confidence measure of the target word string, whereas, when the true word string in the position of the target word string is classified as the alternative candidate word string, a second constant smaller than the first constant is used as the confidence measure of the target word string. ### Claim 5 (depends on 1) The method according to claim 1, wherein a hypothesis which is outputted by a recognition system in the position of the target word string and is other than the target word string is used as the alternative candidate word string. ### Claim 6 (depends on 1) The method according to claim 1, wherein a word string having pronunciation which is similar to pronunciation of the target word string is generated as the alternative candidate word string by referring to a pronunciation dictionary. ### Claim 7 (depends on 1) The apparatus according to claim 1, wherein a word string that a recognition system tends to confuse with the target word string is generated as the alternative candidate word string by referring to an inter-word confusion matrix. ### Claim 8 (depends on 1) The apparatus according to claim 1, wherein the feature value representing the adjacent context in the position of the target word string is calculated on the basis of frequencies of words appearing within a certain distance from the position of the target word string or frequencies of attributes of the words. ### Claim 9 (depends on 1) The method according to claim 1, further comprising: judging the target word string to be a recognition error when the calculated confidence measure of the target word string is smaller than or equal to a predetermined threshold. ### Claim 10 (depends on 1) The method according to claim 1, further comprising: judging the target word string to be a recognition error when a posterior probability of the target word string outputted from a recognition system is smaller than or equal to a first threshold and the calculated confidence measure of the target word string is smaller than or equal to a second threshold. ### Claim 11 (depends on 1) The method according to claim 1, further comprising: carrying out the generating, the training, the extracting and the determining to obtain a confidence measure of the target word string and a confidence measure of the alternative candidate word string; and outputs a word string having a largest confidence measure. ### Claim 12 A confidence measure calculation apparatus that calculates a confidence measure of a target word string specified in a recognition result, the apparatus comprising: a generator generating an alternative candidate word string in a position of the target word string; a training unit training a classifier which is configured to discriminate between the target word string and the alternative candidate word string; an extractor extracting a feature value representing an adjacent context in the position of the target word string; and a calculator determining whether a true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value, and calculating a confidence measure of the target word string on the basis of the determination result. ### Claim 13 (depends on 12) The apparatus according to claim 12, wherein the generator generates a plurality of the alternative candidate word strings, the classifier is configured to discriminate between the target word string and any of the plurality of alternative candidate word strings, and, upon determination the calculator determines whether the true word string in the position of the target word string is the target word string or any of the plurality of alternative candidate word strings. ### Claim 14 (depends on 12) The apparatus according to claim 12, wherein the classifier is configured to output a posterior probability of classification result, and the calculator uses the posterior probability that the target word string is correct as the confidence measure of the target word string. ### Claim 15 (depends on 12) The apparatus according to claim 12, wherein the calculator uses a first constant as the confidence measure of the target word string when the true word string in the position of the target word string is classified as the target word string, whereas the calculator uses a second constant smaller than the first constant as the confidence measure of the target word string when the true word string in the position of the target word string is classified as the alternative candidate word string. ### Claim 16 (depends on 12) The apparatus according to claim 12, wherein the generator uses, as the alternative candidate word string, a hypothesis which is outputted by a recognition system in the position of the target word string and is other than the target word string. ### Claim 17 (depends on 12) The apparatus according to claim 12, wherein the generator refers to a pronunciation dictionary and generates, as the alternative candidate word string, a word string having pronunciation of which is similar to pronunciation of the target word string. ### Claim 18 (depends on 12) The apparatus according to claim 12, wherein the generator refers to an inter-word confusion matrix to generate, as the alternative candidate word string, a word string that a recognition system tends to confuse with the target word string. ### Claim 19 (depends on 12) The apparatus according to claim 12, wherein the feature value representing the adjacent context in the position of the target word string is calculated on the basis of frequencies of words appearing within a certain distance from the position of the target word string or frequencies of attributes of the words. ### Claim 20 (depends on 12) The apparatus according to claim 12, wherein, when the calculated confidence measure of the target word string is smaller than or equal to a predetermined threshold, the target word string is judged to be a recognition error. ### Claim 21 (depends on 12) The apparatus according to claim 12, wherein, when a posterior probability of the target word string outputted from a recognition system is smaller than or equal to a first threshold and the calculated confidence measure of the target word string is smaller than or equal to a second threshold, the target word string is judged to be a recognition error. ### Claim 22 (depends on 12) The apparatus according to claim 12, wherein the apparatus calculates a confidence measure of the target word string and a confidence measure of the alternative candidate word string, and outputs a word string having a largest confidence measure. ### Claim 23 A confidence measure calculation apparatus that calculates a confidence measure of a target word string specified in a recognition result, the apparatus comprising: means for generating an alternative candidate word string in a position of the target word string; means for training a classifier which is configured to discriminate between the target word string and the alternative candidate word string; means for extracting a feature value representing an adjacent context in the position of the target word string; and means for determining whether a true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value to calculate a confidence measure of the target word string on the basis of the determination result. ### Claim 24 A non-transitory computer-readable recoding medium that stores a program which causes a computer to execute the processes of: receiving a recognition result and a target word string which is specified in the recognition result; generating an alternative candidate word string in a position of the target word string; training a classifier which is configured to discriminate between the target word string and the alternative candidate word string; extracting a feature value representing an adjacent context in the position of the target word string; determining whether a true word string in the position of the target word string is the target word string or the alternative candidate word string by using the classifier and the feature value; and calculating a confidence measure of the target word string on the basis of the determination result. ### Claim 25 (depends on 24) The recording medium according to claim 24, wherein the program causes the computer to generate a plurality of the alternative candidate word strings in the generating process, to train a classifier which is configured to discriminate between the target word string and any of the plurality of alternative candidate word strings in the training process, and determine whether the true word string in the position of the target word string is the target word string or any of the plurality of alternative candidate word strings by using the classifier in the determination process. ### Claim 26 (depends on 24) The recording medium according to claim 24, wherein the classifier is configured to output a posterior probability of classification result, and the program causes the computer to use the posterior probability that the target word string is correct as the confidence measure of the target word string in the calculation process. ### Claim 27 (depends on 24) The recording medium according to claim 24, wherein the program causes the computer, in the calculation process, to use a first constant as the confidence measure of the target word string when the true word string in the position of the target word string is classified as the target word string, whereas use a second constant smaller than the first constant as the confidence measure of the target word string when the true word string in the position of the target word string is classified as the alternative candidate word string. ### Claim 28 (depends on 24) The recording medium according to claim 24, wherein the program causes the computer to use, as the alternative candidate word string, a hypothesis being outputted by a recognition system in the position of the target word string and being other than the target word string. ### Claim 29 (depends on 24) The recording medium according to claim 24, wherein the program causes the computer to refer to a pronunciation dictionary and generate a word string having pronunciation which is similar to pronunciation of the target word string as the alternative candidate word string, in the generating process. ### Claim 30 (depends on 24) The recording medium according to claim 24, wherein the program causes the computer to refer to an inter-word confusion matrix and generate a word string that a recognition system tends to confuse with the target word string as the alternative candidate word string, in the generating process. ### Claim 31 (depends on 24) The recording medium according to claim 24, wherein the program causes the computer to calculate the feature value representing the adjacent context in the position of the target word string on the basis of frequencies of words appearing within a certain distance from the position of the target word string or frequencies of attributes of the words. ### Claim 32 (depends on 24) The recording medium according to claim 24, wherein the program further causes the computer to execute the process of judging the target word string to be a recognition error when the calculated confidence measure of the target word string is smaller than or equal to a predetermined threshold. ### Claim 33 (depends on 24) The recording medium according to claim 24, wherein the program further causes the computer to execute the process of judging the target word string to be a recognition error when a posterior probability of the target word string outputted from a recognition system is smaller than or equal to a first threshold and the calculated confidence measure of the target word string is smaller than or equal to a second threshold. ### Claim 34 (depends on 24) The recording medium according to claim 24, wherein the program further causes the computer to execute the processes of: executing the processes of the generating, the training, the extracting and the determining to obtain a confidence measure of the target word string and a confidence measure of the alternative candidate word string, and outputting a word string having a largest confidence measure.