# US 2021/0326525 A1 · Device and method for correcting context sensitive spelling error using masked language model > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0326525 A1 - Document ID: US20210326525A1 - Status: Published application - Kind: A1 - Application number: 17/218,909 - Filed: 2021-03-31 - Published: 2021-10-21 - Earliest claimed: 2020-04-16 - Assignees: Pusan National University Industry-University Cooperation Foundation; Hancom, Inc. - Applicants: Pusan National University Industry-University Cooperation Foundation; Hancom, Inc. - Inventors: Hyukchul KWON; Jung-Hun LEE; Minho KIM - Web page: https://patentgrep.com/patent/US20210326525A1 ## CPC - G06F 40/232: Orthographic correction, e.g. spell checking or vowelisation - G06F 40/279: Recognition of textual entities - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed is a device for correcting a context sensitive spelling error using a masked language model. The device includes an input unit for inputting a sentence for correction; a correction target word check unit for checking an input sentence in units of a word and searching for a context spelling error; a candidate editing distance selection unit for calculating an editing distance between a word to be corrected and a word dictionary to select a candidate word; a mask candidate generation unit for calculating a distance between an entire context around the word to be corrected and candidate words filtered by the candidate editing distance selection unit; and a correction word presentation unit for selecting a final correction word based on a distance calculation value. ## Claims ### Claim 1 A device for correcting a context sensitive spelling error using a masked language model, the device comprising: an input unit for inputting a sentence for correction; a correction target word check unit for checking an input sentence in units of a word and searching for a context spelling error; a candidate editing distance selection unit for calculating an editing distance between a word to be corrected and a word dictionary to select a candidate word; a mask candidate generation unit for calculating a distance between an entire context around the word to be corrected and candidate words filtered by the candidate editing distance selection unit; and a correction word presentation unit for selecting a final correction word based on a distance calculation value. ### Claim 2 (depends on 1) The device of claim 1, wherein the correction target word check unit comprises: a statistical candidate word set construction unit for searching for a 3-gram dictionary and searching for surrounding context words at a center word position and all appearing statistical candidate words to configure a statistical candidate word set; a context probability calculation unit for calculating a context probability of statistical candidate words of the statistical candidate word set construction unit; and an error word presence/absence determination unit for determining whether there is an error word based on whether an error check target word has a higher or lower context probability than that of the statistical candidate words in the candidate word set. ### Claim 3 (depends on 1) The device of claim 1, wherein the candidate editing distance selection unit calculates a preset editing distance for insertion, deletion, and exchange between words in the entire word dictionary using in the masked language model used for correcting context sensitive spelling errors and a word to be corrected to select the corresponding word from the entire word dictionary. ### Claim 4 (depends on 1) The device of claim 1, wherein the mask candidate generation unit comprises: an editing distance calculation unit for calculating an editing distance with the entire dictionary word of the masked language model based on the word to be corrected; a correction candidate word set construction unit for obtaining a correction candidate word that satisfies an editing distance set with the entire insertion word dictionary of the masked language model based on a central word; and a candidate word distance value calculation unit for replacing the word to be corrected with a mask to calculate a distance value between a context around the mask and the candidate words. ### Claim 5 (depends on 1) The device of claim 1, wherein, when a word selected by the candidate editing distance selection unit for an input sentence w1:n is C, the mask candidate generation unit selects C in which a context sensitive spelling error correction distance value dis(w1, w2, . . . , C, . . . , wn-1, wn) is the maximum, and C ^ = arg ⁢ ⁢ max C ⁢ dis ⁡ ( w 1 , w 2 , … ⁢ , C , … ⁢ , w n - 1 , w n ) is defined. ### Claim 6 (depends on 5) The device of claim 5, wherein in the input sentence, when a word wt to be corrected is masked, an entire word w1:n of the sentence is {w1, w2, . . . , , . . . , wn-1, wn} and a set C of words selected by the candidate editing distance selection unit is replaced by the mask, and a distance value between the context and the selected word is calculated. ### Claim 7 (depends on 1) The device of claim 1, wherein the correction word presentation unit determines a correction candidate word with a highest value among correction candidate words to a final correction word based on the calculated distance value, and presents the word as a substitute. ### Claim 8 A method of correcting a context sensitive spelling error using a masked language model, the method comprising: determining a spelling error correction target word by checking a sentence in units of a word; selecting a word through calculation of an editing distance between a word to be corrected and dictionary words in a language model to be a candidate word; calculating a distance between all selected words to replace a masking place and an entire surrounding context using a sentence masked with the word to be corrected in an input sentence; and presenting a final correction word based on ranked information. ### Claim 9 (depends on 8) The method of claim 8, wherein the calculating of a distance comprises calculating a distance value of the sentence through the masked language model when words selected through calculation of an editing distance with an entire context of a sentence including a word to be corrected are included in the sentence as correction candidates. ### Claim 10 (depends on 8) The method of claim 8, wherein the calculating of a distance comprises inputting a masked sentence to the masked language model to calculate a distance between the selected words and the context and to rank the selected words according to the distance. ### Claim 11 (depends on 8) The method of claim 8, wherein context sensitive spelling error correction of an entire sentence comprises checking errors from a first word to a last word of a sentence, obtaining a set of selected candidate words based on a preset editing distance for a word determined to have errors, calculating a distance value between the entire context and each candidate word to rank the words, and presenting a final correction word, and when the final correction word and the word to be corrected are the same, it is determined that there is no error, and when the final correction word and the word to be corrected are different, it is determined that there is an error and the correction word is replaced. ### Claim 12 (depends on 8) The method of claim 8, wherein in the calculating of a distance, a distance value to the context of each candidate word of entire candidate words is obtained as dis ⁡ ( w 1 , w 2 , … ⁢ , C , … ⁢ , w n - 1 , w n ) = MLM ( masksentence , candidate 1 : N ) = candidate ⁢ distance 1 : N by inputting the masking sentence and the N number of correction candidate words using an MLM function, and candidate distance is a probability value formed by each candidate extracted from the masked language model and the surrounding context, and a distance is calculated as a probability using a softmax function using when outputting multiple output values up to 1:N as a result, as in an output in the masked language model. ### Claim 13 (depends on 12) The method of claim 12, wherein the calculation in the masked language model is configured in a form of a vector representing all data in deep learning, and all input sentence words masksentence and candidate are converted into vectors for calculation and then input, and the masked language model calculates the result for the input sentence in a form of a vector using a parameter that calculates an optimal result learned in advance, and calculates (softmax function) the result as a probability in a final output to output a value in which the sum of the entire result is 1.