# US 10,872,203 B2 · Data input system using trained keypress encoder > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,203 B2 - Document ID: US10872203B2 - Status: Granted - Kind: B2 - Application number: 15/355,685 - Filed: 2016-11-18 - Published: 2018-05-24 - Granted: 2020-12-22 - Earliest claimed: 2016-11-18 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Douglas Alexander Harper Orr; Juha Iso-Sipilä; Marco Fiscato; Matthew James Willson; Joseph Osborne - Web page: https://patentgrep.com/patent/US10872203B2 ## CPC - G06F 40/274: Converting codes to words; Guess-ahead of partial word inputs - G06F 3/0237: {using prediction or retrieval techniques} - G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus - G06F 40/232: Orthographic correction, e.g. spell checking or vowelisation - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A data input system is described of the type which has a virtual keyboard which enables a user to type a text sequence into a computing device. The data input system has an input probability generator which is configured to compute keypress evidence. The keypress evidence comprises probabilities that user input events at the virtual keyboard correspond to characters or keyboard functions. The data input system has a trained keypress encoder, having been trained using keypress evidence and corresponding words. The trained keypress encoder encodes the keypress evidence into a numerical encoding. The data input system has a completion/correction predictor which is configured to take as input, the numerical encoding and an encoding of one or more text items of the text sequence already input to the computing device, in order to predict a text item in the text sequence. ## Claims ### Claim 1 A data input system comprising a processor and a memory communicatively coupled to the processor, the memory storing thereon computer-readable instructions that, when executed by the processor, cause the data input system to perform operations comprising: instantiating a virtual keyboard configured to receive a text sequence for entry into the data input system, the text sequence including at least one context word and at least one partial word; identifying keypress evidence for one or more characters in the at least one partial word of the text sequence, the keypress evidence comprising one or more probabilities that user input events at the virtual keyboard correspond to characters or keyboard functions; invoking a trained keypress encoder, wherein the trained keypress encoder is trained using previous keypress evidence and corresponding words, the trained keypress encoder configured to encode the keypress evidence into a numerical encoding, wherein a portion of the numerical encoding corresponds to the at least one partial word; generating, by a trained context encoder, an embedding of one or more candidate text items using a portion of the numerical encoding corresponding to the at least one context word of the text sequence; and determining a predicted text item corresponding to the at least one partial word, using a neural network, based at least in part on inputs to the neural network including a concatenated vector formed from the portion of the numerical encoding corresponding to the at least one partial word and the embedding of the one or more candidate text items. ### Claim 2 (depends on 1) The data input system of claim 1, wherein the trained keypress encoder is trained using data associated with a single user such that the keypress encoder is personalized to the single user. ### Claim 3 (depends on 1) The data input system of claim 1, wherein the trained keypress encoder is trained using data associated with a single type of virtual keyboard such that the keypress encoder is customized to the single type of virtual keyboard. ### Claim 4 (depends on 1) The data input system of claim 1, wherein the trained keypress encoder is trained using data obtained from a single type of electronic device such that the keypress encoder is customized to the single type of electronic device. ### Claim 5 (depends on 1) The data input system of claim 1, wherein the predicted text item one or more of: words, emoji, pictograms, and stickers. ### Claim 6 (depends on 1) The data input system of claim 1, wherein the keypress evidence is computed using a target model set comprising a learned probability distribution for each of a plurality of characters. ### Claim 7 (depends on 1) The data input system of claim 1, wherein the keypress evidence is computed using coordinates of user input events received at the virtual keyboard. ### Claim 8 (depends on 1) The data input system of claim 1, wherein the trained keypress encoder comprises a second neural network. ### Claim 9 (depends on 1) The data input system of claim 1, wherein the trained keypress encoder comprises a recurrent neural network. ### Claim 10 (depends on 1) The data input system of claim 1, wherein the trained keypress encoder comprises a convolutional neural network. ### Claim 11 (depends on 1) The data input system of claim 1, wherein the predicted text item is determined using the neural network trained to predict a word given historical data including numerical encodings of key press evidence and numerical encodings of context words and partial words, the historical data comprising the one or more text items of the text sequence already input to the data input system. ### Claim 12 (depends on 1) The data input system of claim 1, wherein the predicted text item is determined using a third neural network. ### Claim 13 (depends on 1) The data input system of claim 1, further comprising computer-readable instructions that, when executed by the processor, cause the data input system to perform operations comprising rendering the predicted item in the text sequence at the virtual keyboard as a proposed completion or a correction of a text item which a user is trying to input. ### Claim 14 A computer-implemented data input method; the method comprising: determining, by a computing device, user input events based on a text sequence input received via a virtual keyboard, the text sequence including at least one context word and at least one partial word; identifying keypress evidence for one or more characters in the at least one partial word of the text sequence, the keypress evidence comprising one or more probabilities that a plurality of the user input events correspond to each of a plurality of possible characters or keyboard functions; encoding the keypress evidence into a numerical encoding using a trained keypress encoder, the trained keypress encoder trained using previous keypress evidence and corresponding words, wherein a portion of the numerical encoding corresponds to the at least one partial word; generating, by a trained context encoder, an embedding of one or more candidate text items using a portion of the numerical encoding corresponding to the at least one context word of the text sequence; and computing a text item in the text sequence corresponding to the at least one partial word, using a neural network, based on inputs to the neural network including a concatenated vector formed from the portion of the numerical encoding corresponding to the at least one partial word and the embedding of the one or more candidate text items. ### Claim 15 (depends on 14) The method of claim 14, further comprising training the trained keypress encoder using the computed keypress evidence and corresponding words. ### Claim 16 (depends on 14) The method of claim 14, further comprising training the trained keypress encoder using training data determined based on data input behavior of a single user. ### Claim 17 (depends on 14) The method of claim 14, further comprising training the trained keypress encoder using training data determined based on data input behavior at a single type of virtual keyboard. ### Claim 18 (depends on 14) The method of claim 14, further comprising training a completion/correction predictor and a context encoder together with the keypress encoder, wherein the completion/correction predictor is configured to computed the keypress evidence and the context encoder is configured to encode the keypress evidence. ### Claim 19 (depends on 14) The method of claim 14, further comprising computing a measure of similarity of the computed text item with other text items. ### Claim 20 A computer-readable storage medium storing thereon computer-executable instructions that, when executed by a computing system, cause the computing system to perform operations comprising: receiving, via a virtual keyboard, user input events indicative of a text sequence; computing keypress evidence comprising one or more probabilities that the user input events correspond to characters or keyboard functions; encoding the keypress evidence into a numerical encoding using a trained keypress encoder comprising a neural network, wherein a portion of the numerical encoding corresponds to the at least one partial word; and computing a text item in the text sequence corresponding to the at least one partial word, using a second neural network, based on inputs to the neural network including a concatenated vector formed from the portion of the numerical encoding corresponding to the at least one partial word and an encoding of one or more text items of the text sequence already input to the computing system.