# US 12,008,141 B2 · Privacy preserving synthetic string generation using recurrent neural networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,141 B2 - Document ID: US12008141B2 - Status: Granted - Kind: B2 - Application number: 16/836,528 - Filed: 2020-03-31 - Published: 2021-09-30 - Granted: 2024-06-11 - Earliest claimed: 2020-03-31 - Assignees: Intuit Inc. - Applicants: Intuit Inc. - Inventors: Liron Hayman; Shlomi Medalion - Web page: https://patentgrep.com/patent/US12008141B2 ## CPC - G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification} - G06F 18/24143: {Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]} - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/08: Learning methods - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method for privacy preserving synthetic string generation using recurrent neural networks includes receiving input data that includes a plurality of strings with private information. A neural network model is trained using the plurality of strings. The neural network model includes a recurrent neural network (RNN). An anonymous string is generated with the neural network model after training the neural network model with the plurality of strings from the input data. The anonymous string is validated to preclude the private information from the anonymous string. Anonymous data is transmitted that includes the anonymous string and precludes the private information in response to a request for the anonymous data. ## Claims ### Claim 1 A method comprising: obtaining, from a physical storage device, training data comprising a plurality of training strings comprising private information; training a recurrent neural network (RNN) model with the plurality of training strings to predict a next character in a training string, to obtain a trained RNN model, wherein training the RNN model comprises: applying an embedded layer to the training data to generate an embedded layer output, wherein the embedded layer receives as input an encoded vector that represents a character from a string from the training data, wherein the embedded layer output is an embedded vector that includes information about the character within a threshold number of characters of the character represented by the encoded vector, applying a plurality of RNN layers to the embedded layer output to generate a recurrent layer output, wherein the recurrent layer output predicts a next character from a sequence of input characters encoded with the embedded layer, applying a fully connected layer to the recurrent layer output to generate a fully connected layer output, wherein the fully connected layer output predicts the next character in the string from the training data that is inputted to the RNN, and comparing the fully connected layer output to an expected output using a loss function that identifies a difference between an output of the RNN model and the expected output, wherein the difference is then fed back into the plurality of RNN layers of the RNN model to update weights of the RNN model; receiving a request to generate an anonymous string from a plurality of input strings comprising private information; selecting a prime character randomly based on a frequency of the prime character as a starting character in the plurality of input strings; processing, by the trained RNN model, the selected prime character to generate the anonymous string, wherein the prime character is used as the starting character for each of the plurality of input strings, wherein the prime character is a seed character that starts a generation of an output string with the RNN model; generating a list of invalid terms from the private information corresponding to the plurality of input strings; comparing at least one string term from the anonymous string to the list of invalid terms; validating the anonymous string responsive to the anonymous string precluding the list of invalid terms to obtain a validated anonymous string; and transmitting the validated anonymous string in response to the request for the anonymous string. ### Claim 2 (depends on 1) The method of claim 1, further comprising: training the RNN model, wherein the RNN model includes a long short term memory (LSTM) layer. ### Claim 3 (depends on 1) The method of claim 1, further comprising: training the RNN model by comparing a label character from an input string of the plurality of training strings to an output character generated by the RNN model from a set of characters from the input string, wherein the set of characters precede the label character in the input string, and wherein a number of characters in the set of characters is at least three. ### Claim 4 (depends on 1) The method of claim 1, further comprising: generating the list of invalid terms to exclude common terms. ### Claim 5 (depends on 1) The method of claim 1, further comprising: receiving the training data, wherein the plurality of training strings comprises a transaction string that comprises a transaction record that includes a date, a payee, location information, and a payment amount, and wherein the private information includes one or more of the payee, the location information, and the payment amount. ### Claim 6 A system comprising: a processor; a memory coupled to the processor; a physical storage device; the memory comprising an application, wherein the application executes on the processor, uses the memory, and is configured for: obtaining, from the physical storage device, training data comprising a plurality of training strings comprising private information; training a recurrent neural network (RNN) model with the plurality of training strings to predict a next character in a training string, to obtain a trained RNN model, wherein training the RNN model comprises: applying an embedded layer to the training data to generate an embedded layer output, wherein the embedded layer receives as input an encoded vector that represents a character from a string from the training data, wherein the embedded layer output is an embedded vector that includes information about the character within a threshold number of characters of the character represented by the encoded vector, applying a plurality of RNN layers to the embedded layer output to generate a recurrent layer output, wherein the recurrent layer output predicts a next character from a sequence of input characters encoded with the embedded layer, applying a fully connected layer to the recurrent layer output to generate a fully connected layer output, wherein the fully connected layer output predicts the next character in the string from the training data that is inputted to the RNN, and comparing the fully connected layer output to an expected output using a loss function that identifies a difference between an output of the RNN model and the expected output, wherein the difference is then fed back into the plurality of RNN layers of the RNN model to update weights of the RNN model; receiving a request to generate an anonymous string from a plurality of input strings comprising private information; selecting a prime character randomly based on a frequency of the prime character as a starting character in the plurality of input strings; processing, by the trained RNN model, the selected prime character to generate the anonymous string, wherein the prime character is used as the starting character for each of the plurality of input strings, wherein the prime character is a seed character that starts a generation of an output string with the RNN model; generating a list of invalid terms from the private information corresponding to the plurality of input strings; comparing at least one string term from the anonymous string to the list of invalid terms; validating the anonymous string responsive to the anonymous string precluding the list of invalid terms to obtain a validated anonymous string; and transmitting the validated anonymous string in response to the request for the anonymous string. ### Claim 7 (depends on 6) The system of claim 6, wherein the application is further configured for: training the RNN model, wherein the RNN model includes a long short term memory (LSTM) layer. ### Claim 8 (depends on 6) The system of claim 6, wherein the application is further configured for: training the RNN model by comparing a label character from an input string of the plurality of training strings to an output character generated by the RNN model from a set of characters from the input string, wherein the set of characters precede the label character in the input string, and wherein a number of characters in the set of characters is at least three. ### Claim 9 (depends on 6) The system of claim 6, wherein the application is further configured for: generating the list of invalid terms to exclude common terms. ### Claim 10 (depends on 6) The system of claim 6, wherein the application is further configured for: receiving the training data, wherein the plurality of training strings comprises a transaction string that comprises a transaction record that includes a date, a payee, location information, and a payment amount, and wherein the private information includes one or more of the payee, the location information, and the payment amount. ### Claim 11 A set of one or more non-transitory computer readable mediums comprising computer readable program code for: obtaining, from a physical storage device, training data comprising a plurality of training strings comprising private information; training a recurrent neural network (RNN) model with the plurality of training strings to predict a next character in a training string, to obtain a trained RNN model, wherein training the RNN model comprises: applying an embedded layer to the training data to generate an embedded layer output, wherein the embedded layer receives as input an encoded vector that represents a character from a string from the training data, wherein the embedded layer output is an embedded vector that includes information about the character within a threshold number of characters of the character represented by the encoded vector, applying a plurality of RNN layers to the embedded layer output to generate a recurrent layer output, wherein the recurrent layer output predicts a next character from a sequence of input characters encoded with the embedded layer, applying a fully connected layer to the recurrent layer output to generate a fully connected layer output, wherein the fully connected layer output predicts the next character in the string from the training data that is inputted to the RNN, and comparing the fully connected layer output to an expected output using a loss function that identifies a difference between the an output of the RNN model and the expected output, wherein the difference is then fed back into the plurality of RNN layers of the RNN model to update weights of the RNN model; receiving a request to generate an anonymous string from a plurality of input strings comprising private information; selecting a prime character randomly based on a frequency of the prime character as a starting character in the plurality of input strings; processing, by the trained RNN model, the selected prime character to generate the anonymous string, wherein the prime character is used as the starting character for each of the plurality of input strings, wherein the prime character is a seed character that starts a generation of an output string with the RNN model; generating a list of invalid terms from the private information corresponding to the plurality of input strings; comparing at least one string term from the anonymous string to the list of invalid terms; validating the anonymous string responsive to the anonymous string precluding the list of invalid terms to obtain a validated anonymous string; and transmitting the validated anonymous string in response to the request for the anonymous string. ### Claim 12 (depends on 11) The set of one or more non-transitory computer readable mediums of claim 11, further comprising computer readable program code for: training the RNN model, wherein the RNN model includes a long short term memory (LSTM) layer.