# US 10,872,610 B2 · Generating random pass-phrases using word-level recurrent neural networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,610 B2 - Document ID: US10872610B2 - Status: Granted - Kind: B2 - Application number: 16/264,303 - Filed: 2019-01-31 - Published: 2020-08-06 - Granted: 2020-12-22 - Earliest claimed: 2019-01-31 - Assignees: RSA Security Inc. - Applicants: RSA Security LLC - Inventors: Naveen Sunkavally - Web page: https://patentgrep.com/patent/US10872610B2 ## CPC - G06F 40/30: Semantic analysis - G10L 17/24: the user being prompted to utter a password or a predefined phrase - G06F 21/46: by designing passwords or checking the strength of passwords - G06F 40/279: Recognition of textual entities - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G10L 17/04: Training, enrolment or model building - G10L 17/18: Artificial neural networks; Connectionist approaches - H04L 63/0838: {using one-time-passwords} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Systems and methods are provided for generating random pass-phrases using word-level recurrent neural networks (RNNs). A pass-phrase includes a random sequence of words selected from a text corpus used to train an RNN model. The pass-phrase generation process utilizes a seed phrase obtained from the training text corpus, and a random bit string. The seed phrase is processed by the RNN model to generate a set of predicted words and associated likelihood values that the predicted words are a next word following the seed phrase. The prediction results are encoded into a binary tree which is traversed using a portion of the random bit string to identify a word at a leaf node which matches the portion of the random bit string. The identified word is selected as a constituent of the random pass-phrase, and the process is repeated until the random bit stream is exhausted. ## Claims ### Claim 1 A method, comprising: receiving a request for a random pass-phrase for use as an authentication factor for a registered user to access an information system; performing an iterative pass-phrase generation process to generate the random pass-phrase in response to the received request, wherein the iterative pass-phrase generation process comprises at least an first iteration which comprises: obtaining an initial seed phrase comprising at least a portion of a text sentence from a text corpus of training data associated with a word-level recurrent neural network (RNN) model trained using the text corpus of training data; obtaining a random bit string having a specified bit-length; inputting the initial seed phrase to the input of the word-level RNN model to obtain prediction results, wherein the prediction results comprises a plurality of predicted words from the text corpus and associated likelihood values, wherein the likelihood value for a given predicted word indicates a likelihood that the given predicted word is a next word following the initial seed phrase; generating a binary tree data structure representing the prediction results, wherein leaf nodes of the binary tree data structure each correspond to one of the predicted words; traversing the binary tree data structure using a portion of the random bit string to identify a word at a leaf node of the binary tree data structure which matches the portion of the random bit string; and selecting the identified word as a first constituent word of the random pass-phrase. ### Claim 2 (depends on 1) The method of claim 1, further comprising: generating a truncated random bit string by removing the portion of the random bit string used to traverse the binary tree data structure to identify the matching word for the first iteration of the pass-phrase generation process; and responsive to determining that the random bit string has not been exhausted following the removal of the portion of the random bit string, performing an additional iteration of the pass-phrase generation process, wherein the additional iteration comprises: modifying the initial seed phrase by appending a previously selected constituent word of the random pass-phrase to an end of the initial seed phrase and removing a first word of the initial seed phrase; inputting the modified seed phrase to the input of the word-level RNN model to obtain prediction results, wherein the prediction results comprises a plurality of predicted words from the text corpus and associated likelihood values, wherein the likelihood value for a given predicted word indicates a likelihood that the given predicted word is a next word following the modified seed phrase; generating a binary tree data structure representing the prediction results obtained using the modified seed phrase, wherein leaf nodes of the binary tree data structure each correspond to one of the predicted words; traversing the binary tree data structure using a portion of the truncated random bit string to identify a word at a leaf node of the binary tree data structure which matches the portion of the random bit string; and selecting the identified word as a next constituent word of the random pass-phrase. ### Claim 3 (depends on 2) The method of claim 2, comprising: terminating the pass-phrase generation process responsive to determining that the random bit string has been exhausted; and outputting a random pass-phrase comprising a sequence of the constituent words selected in all iterations of the pass-phrase generation process. ### Claim 4 (depends on 1) The method of claim 1, wherein generating the binary tree data structure representing the prediction results comprises performing a Huffman encoding process to generate a Huffman tree data structure. ### Claim 5 (depends on 1) The method of claim 1, wherein traversing the binary tree data structure comprises performing a root node-to-leaf node path traversal based on a sequence of bit values at a beginning portion of the random bit string. ### Claim 6 (depends on 1) The method of claim 1, wherein obtaining the random bit string having a specified bit-length comprises: receiving a passcode from one of a (i) security token device and (ii) a soft security token; and utilizing a bit stream representation of the received passcode as the random bit string. ### Claim 7 (depends on 1) The method of claim 1, further comprising pre-processing the prediction results prior to generating the binary tree data structure representing the prediction results, wherein pre-processing comprises normalizing the likelihood values of the predicted words. ### Claim 8 (depends on 7) The method of claim 7, wherein pre-processing comprising applying a diversity factor value as part of the normalizing process, wherein the diversity factor value is applied to achieve a desired level of user memorability of the random pass-phrase to be generated. ### Claim 9 (depends on 7) The method of claim 7, wherein pre-processing comprises setting a likelihood value to zero for a predicted word that comprises a stop character. ### Claim 10 (depends on 1) The method of claim 1, wherein an iterative pass-phrase generation process is performed by an authentication server of the information system. ### Claim 11 (depends on 1) The method of claim 1, wherein the iterative pass-phrase generation process is executed on a computing device of the registered user. ### Claim 12 (depends on 1) The method of claim 1, wherein the text corpus of training data is obtained by pre-processing a raw text corpus to generate the text corpus of training data, wherein pre-processing the raw text corpus comprises: converting uppercase text characters in the raw text corpus to lower-case characters; and removing punctuation characters from the raw text except for stop characters; separating the stop characters from text words preceding the stop characters to make the stop characters appear as independent words in the raw text corpus. ### Claim 13 (depends on 12) The method of claim 12, further comprising training the word-level RNN model using the text corpus of training generated as a result of the pre-processing of the raw text corpus, wherein training the word-level RNN model comprising training a multi-layer RNN model comprising a plurality of activation layers comprising one of (i) long short-term memory (LSTM) units and (ii) gated recurrent units (GRUs). ### Claim 14 An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code is executable by one or more processors to perform a method which comprises: receiving a request for a random pass-phrase for use as an authentication factor for a registered user to access an information system; performing an iterative pass-phrase generation process to generate the random pass-phrase in response to the received request, wherein the iterative pass-phrase generation process comprises at least an first iteration which comprises: obtaining an initial seed phrase comprising at least a portion of a text sentence from a text corpus of training data associated with a word-level recurrent neural network (RNN) model trained using the text corpus of training data; obtaining a random bit string having a specified bit-length; inputting the initial seed phrase to the input of the word-level RNN model to obtain prediction results, wherein the prediction results comprises a plurality of predicted words from the text corpus and associated likelihood values, wherein the likelihood value for a given predicted word indicates a likelihood that the given predicted word is a next word following the initial seed phrase; generating a binary tree data structure representing the prediction results, wherein leaf nodes of the binary tree data structure each correspond to one of the predicted words; traversing the binary tree data structure using a portion of the random bit string to identify a word at a leaf node of the binary tree data structure which matches the portion of the random bit string; and selecting the identified word as a first constituent word of the random pass-phrase. ### Claim 15 (depends on 14) The article of manufacture of claim 14, wherein the program code is executable by the one or more processors to perform a method which further comprises: generating a truncated random bit string by removing the portion of the random bit string used to traverse the binary tree data structure to identify the matching word for the first iteration of the pass-phrase generation process; and responsive to determining that the random bit string has not been exhausted following the removal of the portion of the random bit string, performing an additional iteration of the pass-phrase generation process, wherein the additional iteration comprises: modifying the initial seed phrase by appending a previously selected constituent word of the random pass-phrase to an end of the initial seed phrase and removing a first word of the initial seed phrase; inputting the modified seed phrase to the input of the word-level RNN model to obtain prediction results, wherein the prediction results comprises a plurality of predicted words from the text corpus and associated likelihood values, wherein the likelihood value for a given predicted word indicates a likelihood that the given predicted word is a next word following the modified seed phrase; generating a binary tree data structure representing the prediction results obtained using the modified seed phrase, wherein leaf nodes of the binary tree data structure each correspond to one of the predicted words; traversing the binary tree data structure using a portion of the truncated random bit string to identify a word at a leaf node of the binary tree data structure which matches the portion of the random bit string; and selecting the identified word as a next constituent word of the random pass-phrase. ### Claim 16 (depends on 15) The article of manufacture of claim 15, wherein the program code is executable by the one or more processors to perform a method which further comprises: terminating the pass-phrase generation process responsive to determining that the random bit string has been exhausted; and outputting a random pass-phrase comprising a sequence of the constituent words selected in all iterations of the pass-phrase generation process. ### Claim 17 (depends on 14) The article of manufacture of claim 14, wherein the program code is executable by the one or more processors to perform a method which further comprises pre-processing the prediction results prior to generating the binary tree data structure representing the prediction results, wherein pre-processing comprises: setting a likelihood value to zero for each predicted word that comprises a stop character; normalizing the likelihood values of the predicted words; and applying a diversity factor value as part of the normalizing process, wherein the diversity factor value is applied to achieve a desired level of user memorability of the random pass-phrase to be generated. ### Claim 18 A system, comprising: memory to store software instructions; and one or more processors to execute the software instructions to perform a method which comprises: receiving a request for a random pass-phrase for use as an authentication factor for a registered user to access an information system; performing an iterative pass-phrase generation process to generate the random pass-phrase in response to the received request, wherein the iterative pass-phrase generation process comprises at least an first iteration which comprises: obtaining an initial seed phrase comprising at least a portion of a text sentence from a text corpus of training data associated with a word-level recurrent neural network (RNN) model trained using the text corpus of training data; obtaining a random bit string having a specified bit-length; inputting the initial seed phrase to the input of the word-level RNN model to obtain prediction results, wherein the prediction results comprises a plurality of predicted words from the text corpus and associated likelihood values, wherein the likelihood value for a given predicted word indicates a likelihood that the given predicted word is a next word following the initial seed phrase; generating a binary tree data structure representing the prediction results, wherein leaf nodes of the binary tree data structure each correspond to one of the predicted words; traversing the binary tree data structure using a portion of the random bit string to identify a word at a leaf node of the binary tree data structure which matches the portion of the random bit string; and selecting the identified word as a first constituent word of the random pass-phrase. ### Claim 19 (depends on 18) The system of claim 18, wherein the one or more processors execute the software instructions to perform a method which further comprises: generating a truncated random bit string by removing the portion of the random bit string used to traverse the binary tree data structure to identify the matching word for the first iteration of the pass-phrase generation process; and responsive to determining that the random bit string has not been exhausted following the removal of the portion of the random bit string, performing an additional iteration of the pass-phrase generation process, wherein the additional iteration comprises: modifying the initial seed phrase by appending a previously selected constituent word of the random pass-phrase to an end of the initial seed phrase and removing a first word of the initial seed phrase; inputting the modified seed phrase to the input of the word-level RNN model to obtain prediction results, wherein the prediction results comprises a plurality of predicted words from the text corpus and associated likelihood values, wherein the likelihood value for a given predicted word indicates a likelihood that the given predicted word is a next word following the modified seed phrase; generating a binary tree data structure representing the prediction results obtained using the modified seed phrase, wherein leaf nodes of the binary tree data structure each correspond to one of the predicted words; traversing the binary tree data structure using a portion of the truncated random bit string to identify a word at a leaf node of the binary tree data structure which matches the portion of the random bit string; and selecting the identified word as a next constituent word of the random pass-phrase. ### Claim 20 (depends on 18) The system of claim 18, wherein the system comprises one of (i) a computing device of the registered user and (ii) an authentication server of the information system.