# US 12,111,751 B2 · Debugging tool for code generation neural language models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,111,751 B2 - Document ID: US12111751B2 - Status: Granted - Kind: B2 - Application number: 18/082,366 - Filed: 2022-12-15 - Published: 2024-03-28 - Granted: 2024-10-08 - Earliest claimed: 2022-09-20 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC. - Inventors: Colin Bruce Clement; David Alberto Nader Palacio; Neelakantan Sundaresan; Alexey Svyatkovskiy; Michele Tufano - Web page: https://patentgrep.com/patent/US12111751B2 ## CPC - G06F 8/33: Intelligent editors - G06F 11/3636: {by tracing the execution of the program} - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A debugging tool identifies the smallest subset of an input sequence or rationales that influenced a neural language model to generate an output sequence. The debugging tool uses the rationales to understand why the model made its predictions and in particular, the particular input tokens that had the most impact on the output sequence. In the case of erroneous output, the rationales are used to alter the input sequence to avoid the error or to tailor a new training dataset to retrain the model to improve its performance. ## Claims ### Claim 1 A system comprising: one or more processors; and a memory that stores one or more programs that are configured to be executed by the one or more processors, the one or more programs including instructions to perform actions that: obtain a source code snippet generated by a neural language model given an input sequence of source code, wherein the input sequence includes a plurality of input source code tokens, wherein the source code snippet includes a plurality of output source code tokens; locate an error in the source code snippet generated by the neural language model, wherein the error is associated with a select token in the source code snippet; generate a set of rationales for the select token in the source code snippet, wherein the set of rationales comprises a smallest subset of tokens of the input sequence of source code tokens that led to generation of the select token in the source code snippet generated by the neural language model; and identify a source of the error in the source code snippet generated by the neural language model to at least one token of the set of rationales associated with the select token in the source code snippet. ### Claim 2 (depends on 1) The system of claim 1, wherein the one or more programs include further instructions to perform actions that: alter the input sequence of source code based on the identified source of the error; and generate a new source code snippet from the neural language model given the altered input sequence of source code. ### Claim 3 (depends on 1) The system of claim 1, wherein the one or more programs including instructions to perform actions that: classify each of the tokens in the set of rationales with a corresponding token aggregation category; identify a token aggregation category of the source of the error; and retrain the neural language model with a training dataset of source code snippets associated with the token aggregation category. ### Claim 4 (depends on 3) The system of claim 3, wherein the token aggregation category includes non-terminal symbols of production rules of a grammar of a programming language of the source code snippet. ### Claim 5 (depends on 1) The system of claim 1, the one or more programs including instructions to perform actions that: classify each of the tokens in the set of rationales with a corresponding token scope category; and retrain the neural language model with a training dataset of source code snippets associated with the token scope category. ### Claim 6 (depends on 5) The system of claim 5, wherein the token scope category consists of method, package, field, and class. ### Claim 7 (depends on 1) The system of claim 1, wherein the neural language model includes a neural transformer model with attention configured to generate source code. ### Claim 8 (depends on 1) The system of claim 1, wherein the neural language model includes a recurrent neural network configured to generate source code. ### Claim 9 (depends on 1) The system of claim 1, generate a set of rationales for the select token in the source code snippet is based on a conditional probability of the selected token conditioned on only the source code tokens of the set of rationales having a highest probability. ### Claim 10 A computer-implemented method, comprising: obtaining in a source code editor an output source code snippet generated from a neural language model given an input source code snippet, wherein the output source code snippet includes an output sequence of source code tokens, wherein the input source code snippet includes an input sequence of tokens; selecting a select one of the source code tokens of the output source code snippet generated by the neural language model; computing a smallest subset of source code tokens of the input source code sequence that led to generation of the selected one of the source code tokens of the output source code snippet by the neural language model; and identifying in the source code editor, each of the source code tokens of the input source code sequence of the smallest subset. ### Claim 11 (depends on 10) The computer-implemented method of claim 10, wherein the smallest subset of source code tokens of the input source code sequence is based on a conditional probability of the selected token conditioned on only the source code tokens of the smallest subset having a highest probability. ### Claim 12 (depends on 11) The computer-implemented method of claim 11, wherein the conditional probability is generated by the neural language model. ### Claim 13 (depends on 10) The computer-implemented method of claim 10, further comprising: associating a token scope category with each of the source code tokens of the smallest subset; and retraining the neural language model with a training dataset comprising source code associated with the token scope. ### Claim 14 (depends on 10) The computer-implemented method of claim 10, further comprising: associating a token aggregation category with each of the source code tokens of the smallest subset; and retraining the neural language model with a training dataset comprising source code associated with the token aggregation structure. ### Claim 15 (depends on 10) The computer-implemented method of claim 10, wherein the neural language model includes a neural transformer model with attention or a recurrent neural network. ### Claim 16 One or more hardware storage devices having stored thereon computer executable instructions that are structured to be executable by one or more processors of a computing device to thereby cause the computing device to: obtain a source code snippet generated by a neural language model given an input sequence of natural language text, wherein the input sequence includes a plurality of input natural language tokens, wherein the source code snippet includes a plurality of source code tokens; select a select one of the source code tokens of the output source code snippet generated by the neural language model; generate a smallest subset of natural language tokens of the input sequence that led to generation of the selected one of the source code tokens of the output source code snippet by the neural language model; and identify in a user interface, each of the natural language tokens of the input source code sequence of the smallest subset. ### Claim 17 (depends on 16) The one or more hardware storage devices of claim 16, wherein the smallest set of natural language tokens of the input sequence is based on a conditional probability distribution over the select one of the source code tokens of the output source code snippet when conditioned on each natural language token of the smallest subset having a highest probability. ### Claim 18 (depends on 16) The one or more hardware storage devices of claim 16 having stored thereon computer executable instructions that are structured to be executable by one or more processors of a computing device to thereby cause the computing device to: associate a token scope category with each of the natural language tokens of the smallest subset; and retrain the neural language model with a training dataset containing natural language text associated with the token scope category. ### Claim 19 (depends on 16) The one or more hardware storage devices of claim 16 having stored thereon computer executable instructions that are structured to be executable by one or more processors of a computing device to thereby cause the computing device to: associate a token aggregation category with each of the natural language tokens of the smallest subset; and retrain the neural language model with a training dataset containing natural language text associated with the token aggregation category. ### Claim 20 (depends on 16) The one or more hardware storage devices of claim 16, wherein the neural language model is a neural transformer model with attention or a recurrent neural network.