# US 12,182,547 B2 · Systems and methods for code repository embedding for tagging and summarization tasks using attention on multiple code domains > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,547 B2 - Document ID: US12182547B2 - Status: Granted - Kind: B2 - Application number: 17/821,643 - Filed: 2022-08-23 - Published: 2023-03-02 - Granted: 2024-12-31 - Earliest claimed: 2021-08-24 - Assignees: JPMorgan Chase Bank, N.A. - Applicants: Jpmorgan Chase Bank, N.A. - Inventors: Shaltiel Eloul; Agathe Lhérondelle; Sean Moran; Fanny Silavong; Yash Satsangi - Web page: https://patentgrep.com/patent/US12182547B2 ## CPC - G06F 8/36: Software reuse - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/09: Supervised learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Systems and methods for code repository embedding using attention mechanism for tagging and summarization are disclosed. According to one embodiment, a method for code repository embedding may include: (1) extracting, by a computer program executed by an electronic device, docstring embeddings, code embeddings, and dependency embeddings from scripts in a repository (2) applying, by the computer program, a machine learning algorithm to each of the docstring embeddings, code embeddings, and dependency embeddings; (3) concatenating, by the computer program, outputs of the machine learning algorithm; (4) weighting, by the computer program, the concatenated outputs of the machine learning algorithm using an attention mechanism, resulting in a repository representation comprising an abstract vector; and (5) tagging or summarizing, by the computer program, the script using its repository representation. ## Claims ### Claim 1 A method for code repository embedding, comprising: crawling, by a computer program executed by an electronic device, a code repository to identify a plurality of scripts in the code repository; generating, by the computer program, docstring embeddings, code embeddings, and dependency embeddings from the plurality of scripts in the code repository using one or more encoders; applying, by the computer program, a machine learning algorithm to each of the docstring embeddings, the code embeddings, and the dependency embeddings, resulting in a plurality of code repository representations; concatenating, by the computer program, the plurality of code repository representations; weighting, by the computer program, the plurality of code repository representations using attention weights for an attention mechanism, resulting in an abstract vector, wherein the attention weights are trained from a curated dataset collected from a labelled dataset; tagging or summarizing, by the computer program, the code repository based on the abstract vector; and classifying, by the computer program, the code repository representations based on the tagging or summarizing. ### Claim 2 (depends on 1) The method of claim 1, wherein the encoder comprises a Bi-directional Encoder Representations from Transformers. ### Claim 3 (depends on 1) The method of claim 1, wherein the machine learning algorithm comprises a gated recurrent unit (GRU) that is part of a Recurrent Neural Network (RNN). ### Claim 4 (depends on 1) The method of claim 1, wherein the abstract vector comprises a numerical representation of the repository. ### Claim 5 A system, comprising: an electronic device comprising: a computer processor; and a memory storing a computer program comprising instructions; a database comprising a curated dataset of labelled data; and a code repository database comprising a plurality of scripts; wherein, when read and executed by the computer processor, the instructions cause the one computer processor to perform steps comprising: crawling the code repository database to identify a plurality of scripts in the code repository; generating docstring embeddings, code embeddings, and dependency embeddings from the plurality of scripts in the code repository database using one or more encoders; applying a machine learning algorithm to each of the docstring embeddings, the code embeddings, and the dependency embeddings, resulting in a plurality of code repository representations; concatenating the plurality of code repository representations; weighing the plurality of code repository representations using attention weights for an attention mechanism, resulting in an abstract vector, wherein the attention weights are trained from a curated dataset collected from a labelled dataset; tagging or summarizing the code repository based on the abstract vector; and classifying the code repository representations based on the tagging or summarizing. ### Claim 6 (depends on 5) The system of claim 5, wherein the encoder comprises a Bi-directional Encoder Representations from Transformers. ### Claim 7 (depends on 5) The system of claim 5, wherein the machine learning algorithm comprises a gated recurrent unit (GRU) that is part of a Recurrent Neural Network (RNN). ### Claim 8 (depends on 5) The system of claim 5, wherein the abstract vector comprises a numerical representation of the repository. ### Claim 9 A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising: crawling a code repository database to identify a plurality of scripts in a code repository; generating docstring embeddings, code embeddings, and dependency embeddings from the plurality of scripts in the code repository applying a machine learning algorithm to each of the docstring embeddings, the code embeddings, and the dependency embeddings, resulting in a plurality of code repository representations; concatenating the plurality of code repository representations; weighting the plurality of code repository representations using attention weights for an attention mechanism, resulting in an abstract vector, wherein the attention weights are trained from a curated dataset collected from a labelled dataset; tagging or summarizing the code repository based on the abstract vector; and classifying the code repository representations based on the tagging or summarizing. ### Claim 10 (depends on 9) The non-transitory computer readable storage medium of claim 9, wherein the machine learning algorithm comprises a gated recurrent unit (GRU) that is part of a Recurrent Neural Network (RNN). ### Claim 11 (depends on 9) The non-transitory computer readable storage medium of claim 9, wherein the abstract vector comprises a numerical representation of the repository.