# US 2024/0202344 A1 · Use of word embeddings to locate sensitive text in computer programming scripts

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202344 A1
- Document ID: US20240202344A1
- Status: Published application
- Kind: A1
- Application number: 18/391,440
- Filed: 2023-12-20
- Published: 2024-06-20
- Earliest claimed: 2019-12-20
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Vincent PHAM; Kenneth TAYLOR; Jeremy Edward GOODSITT; Fardin ABDI TAGHI ABAD; Austin Grant WALTERS; Reza FARIVAR; Anh TRUONG; Mark Louis WATSON
- Web page: https://patentgrep.com/patent/US20240202344A1

## CPC

- G06F 21/577: {Assessing vulnerabilities and evaluating computer system security}
- G06F 21/31: User authentication
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06F 2221/033: Test or assess software
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Exemplary embodiments may use word embeddings to enhance scanning of programming code scripts for sensitive subject matter, such as confidential subject matter. The scanning may be performed by a neural network in some exemplary embodiments. The neural network initially may be trained on a corpus of programming code scripts to identify keywords relating to sensitive subject matter, such as passwords, tokens or credentials. The neural network may not only identify instances of the keywords but also may identify related terms as well. The output of the scan may be a ranked list of terms in the programming code script that may relate to sensitive subject matter.

## Claims

### Claim 1

(canceled)

### Claim 2

A system, comprising:
memory to store at least a portion of a neural network; and
one or more processors coupled with the memory to perform operations to:
train a neural network on a corpus of content, wherein the training results in word embeddings for words in the corpus, wherein each of the word embeddings is a numeric vector in a vector or matrix space;
identify an initial word of interest;
locate a vector that encodes the initial word of interest in the vector or matrix space;
identify vectors in the vector or matrix space that lie in a specified proximity to the vector for the initial word of interest and identifying words encoded by the identified vectors as additional words of interest, wherein the identifying comprises identifying ones of the vectors in the vector or matrix space that have distances or cosine values within a specified range as being in the specified proximity;
perform a security scan of a set of input to identify instances of the initial word of interest and instances of the additional words of interest in the input; and
generate output that specifies the identified instances of the initial word of interest in the set of input and that specifies that the instances of the additional words of interest in the set of input may be of interest.

### Claim 3 (depends on 2)

The system of claim 2, wherein the one or more processors comprise more than one processors residing in more than one servers in a distributed environment, wherein each of the more than one servers may perform a distributed portion of the operations.

### Claim 4 (depends on 2)

The system of claim 2, wherein the initial word of interest relates to one of a password, token or credential.

### Claim 5 (depends on 2)

The system of claim 2, wherein the corpus resides in a storage medium coupled with the one or more processors and contains content stored in multiple code repositories.

### Claim 6 (depends on 2)

The system of claim 2, wherein the output includes a ranked probability that the initial word of interest and additional words of interest instances relate to sensitive information.

### Claim 7 (depends on 2)

The system of claim 2, wherein the output produces ranked probabilities that the additional words of interest relate to sensitive content.

### Claim 8 (depends on 2)

The system of claim 2, wherein the security scan is performed by the neural network.

### Claim 9

An apparatus, comprising:
memory to store at least a portion of a neural network; and
one or more processors coupled with the memory to perform operations to:
identify a word embedding in a vector or matrix space for a word that is indicative of a password being in the input;
calculate distance metrics for distances between the identified word embedding and word embeddings having vectors in proximity to the identified word embedding,
wherein the word embeddings having vectors in proximity to the identified word embedding are word embeddings for which the calculated distance metrics fall within a specified range;
based on the calculated distance metrics, generate a probability for each of multiple words having word embeddings in proximity to the identified word embedding, the probability being a probability that the word is indicative of a password;
identify a subset of the multiple words that have a probability that exceeds a threshold; and
generate an output that identifies the identified subset.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the output identifies the words in the subset and provides the generated probability for the words in the subset.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein the input is fed to the neural network a word at a time.

### Claim 12 (depends on 9)

The apparatus of claim 9, wherein the input is fed an n-gram at a time where n is an integer greater than or equal to 2.

### Claim 13 (depends on 9)

The apparatus of claim 9, wherein calculation of the distance metrics comprises calculation of linear distance or calculation of cosine distance.

### Claim 14

An apparatus, comprising:
memory to store at least a portion of a trained neural network; and
one or more processors coupled with the memory to perform operations to:
input words to the trained neural network;
with the trained neural network, determine word embeddings for the input words, wherein each word embedding is a vector in a vector or matrix space;
compute proximity in the vector or matrix space between the word embeddings of input words and the word embeddings of known words of interest, wherein the computing proximity comprises calculating distances in the vector or matrix space between the input words and the word embeddings of the known words of interest;
based on the computed proximity, determine probabilities for at least some of the input words that the input words are of interest, where the calculated distances for the at least some of the input words are each within a specified range; and
output a ranked list of the determined probabilities for at least some of the input words wherein only probabilities over a threshold are output as part of the outputting.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the one or more processors comprise more than one processors residing in more than one servers in a distributed environment, wherein each of the more than one servers may perform a distributed portion of the operations.

### Claim 16 (depends on 14)

The apparatus of claim 14, wherein the known words of interest are at least one of passwords, tokens or credentials.

### Claim 17 (depends on 14)

The apparatus of claim 14, wherein the corpus contains content stored in multiple code repositories.

### Claim 18 (depends on 14)

The apparatus of claim 14, wherein the operations to calculate distances comprises operations to calculate cosine distances or calculate linear distances.

### Claim 19 (depends on 14)

The apparatus of claim 14, wherein the operations further cause the one or more processors to train a neural network to create the trained neural network.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein operations to train the neural network, a corpus of content having known words of interest is processed by the neural network.

### Claim 21 (depends on 14)

The apparatus of claim 14, wherein the probabilities for the input words increase as the calculated distances decrease.
