# US 2025/0094570 A1 · Systems and methods for using vector model normal exclusion in natural language processing to characterize a category of messages

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0094570 A1
- Document ID: US20250094570A1
- Status: Published application
- Kind: A1
- Application number: 18/901,473
- Filed: 2024-09-30
- Published: 2025-03-20
- Earliest claimed: 2021-08-23
- Assignees: Fortinet, Inc.
- Applicants: Fortinet, Inc.
- Inventors: Sameer Khanna
- Web page: https://patentgrep.com/patent/US20250094570A1

## CPC

- G06F 21/552: {involving long-term monitoring or reporting}
- G01V 1/01: Measuring or predicting earthquakes
- G01W 1/10: Devices for predicting weather conditions (computers per se G06; display devices G09)
- G06F 18/24: Classification techniques
- G06F 21/316: {by observing the pattern of computer usage, e.g. typical user behaviour}
- G06F 21/33: using certificates
- G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database}
- G06F 40/242: Dictionaries
- G06F 40/279: Recognition of textual entities
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06Q 20/027: {involving a payment switch or gateway}
- G06Q 20/389: {Keeping log of transactions for guaranteeing non-repudiation of a transaction}
- G06Q 40/02: Banking, e.g. interest calculation or account maintenance (credit or loans G06Q40/03)
- G06Q 40/08: Insurance
- G06Q 50/22: Social work or social welfare, e.g. community support activities or counselling services
- G06V 10/56: relating to colour
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/776: Validation; Performance evaluation
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- H04L 43/045: for graphical visualisation of monitoring data
- H04L 63/1416: {Event detection, e.g. attack signature detection}
- H04L 63/1425: {Traffic logging, e.g. anomaly detection}
- G06F 40/157: using dictionaries or tables
- G06F 40/205: Parsing
- G06F 40/295: Named entity recognition
- G06F 40/30: Semantic analysis
- G06N 20/00: Machine learning
- G06Q 50/26: Government or public services (business processes related to the transportation industry G06Q50/40)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G01V: GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS (means for indicating the location of accidentally buried, e.g. snow-buried, persons A63B29/02)
- G01W: METEOROLOGY (radar, sonar, lidar or analogous systems, designed for meteorological use G01S13/95, G01S15/88, G01S17/95)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Systems, devices, and methods are disclosed in relation to a vector space model that may be used to characterize a category of messages. In one of many possible implementations, the frequency of words found within a piece of text is determined. These frequencies are compared against the frequencies of words within a given corpus like the Oxford English Corpus by first converting the frequencies to probabilities via the inverse cumulative distribution function assuming a normal distribution of frequencies then via taking the absolute difference in frequencies. A small difference reduces the weight of the given word whereas a large weight increases the weight of the word, leading to excellent word ranking for automated feature selection filtering without the need for a negative corpus.

## Claims

### Claim 1

A method comprising:
determining, by a processing resource, a set of normal exclusion values by for each unique word in each model document of a plurality of model documents that relate to a particular topic, calculating a normal exclusion value for inclusion within the set of normal exclusion values based on a frequency of the unique word in the model document and a frequency of the unique word in a dictionary, wherein each of the plurality of model documents includes text content;
creating, by the processing resource, a plurality of model document vectors by forming for a given model document of the plurality of model documents a vector having a plurality of normal exclusion values from the set of normal exclusion values in which each of the plurality of normal exclusion values correspond to a given unique word in the given model document;
identifying, by the processing resource, a largest value in a first dimension within the plurality of model document vectors as a first topic extreme, and a largest value in a second dimension within the plurality of model document vectors as a second topic extreme; and
categorizing, by the processing resource, a document at issue as included within or excluded from the particular topic by creating a vector for the document at issue based on a model vector definition and comparing the vector to one or both of the first topic extreme and the second topic extreme.

### Claim 2 (depends on 1)

The method of claim 1, wherein the dictionary includes more than one hundred thousand unique words and corresponding frequencies of each unique word of the more than one hundred thousand unique words.

### Claim 3 (depends on 1)

The method of claim 1, wherein the dictionary comprises at least a subset of words in the Oxford English Corpus.

### Claim 4 (depends on 1)

The method of claim 1, wherein the normal exclusion value is calculated in accordance with the following equation:
NE
=
❘
"\[LeftBracketingBar]"
F
-
1
(
tpr
+
ε
)
-
F
-
1
(
Dictionary
[
selected
⁢
unique
⁢
word
]
+
ε
)
❘
"\[RightBracketingBar]"
,
wherein F−1 is an inverse normal cumulative distribution function, tpr is the frequency of the particular unique word in the particular message, ε is a small number to prevent an undefined case of F−1(0), and Dictionary [selected unique word] is the frequency of the unique word in the dictionary.

### Claim 5 (depends on 1)

The method of claim 1, the method further comprising generating the model vector definition for the particular topic, wherein forming the vector for the given model document includes ordering the plurality of normal exclusion values of the vector according to the model vector definition.

### Claim 6 (depends on 5)

The method of claim 5, wherein the vector definition includes a position for each normal exclusion value for each unique word found across all of the plurality of model documents.

### Claim 7 (depends on 1)

The method of claim 1, wherein identifying the largest value in the first dimension includes comparing a first dimension value from each of the plurality of model document vectors.

### Claim 8 (depends on 1)

The method of claim 1, wherein identifying the largest value in the second dimension includes comparing a second dimension value from each of the plurality of model document vectors.

### Claim 9 (depends on 1)

The method of claim 1, wherein the dictionary is maintained in a look-up table.

### Claim 10

A non-transitory computer-readable storage medium embodying a set of instructions, which when executed by a processing resource, causes the processing resource to:
determine a set of normal exclusion values by for each unique word in each model document of a plurality of model documents that relate to a particular topic, calculating a normal exclusion value for inclusion within the set of normal exclusion values based on a frequency of the unique word in the model document and a frequency of the unique word in a dictionary, wherein each of the plurality of model documents includes text content;
create a plurality of model document vectors by forming for a given model document of the plurality of model documents a vector having a plurality of normal exclusion values from the set of normal exclusion values in which each of the plurality of normal exclusion values correspond to a given unique word in the given model document;
identify a largest value in a first dimension within the plurality of model document vectors as a first topic extreme, and a largest value in a second dimension within the plurality of model document vectors as a second topic extreme; and
categorize a document at issue as included within or excluded from the particular topic by creating a vector for the document at issue based on a model vector definition and comparing the vector to one or both of the first topic extreme and the second topic extreme.

### Claim 11 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein the dictionary includes more than one hundred thousand unique words and corresponding frequencies of each unique word of the more than one hundred thousand unique words.

### Claim 12 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein the dictionary comprises at least a most frequently occurring portion of words in the Oxford English Corpus.

### Claim 13 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein the normal exclusion value is calculated in accordance with the following equation:
NE
=
❘
"\[LeftBracketingBar]"
F
-
1
(
tpr
+
ε
)
-
F
-
1
(
Dictionary
[
selected
⁢
unique
⁢
word
]
+
ε
)
❘
"\[RightBracketingBar]"
,
wherein F−1 is an inverse normal cumulative distribution function, tpr is the frequency of the particular unique word in the particular message, ε is a small number to prevent an undefined case of F−1 (0), and Dictionary [selected unique word] is the frequency of the unique word in the dictionary.

### Claim 14 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein the instructions further cause the processing resource to generate the model vector definition for the particular topic, wherein forming the vector for the given model document includes ordering the plurality of normal exclusion values of the vector according to the model vector definition.

### Claim 15 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein the vector definition includes a position for a subset of each normal exclusion value for each unique word found across all of the plurality of model documents.

### Claim 16 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein identification of the largest value in the first dimension includes comparing a first dimension value from each of the plurality of model document vectors.

### Claim 17 (depends on 10)

The non-transitory computer-readable storage medium of claim 10, wherein identification of the largest value in the second dimension includes comparing a second dimension value from each of the plurality of model document vectors.

### Claim 18 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the dictionary is maintained in a look-up table.

### Claim 19

A system comprising:
a processing resource; and
a non-transitory computer-readable medium, coupled to the processing resource, having stored therein instructions that when executed by the processing resource cause the processing resource to:
determine a set of normal exclusion values by for each unique word in each model document of a plurality of model documents that relate to a particular topic, calculating a normal exclusion value for inclusion within the set of normal exclusion values based on a frequency of the unique word in the model document and a frequency of the unique word in a dictionary, wherein each of the plurality of model documents includes text content;
create a plurality of model document vectors by forming for a given model document of the plurality of model documents a vector having a plurality of normal exclusion values from the set of normal exclusion values in which each of the plurality of normal exclusion values correspond to a given unique word in the given model document;
identify a largest value in a first dimension within the plurality of model document vectors as a first topic extreme, and a largest value in a second dimension within the plurality of model document vectors as a second topic extreme; and
categorize a document at issue as included within or excluded from the particular topic by creating a vector for the document at issue based on a model vector definition and comparing the vector to one or both of the first topic extreme and the second topic extreme.

### Claim 20 (depends on 19)

The system of claim 19, wherein the normal exclusion value is calculated in accordance with the following equation:
NE
=
❘
"\[LeftBracketingBar]"
F
-
1
(
tpr
+
ε
)
-
F
-
1
(
Dictionary
[
selected
⁢
unique
⁢
word
]
+
ε
)
❘
"\[RightBracketingBar]"
,
wherein F−1 is an inverse normal cumulative distribution function, tpr is the frequency of the particular unique word in the particular message, ε is a small number to prevent an undefined case of F−1(0), and Dictionary [selected unique word] is the frequency of the unique word in the dictionary.
