# US 2021/0241331 A1 · Systems and methods of business categorization and service recommendation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241331 A1
- Document ID: US20210241331A1
- Status: Published application
- Kind: A1
- Application number: 16/779,781
- Filed: 2020-02-03
- Published: 2021-08-05
- Earliest claimed: 2020-02-03
- Assignees: Intuit Inc.
- Applicants: Intuit Inc.
- Inventors: Erez Katzenelson; Elik Sror; Shlomi Medalion; Shimon Shahar; Shir Meir Lador; Sigalit Bechler; Alexander Zhicharevich; Onn Bar
- Web page: https://patentgrep.com/patent/US20210241331A1

## CPC

- G06Q 30/04: Billing or invoicing
- G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30)
- G06F 40/216: using statistical methods
- G06F 40/268: Morphological analysis
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06Q 10/067: Enterprise or organisation modelling
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

In one aspect, the present disclosure relates to a method of generating business descriptions performed by a server, said method may include: receiving a plurality of invoices, each invoice being associated with a business of a plurality of businesses; extracting a plurality of texts from the plurality of invoices; embedding the plurality of texts to a vector space to obtain a plurality of invoice vectors; generating a plurality of clusters in the vector space, each cluster of the plurality of clusters comprising at least one invoice vector of the plurality of invoice vectors; generating a description for a cluster, the description for the cluster representing all invoice vectors assigned to the cluster; for each business of the plurality of businesses that has at least one invoice vector assigned to the cluster, associating the business with the description; and indexing the plurality of businesses within a database by the generated descriptions.

## Claims

### Claim 1

A method of generating business descriptions performed by a server, said method comprising:
receiving a plurality of invoices, each invoice being associated with a business of a plurality of businesses;
extracting a plurality of texts from the plurality of invoices;
embedding the plurality of texts to a vector space to obtain a plurality of invoice vectors;
generating a plurality of clusters in the vector space, each cluster of the plurality of clusters comprising at least one invoice vector of the plurality of invoice vectors;
generating a description for a cluster, the description for the cluster representing all invoice vectors assigned to the cluster;
for each business of the plurality of businesses that has at least one invoice vector assigned to the cluster, associating the business with the description; and
indexing the plurality of businesses within a database by the generated descriptions.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
detecting stop words in the plurality of texts from a pre-defined list of stop words;
removing the detected stop words from the extracted plurality of texts;
lemmatizing words within the extracted plurality of texts to obtain lemmatized texts; and
embedding the lemmatized texts to obtain the plurality of invoice vectors.

### Claim 3 (depends on 2)

The method of claim 2 further comprising:
calculating a term frequency-inverse document frequency (TFIDF) value for each word of the lemmatized texts;
ranking the calculated TFIDF values;
identifying a top pre-defined number of words with a highest rank; and
embedding the identified words to create the plurality of invoice vectors.

### Claim 4 (depends on 2)

The method of claim 2 further comprising:
calculating a TFIDF value for each word of the lemmatized texts;
identifying a pre-defined number of words with TFIDF values above a certain threshold; and
embedding the identified words to obtain the plurality of invoice vectors.

### Claim 5 (depends on 1)

The method of claim 1, wherein embedding the plurality of texts comprises:
applying a word2vec model, the applying comprising, for each text of an invoice:
creating a bag-of-words representing the text, the bag-of-words including each word in the text and an associated multiplicity of each word in the text; and
converting each word in the text into a vector based on the bag of words and not based on grammar and word order.

### Claim 6 (depends on 5)

The method of claim 5, wherein embedding the plurality of texts comprises, for a line of an invoice comprising a plurality of words:
embedding each word of the invoice line using the word2vec model to obtain a plurality of word vectors;
feeding the plurality of word vectors to a long short-term memory (LSTM) layer; and
combining, via the LSTM layer, the plurality of word vectors to obtain a single vector representing the invoice line, wherein combining includes at least one of vector addition, vector subtraction, scalar multiplication, sigmoid function multiplication, or hyperbolic function multiplication.

### Claim 7 (depends on 3)

The method of claim 3 further comprising:
generating an invoice line vector for each line of the plurality of invoices;
generating the plurality of clusters, wherein at least one invoice line vector is assigned to each cluster; and
generating the description for a cluster, the description for the cluster representing all invoice line vectors assigned to the cluster.

### Claim 8 (depends on 1)

The method of claim 1, wherein generating the description comprises:
calculating a term frequency-inverse document frequency (TFIDF) value for each word in a cluster;
ranking the calculated TFIDF values;
identifying a top pre-defined number of words with a highest rank; and
generating the description for the cluster with the identified words.

### Claim 9 (depends on 1)

The method of claim 1, wherein generating the plurality of clusters comprises clustering the plurality of invoice vectors in the vector space with at least one of mean-shift clustering, k-means clustering, affinity propagation clustering, spectral clustering, hierarchical clustering, density-based spatial cluster of applications with noise (DBSCAN), ordering points to identify a clustering structure (OPTICS), Gaussian mixture modeling, or Birch clustering.

### Claim 10 (depends on 1)

The method of claim 1 further comprising:
receiving a plurality of bank transactions, each bank transaction being associated with a business of the plurality of businesses;
embedding text from the plurality of bank transactions to the vector space to obtain a plurality of transaction vectors;
generating the plurality of clusters in the vector space, each cluster of the plurality of clusters comprising at least one an invoice vector of the plurality of invoice vectors or a transaction vector of the plurality of transaction vectors;
generating the description for a cluster, the description for the cluster representing all invoice vectors and transaction vectors assigned to the cluster; and
for each business of the plurality of businesses that has at least one invoice vector or transaction vector assigned to the cluster, associating the business with the description.

### Claim 11

A method for determining a list of offerings for a business performed by a server, said method comprising:
receiving a plurality of invoices associated with the business from one or more devices associated with the business;
extracting a plurality of texts from the plurality of invoices;
embedding each of the plurality of texts to a vector space to obtain a plurality of invoice vectors;
for each invoice vector of the plurality of invoice vectors:
calculating a plurality of Euclidean distances between the invoice vector and a plurality of points, the plurality of points comprising a point from each of a pre-defined plurality of clusters;
selecting, based on the plurality of Euclidean distances, a pre-defined cluster closest to the invoice vector; and
assigning the invoice vector to the selected pre-defined cluster;
creating the list of offerings for the business, the list of offerings comprising a description from each selected pre-defined cluster;
receiving a request from a device of the one or more devices associated with the business; and
causing the list of offerings to be displayed on the device.

### Claim 12 (depends on 11)

The method of claim 11 further comprising:
detecting stop words in the extracted plurality of texts from a pre-defined list of stop words;
removing the detected stop words from the extracted plurality of texts;
lemmatizing words within the extracted plurality of texts to obtain lemmatized texts; and
embedding the lemmatized texts to obtain the plurality of invoice vectors.

### Claim 13 (depends on 12)

The method of claim 12 further comprising:
calculating a term frequency-inverse document frequency (TFIDF) value for each word of the lemmatized texts;
ranking the calculated TFIDF values;
identifying a top pre-defined number of words with a highest rank; and
embedding the identified words to obtain the plurality of invoice vectors.

### Claim 14 (depends on 11)

The method of claim 11, wherein embedding each of the plurality of texts comprises applying a word2vec model, the applying comprising, for each text of an invoice:
creating a bag-of-words representing the text, the bag-of-words including each word in the text and an associated multiplicity of each word in the text; and
converting each word in the text into a vector based on the bag of words and not based on grammar and word order.

### Claim 15 (depends on 14)

The method of claim 14, wherein embedding each of the plurality of texts comprises, for an invoice line comprising a plurality of words:
embedding each word of the invoice line using the word2vec model to obtain a plurality of word vectors;
feeding the plurality of word vectors to a long short-term memory (LSTM) layer;
combining, via the LSTM layer, the plurality of word vectors to obtain a single vector representing the invoice line, wherein combining includes at least one of vector addition, vector subtraction, scalar multiplication, sigmoid function multiplication, or hyperbolic function multiplication.

### Claim 16 (depends on 11)

The method of claim 11 further comprising:
receiving a plurality of bank transactions associated with a business from one or more devices associated with the business;
embedding texts from the plurality of bank transactions to the vector space to obtain a plurality of transaction vectors;
for each transaction vector of the plurality of transaction vectors:
calculating a plurality of Euclidean distances between the transaction vector and the plurality of points;
selecting, based on the plurality of Euclidean distances, a pre-defined cluster closest to the transaction vector; and
assigning the transaction vector to the selected pre-defined cluster;
creating the list of offerings for the business; and
causing the list of offerings to be displayed on the device.

### Claim 17 (depends on 11)

The method of claim 11 wherein the description from each selected pre-defined cluster is determined by:
calculating a term frequency-inverse document frequency (TFIDF) value for each word in a cluster;
ranking the calculated TFIDF values;
identifying a top pre-defined number of words with a highest rank; and
generating the description for the cluster with the identified words.

### Claim 18

A method for training a model to predict a category for a business performed by a server, said method comprising:
receiving a plurality of invoices associated with the business;
receiving a category associated with the business;
extracting a plurality of texts from the plurality of invoices, the plurality of texts comprising a text from each of the plurality of invoices;
embedding the plurality of texts to a vector space to obtain a plurality of invoice vectors;
generating, based on the plurality of invoice vectors, a business vector representing the business; and
training a neural network to predict the category based on the business vector.

### Claim 19 (depends on 18)

The method of claim 18, wherein generating the business vector comprises averaging all invoice vectors associated with the business.

### Claim 20 (depends on 18)

The method of claim 18, wherein the neural network comprises three fully connected layers and training the neural network comprises:
receiving a plurality of business pairs as labelled training data, each business pair including a business vector and a label indicating a category that the business vector belongs to;
applying a back-propagation algorithm to train the neural network based on the labelled training data; and
training a softmax layer within the neural network to map category predictions of the neural network to probabilities.