# US 2021/0173829 A1 · Natural Language Processing Engine For Translating Questions Into Executable Database Queries

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0173829 A1
- Document ID: US20210173829A1
- Status: Published application
- Kind: A1
- Application number: 16/866,034
- Filed: 2020-05-04
- Published: 2021-06-10
- Earliest claimed: 2019-12-09
- Assignees: salesforce.com, inc.
- Applicants: salesforce.com, inc.
- Inventors: Jichuan Zeng; Xi Lin; Chu Hong Hoi
- Web page: https://patentgrep.com/patent/US20210173829A1

## CPC

- G06F 16/243: {Natural language query formulation}
- G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
- G06F 16/24522: {Translation of natural language queries to structured queries}
- G06F 16/24524: {Access plan code generation and invalidation; Reuse of access plans}
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06F 40/47: Machine-assisted translation, e.g. using translation memory
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for translating questions into database queries are provided. A text to database query system receives a natural language question and a structure in a database. Question tokens are generated from the question and query tokens are generated from the structure in the database. The question tokens and query tokens are concatenated into a sentence and a sentence token is added to the sentence. A BERT network generates question hidden states for the question tokens, query hidden states for the query tokens, and a classifier hidden state for the sentence token. A translatability predictor network determines if the question is translatable or untranslatable. A decoder converts a translatable question into an executable query. A confusion span predictor network identifies a confusion span in the untranslatable question that causes the question to be untranslatable. An auto-correction module to auto-correct the tokens in the confusion span.

## Claims

### Claim 1

A method comprising:
receiving a natural language question and a structure in a database;
concatenating a sentence that includes question tokens from the natural language question, query tokens from the structure in the database, and a sentence token;
generating, using a table aware bidirectional encoder representations from transformers (BERT) network, question hidden states for the question tokens, query hidden states for the query tokens and a classification hidden state for the sentence token;
determining, using a translatability predictor network and the classification hidden state, that the natural language question is translatable or untranslatable;
if the natural language question is determined to be translatable, translating, using a decoder network, the natural language question into an executable query; and
if the natural language question is determined to be untranslatable, determining, using a confusion span predictor network, a confusion span that identifies at least one question token in the question tokens that causes the natural language question to be untranslatable given the structure in the database.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
substituting in the sentence the at least one question token that is in the confusion span with at least one mask token;
determining, using a masked language model (MLM) BERT network and the sentence with the at least one mask token, at least one token candidate from the structure in the database;
substituting the at least one mask token in the sentence with the at least one token candidate; and
translating, using the table aware BERT network the sentence into the executable query.

### Claim 3 (depends on 1)

The method of claim 1, wherein the structure in the database includes a database schema and at least one table name.

### Claim 4 (depends on 1)

The method of claim 1, wherein the query tokens are separated using a separation token.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
concatenating the sentence token to the sentence before a first question token in the question tokens.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
training at least one of a table aware BERT network, the translatability predictor network, or the confusion span predictor network using a dataset that includes pairs, each pair including translatable or untranslatable natural language question and corresponding structure in the database.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
generating the untranslatable language question in the dataset by randomly substituting one of the question tokens in the natural language question in a pair from pairs with another token that has a similar meaning as the one question token.

### Claim 8 (depends on 6)

The method of claim 6, further comprising:
generating the untranslatable natural language question in the dataset by randomly removing one of the question tokens in the translatable natural language question in a pair from pairs.

### Claim 9 (depends on 6)

The method of claim 6, further comprising:
generating the untranslatable natural language question by removing one of table names from a structure in the database that corresponds to the translatable natural language question in a pair.

### Claim 10

A text to database query translating system, the system comprising:
a memory;
at least one processor coupled to a memory and configured to:
receive a natural language question and a structure in a database;
concatenate a sentence that includes question tokens from the natural language question, query tokens from the structure in the database, and a sentence token;
generate, using a table aware bidirectional encoder representations from transformers (BERT) neural network, question hidden states for the question tokens, query hidden states for the query tokens and a classification hidden state for the sentence token;
determine, using a translatability predictor network and the classification hidden state, that the natural language question is untranslatable; and
determine, using a confusion span predictor network, a confusion span that identifies at least one question token in the question tokens that causes the natural language question to be untranslatable given the structure in the database.

### Claim 11 (depends on 10)

The text to database query translating system of claim 10, wherein the processor is further configured to:
substitute in the sentence the at least one question token that is in the confusion span with at least one mask token;
determine, using a masked language model (MLM) BERT network and the sentence with the at least one mask token, at least one token candidate from the structure in the database;
substitute the at least one mask token in the sentence with the at least one token candidate; and
translate, using the table aware BERT network the sentence into an executable query.

### Claim 12 (depends on 10)

The text to database query translating system of claim 10, wherein the structure in the database includes a database schema.

### Claim 13 (depends on 10)

The text to database query translating system of claim 10, wherein the query tokens are separated using a separation token.

### Claim 14 (depends on 10)

The text to database query translating system of claim 10, wherein the processor is further configured to:
concatenate the sentence token to the sentence prior to a first question token in the question tokens.

### Claim 15 (depends on 10)

The text to database query translating system of claim 10, wherein the processor is further configured to:
train at least one of a table aware BERT network, the translatability predictor network, or the confusion span predictor network using a dataset that includes pairs, each pair including translatable or untranslatable natural language question and corresponding structure in the database.

### Claim 16 (depends on 15)

The text to database query translating system of claim 15, wherein the processor is further configured to:
generate an untranslatable language question in the dataset by randomly substituting one of the question tokens in the natural language question in a pair in pairs with another token that has a similar meaning as the one question token.

### Claim 17 (depends on 15)

The text to database query translating system of claim 15, wherein the processor is further configured to:
generate the untranslatable natural language question in the dataset by randomly removing one of the question tokens in the natural language question in a pair of pairs.

### Claim 18 (depends on 15)

The text to database query translating system of claim 15, wherein the processor is further configured to:
generating the untranslatable natural language question by removing one of table names from a structure in the database that corresponds to the translatable natural language question in a pair.

### Claim 19

A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations that process a natural language, the operations comprising:
receiving a natural language question and a structure in a database
concatenating a sentence that includes question tokens from the natural language question, query tokens from the structure in the database, and a sentence token;
generating, using a table aware bidirectional encoder representations from transformers (BERT) network, question hidden states for the question tokens, query hidden states for the query tokens and a classification hidden state for the sentence token;
determining, using a translatability predictor network and the classification hidden state, that the natural language question is translatable or untranslatable;
if the natural language question is determined to be translatable, translating, using a decoder network, the natural language question into an executable query; and
if the natural language question is determined to be untranslatable, determining, using a confusion span predictor network, a confusion span that identifies at least one question token in the question tokens that causes the natural language question to be untranslatable given the structure in the database.

### Claim 20 (depends on 19)

The non-transitory machine-readable medium of claim 19, wherein the machine is further configured to perform the operations comprising:
substituting in the sentence the at least one question token that is in the confusion span with at least one mask token;
determining, using a masked language model (MLM) BERT network and the sentence with the at least one mask token, at least one token candidate from the structure in the database;
substituting the at least one mask token in the sentence with the at least one token candidate; and
translating, using the table aware BERT network the sentence into the executable query.
