# US 2025/0225129 A1 · Techniques for efficient encoding in neural semantic parsing systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0225129 A1
- Document ID: US20250225129A1
- Status: Published application
- Kind: A1
- Application number: 18/409,676
- Filed: 2024-01-10
- Published: 2025-07-10
- Earliest claimed: 2024-01-10
- Assignees: Oracle International Corporation
- Applicants: Oracle International Corporation
- Inventors: Cong Duy Vu Hoang; Poorya Zaremoodi; Thanh Tien Vu; Gioacchino Tangari; Mark Edward Johnson; Thanh Long Duong; Vishal Vishnoi
- Web page: https://patentgrep.com/patent/US20250225129A1

## CPC

- G06F 16/24522: {Translation of natural language queries to structured queries}
- G06F 16/24552: {Database cache management}
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/1822: {Parsing for meaning understanding}
- G06F 16/24526: {Internal representations for queries}
- G06F 16/24528: {Standardisation; Simplification}
- G06F 16/284: {Relational databases}
- G06F 40/30: Semantic analysis
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G10L 15/18: using natural language modelling
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Techniques for natural language processing include accessing an input string comprising a natural language utterance and a database schema representation for a database; providing the natural language utterance to a first encoder to generate one or more embeddings of the natural language utterance; providing the database schema representation to the first encoder to generate one or more embeddings of the database schema representation; encoding, by a second encoder, relations between elements in the database schema representation and words in the natural language utterance based on the one or more embeddings of the natural language utterance and the one or more embeddings of the database schema representation; and generating a logical form for the natural language utterance based on the encoded relations, the one or more embeddings of the natural language utterance, and the one or more embeddings of the database schema representation.

## Claims

### Claim 1

A computer-implemented method comprising:
accessing an input string comprising a natural language utterance and a database schema representation for a database;
providing the natural language utterance to a first encoder to generate one or more embeddings of the natural language utterance;
providing the database schema representation to the first encoder to generate one or more embeddings of the database schema representation;
encoding, by a second encoder, relations between elements in the database schema representation and words in the natural language utterance based on the one or more embeddings of the natural language utterance and the one or more embeddings of the database schema representation; and
generating a logical form for the natural language utterance based on the encoded relations, the one or more embeddings of the natural language utterance, and the one or more embeddings of the database schema representation.

### Claim 2 (depends on 1)

The method of claim 1, wherein the input string is a first input string and the natural language utterance is a first natural language utterance, the method further comprising:
storing the one or more embeddings of the database schema representation to a cache;
accessing a second input string comprising a second natural language utterance and the database schema representation;
providing the second natural language utterance to the first encoder to generate one or more embeddings of the second natural language utterance;
retrieving the one or more embeddings of the database schema representation from the cache;
encoding, by the second encoder, relations between elements in the database schema representation and words in the second natural language utterance based on the one or more embeddings of the second natural language utterance and the one or more embeddings of the database schema representation; and
generating a logical form for the second natural language utterance based on the encoded relations, the one or more embeddings of the second natural language utterance, and the one or more embeddings of the database schema representation.

### Claim 3 (depends on 2)

The method of claim 2, wherein the cache comprises a local cache.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first encoder comprises a plurality of layers, and wherein:
providing the natural language utterance to the first encoder to generate one or more embeddings of the natural language utterance comprises providing the natural language utterance to a first subset of the layers of the first encoder to generate one or more utterance embeddings;
providing the database schema representation to the first encoder to generate one or more embeddings of the database schema representation comprises providing the database schema representation to the first subset of the layers of the first encoder to generate one or more schema embeddings;
the method further comprising:
concatenating the one or more utterance embeddings and the one or more schema embeddings to generate one or more concatenated embeddings; and
providing the one or more concatenated embeddings to a second subset of the layers of the first encoder to generate the one or more embeddings of the natural language utterance and the one or more embeddings of the database schema representation.

### Claim 5 (depends on 4)

The method of claim 4, wherein:
the first encoder comprises N layers;
the first subset of the layers comprises K layers, wherein K is a configurable integer less than N; and
the second subset of the layers comprises N-K layers.

### Claim 6 (depends on 1)

The method of claim 1, wherein the input string further comprises one or more tokens, and the method further comprises:
providing the one or more tokens to the first encoder to generate one or more embeddings of the one or more tokens; and
caching the one or more embeddings of the one or more tokens.

### Claim 7 (depends on 1)

The method of claim 1, wherein:
the first encoder is a Pre-trained Language Model (PLM); and
the second encoder is a Relation-Aware Transformer (RAT).

### Claim 8

A system comprising:
one or more processors; and
one or more computer-readable media storing instructions which, when executed by the one or more processors, cause the system to perform operations comprising:
accessing an input string comprising a natural language utterance and a database schema representation for a database;
providing the natural language utterance to a first encoder to generate one or more embeddings of the natural language utterance;
providing the database schema representation to the first encoder to generate one or more embeddings of the database schema representation;
encoding, by a second encoder, relations between elements in the database schema representation and words in the natural language utterance based on the one or more embeddings of the natural language utterance and the one or more embeddings of the database schema representation; and
generating a logical form for the natural language utterance based on the encoded relations, the one or more embeddings of the natural language utterance, and the one or more embeddings of the database schema representation.

### Claim 9 (depends on 8)

The system of claim 8, wherein the input string is a first input string and the natural language utterance is a first natural language utterance, the operations further comprising:
storing the one or more embeddings of the database schema representation to a cache;
accessing a second input string comprising a second natural language utterance and the database schema representation;
providing the second natural language utterance to the first encoder to generate one or more embeddings of the second natural language utterance;
retrieving the one or more embeddings of the database schema representation from the cache;
encoding, by the second encoder, relations between elements in the database schema representation and words in the second natural language utterance based on the one or more embeddings of the second natural language utterance and the one or more embeddings of the database schema representation; and
generating a logical form for the second natural language utterance based on the encoded relations, the one or more embeddings of the second natural language utterance, and the one or more embeddings of the database schema representation.

### Claim 10 (depends on 9)

The system of claim 9, wherein the cache comprises a local cache.

### Claim 11 (depends on 8)

The system of claim 8, wherein the first encoder comprises a plurality of layers, and wherein:
providing the natural language utterance to the first encoder to generate one or more embeddings of the natural language utterance comprises providing the natural language utterance to a first subset of the layers of the first encoder to generate one or more utterance embeddings;
providing the database schema representation to the first encoder to generate one or more embeddings of the database schema representation comprises providing the database schema representation to the first subset of the layers of the first encoder to generate one or more schema embeddings;
the operations further comprising:
concatenating the one or more utterance embeddings and the one or more schema embeddings to generate one or more concatenated embeddings; and
providing the one or more concatenated embeddings to a second subset of the layers of the first encoder to generate the one or more embeddings of the natural language utterance and the one or more embeddings of the database schema representation.

### Claim 12 (depends on 11)

The system of claim 11, wherein:
the first encoder comprises N layers;
the first subset of the layers comprises K layers, wherein K is a configurable integer less than N; and
the second subset of the layers comprises N-K layers.

### Claim 13 (depends on 8)

The system of claim 8, wherein the input string further comprises one or more tokens, and the operations further comprise:
providing the one or more tokens to the first encoder to generate one or more embeddings of the one or more tokens; and
caching the one or more embeddings of the one or more tokens.

### Claim 14 (depends on 8)

The system of claim 8, wherein:
the first encoder is a Pre-trained Language Model (PLM); and
the second encoder is a Relation-Aware Transformer (RAT).

### Claim 15

One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause a system to perform operations comprising:
accessing an input string comprising a natural language utterance and a database schema representation for a database;
providing the natural language utterance to a first encoder to generate one or more embeddings of the natural language utterance;
providing the database schema representation to the first encoder to generate one or more embeddings of the database schema representation;
encoding, by a second encoder, relations between elements in the database schema representation and words in the natural language utterance based on the one or more embeddings of the natural language utterance and the one or more embeddings of the database schema representation; and
generating a logical form for the natural language utterance based on the encoded relations, the one or more embeddings of the natural language utterance, and the one or more embeddings of the database schema representation.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the input string is a first input string and the natural language utterance is a first natural language utterance, the operations further comprising:
storing the one or more embeddings of the database schema representation to a cache;
accessing a second input string comprising a second natural language utterance and the database schema representation;
providing the second natural language utterance to the first encoder to generate one or more embeddings of the second natural language utterance;
retrieving the one or more embeddings of the database schema representation from the cache;
encoding, by the second encoder, relations between elements in the database schema representation and words in the second natural language utterance based on the one or more embeddings of the second natural language utterance and the one or more embeddings of the database schema representation; and
generating a logical form for the second natural language utterance based on the encoded relations, the one or more embeddings of the second natural language utterance, and the one or more embeddings of the database schema representation.

### Claim 17 (depends on 16)

The one or more non-transitory computer-readable media of claim 16, wherein the cache comprises a local cache.

### Claim 18 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the first encoder comprises a plurality of layers, and wherein:
providing the natural language utterance to the first encoder to generate one or more embeddings of the natural language utterance comprises providing the natural language utterance to a first subset of the layers of the first encoder to generate one or more utterance embeddings;
providing the database schema representation to the first subset of the layers of the first encoder to generate one or more embeddings of the database schema representation comprises providing the database schema representation to the first encoder to generate one or more schema embeddings;
the operations further comprising:
concatenating the one or more utterance embeddings and the one or more schema embeddings to generate one or more concatenated embeddings; and
providing the one or more concatenated embeddings to a second subset of the layers of the first encoder to generate the one or more embeddings of the natural language utterance and the one or more embeddings of the database schema representation.

### Claim 19 (depends on 18)

The one or more non-transitory computer-readable media of claim 18, wherein:
the first encoder comprises N layers;
the first subset of the layers comprises K layers, wherein K is a configurable integer less than N; and
the second subset of the layers comprises N-K layers.

### Claim 20 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the input string further comprises one or more tokens, and the method further comprises:
providing the one or more tokens to the first encoder to generate one or more embeddings of the one or more tokens; and
caching the one or more embeddings of the one or more tokens.
