# US 2017/0286401 A1 · Method And System For Processing An Input Query

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286401 A1
- Document ID: US20170286401A1
- Status: Published application
- Kind: A1
- Application number: 15/476,262
- Filed: 2017-03-31
- Published: 2017-10-05
- Earliest claimed: 2016-03-31
- Assignees: Maluuba Inc.
- Applicants: Maluuba Inc.
- Inventors: Jing HE; Jean MERHEB-HARB; Zheng YE; Kaheer SULEMAN
- Web page: https://patentgrep.com/patent/US20170286401A1

## CPC

- G06F 16/3344: {using natural language analysis}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 40/30: Semantic analysis
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06Q 30/0613: {using intermediate agents}
- G06F 18/295: {Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models}
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G10L 15/16: using artificial neural networks
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Disclosed embodiments include systems and methods relevant to improvements to natural language processing used to determine an intent and one or more associated parameters from a given input string. In an example, an input string is received and first and second different n-grams are applied to the input string. Recurrent neural network models are then used to generate output data based in part on the first and second different n-grams. Intent detection and semantic labeling are applied to the output of the recurrent neural network models.

## Claims

### Claim 1

A method of processing an input query, the method comprising:
receiving the input query, wherein the input query comprises a plurality of words;
storing the input query in a memory storage unit;
assigning a first n-gram and a second n-gram to the plurality of words, wherein the first n-gram is different from the second n-gram;
applying a first recurrent neural network model to the first n-gram to generate first output data;
applying a second recurrent neural network model to the second n-gram to generate second output data; and
applying intent detection and semantic labelling to the first output data and the second output data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first n-gram comprises unigrams.

### Claim 3 (depends on 2)

The method of claim 2, wherein the second n-gram comprises bigrams.

### Claim 4 (depends on 1)

The method of claim 1, further comprising assigning a third n-gram to the plurality of words.

### Claim 5 (depends on 4)

The method of claim 4, further comprising applying a third recurrent neural network model to the third n-gram to generate third output data, wherein the intent detection and semantic labelling are further applied to the third output data.

### Claim 6 (depends on 4)

The method of claim 4, further comprising applying a concept vector recurrent neural network model to the third n-gram to generate third output data, wherein the intent detection and semantic labelling are further applied to the third output data.

### Claim 7 (depends on 4)

The method of claim 4, wherein the third n-gram comprises trigrams.

### Claim 8 (depends on 1)

The method of claim 1, further comprising detecting an intent by applying a softmax regression analysis on the first output data and the second output data.

### Claim 9 (depends on 1)

The method of claim 1, further comprising labelling each word from the plurality of words with a semantic label by applying conditional random field analysis.

### Claim 10 (depends on 9)

The method of claim 9, wherein the conditional random field analysis uses the intent for classification.

### Claim 11

A non-transitory computer readable medium encoded with codes, the codes for directing a processor to:
receive an input query, wherein the input query comprises a plurality of words;
store the input query in a memory storage unit;
assign a first n-gram and a second n-gram to the plurality of words, wherein the first n-gram is different from the second n-gram;
apply a first recurrent neural network model to the first n-gram to generate first output data;
apply a second recurrent neural network model to the second n-gram to generate second output data; and
apply intent detection and semantic labelling to the first output data and the second output data.

### Claim 12 (depends on 11)

The non-transitory computer readable medium of claim 11, wherein the codes further direct the processor to pool the first output data and the second output data.

### Claim 13 (depends on 12)

The non-transitory computer readable medium of claim 12,
wherein the codes further direct the processor to provide the pooled first and second output data as input to a third neural network model; and
wherein applying intent detection and semantic labelling to the first output data and the second output data comprises applying intent detection and semantic labeling to output of the third neural network model.

### Claim 14 (depends on 11)

The non-transitory computer readable medium of claim 11, wherein applying the first recurrent neural network model comprises applying the first recurrent neural network model in a forward direction to generate forward output data.

### Claim 15 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein applying the second recurrent neural network model comprises applying the second recurrent neural network model in a backward direction to generate backward output data.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, wherein the codes further direct the processor to: concatenate the forward and backward output data.

### Claim 17

A method of processing an input query, the method comprising:
receiving the input query, wherein the input query comprises a plurality of words;
storing the input query in a memory storage unit;
assigning a first n-gram and a second n-gram to the plurality of words, wherein the first n-gram is different from the second n-gram;
applying, in a first layer, a first recurrent neural network model to the first n-gram to generate first output data, and applying a second recurrent neural network model to the second n-gram to generate second output data;
pooling the first output data and the second output data from the first layer to generate input data for a second layer; and
applying, in a second layer, the recurrent neural network model to the input data to generate final output data.

### Claim 18 (depends on 17)

The method of claim 17, wherein pooling the first output data and the second output data comprises applying a linear transformation on at least one of the first output data and the second output data.

### Claim 19 (depends on 18)

The method of claim 18, wherein the linear transformation is configured to fit the second layer.

### Claim 20 (depends on 17)

The method of claim 17, wherein the first recurrent neural network model is trained for n-grams of a same type as the first n-gram; and wherein the second recurrent neural network model is trained for n-grams of a same type as the first n-gram.