# US 2021/0173837 A1 · Generating followup questions for interpretable recursive multi-hop question answering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0173837 A1
- Document ID: US20210173837A1
- Status: Published application
- Kind: A1
- Application number: 17/109,781
- Filed: 2020-12-02
- Published: 2021-06-10
- Earliest claimed: 2019-12-06
- Assignees: NEC Laboratories America, Inc.
- Applicants: NEC Laboratories America, Inc.
- Inventors: Christopher Malon; Bing Bai
- Web page: https://patentgrep.com/patent/US20210173837A1

## CPC

- G06F 16/24522: {Translation of natural language queries to structured queries}
- G06F 16/3329: Natural language query formulation
- G06F 16/355: Creation or modification of classes or clusters
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/08: Learning methods
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 5/04: Inference or reasoning models
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method is provided for generating following up questions for multi-hop bridge-type question answering. The method includes retrieving a premise for an input multi-hop bridge-type question. The method further includes assigning, by a three-way neural network based controller, a classification of the premise against the input multi-hop bridge-type question as being any of irrelevant, including a final answer, or including intermediate information. The method also includes outputting the final answer in relation to a first hop of the multi-hop bridge-type question responsive to the classification being including the final answer. The method additionally includes generating a followup question by a neural network and repeating said retrieving, assigning, outputting and generating steps for the followup question, responsive to the classification being including the intermediate information.

## Claims

### Claim 1

A computer-implemented method for generating following up questions for multi-hop bridge-type question answering, the method comprising:
retrieving a premise for an input multi-hop bridge-type question;
assigning, by a three-way neural network based controller, a classification of the premise against the input multi-hop bridge-type question as being any of irrelevant, including a final answer, or including intermediate information;
outputting the final answer in relation to a first hop of the multi-hop bridge-type question responsive to the classification being including the final answer; and
generating a followup question by a neural network and repeating said retrieving, assigning, outputting and generating steps for the followup question, responsive to the classification being including the intermediate information.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising retrieving another premise for the input multi-hop bridge-type question and repeating said assigning, outputting, and generating steps, responsive to the classification being irrelevant.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the assigning neural network is trained using one or more cross-entropy losses for ternary classification.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein said generating step generates the followup question based on the input multi-hop bridge-type question and the retrieved premise for the input multi-hop bridge-type question.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the generating neural network comprises a sequence-to-sequence model having a decoder configured to selectively generate a word from a fixed vocabulary or copy a word from the input multi-hop bridge-type question.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the assigning step comprises a neural network including one or more self-attention layers.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein said outputting step is performed by a neural network that is trained for single hop question answering.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the input multi-hop bridge-type question and the followup question are in natural text.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising controlling a hardware object to perform a function based on the final answer.

### Claim 10

A computer program product for generating following up questions for multi-hop bridge-type question answering, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
retrieving a premise for an input multi-hop bridge-type question;
assigning, by a three-way neural network based controller, a classification of the premise against the input multi-hop bridge-type question as being any of irrelevant, including a final answer, or including intermediate information;
outputting the final answer in relation to a first hop of the multi-hop bridge-type question responsive to the classification being including the final answer; and
generating a followup question by a neural network and repeating said retrieving, assigning, outputting and generating steps for the followup question, responsive to the classification being including the intermediate information.

### Claim 11 (depends on 10)

The computer program product of claim 10, further comprising retrieving another premise for the input multi-hop bridge-type question and repeating said assigning, outputting, and generating steps, responsive to the classification being irrelevant.

### Claim 12 (depends on 10)

The computer program product of claim 10, wherein the assigning neural network is trained using one or more cross-entropy losses for ternary classification.

### Claim 13 (depends on 10)

The computer program product of claim 10, wherein said generating step generates the followup question based on the input multi-hop bridge-type question and the retrieved premise for the input multi-hop bridge-type question.

### Claim 14 (depends on 10)

The computer program product of claim 10, wherein the generating neural network comprises a sequence-to-sequence model having a decoder configured to selectively generate a word from a fixed vocabulary or copy a word from the input multi-hop bridge-type question.

### Claim 15 (depends on 10)

The computer program product of claim 10, wherein the assigning step comprises a neural network including one or more self-attention layers.

### Claim 16 (depends on 10)

The computer program product of claim 10, wherein said outputting step is performed by a neural network that is trained for single-hop question answering.

### Claim 17 (depends on 10)

The computer program product of claim 10, wherein the input multi-hop bridge-type question and the followup question are in natural text.

### Claim 18 (depends on 10)

The computer program product of claim 10, further comprising controlling a hardware object to perform a function based on the final answer.

### Claim 19

A computer processing system for generating following up questions for multi-hop bridge-type question answering, the computer processing system comprising:
a memory device for storing program code; and
a processor device, operatively coupled to the memory device, for running the program code to
retrieve a premise for an input multi-hop bridge-type question;
assign, using a three-way neural network based controller, a classification of the premise against the input multi-hop bridge-type question as being any of irrelevant, including a final answer, or including intermediate information;
output the final answer in relation to a first hop of the multi-hop bridge-type question responsive to the classification being including the final answer; and
generate a followup question using a neural network and repeat the running of the program code to for the followup question, responsive to the classification being including the intermediate information.

### Claim 20 (depends on 19)

The computer processing system of claim 19, wherein the assigning neural network is trained using one or more cross-entropy losses for ternary classification.
