# US 2022/0277145 A1 · Domain Context Ellipsis Recovery for Chatbot

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277145 A1
- Document ID: US20220277145A1
- Status: Published application
- Kind: A1
- Application number: 17/631,666
- Filed: 2020-06-17
- Published: 2022-09-01
- Earliest claimed: 2019-09-12
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Pingping Lin; Ruihua SONG; Lei DING; Yue LIU; Min ZENG
- Web page: https://patentgrep.com/patent/US20220277145A1

## CPC

- G06F 40/35: Discourse or dialogue representation
- G06F 16/3329: Natural language query formulation
- G06F 40/253: Grammatical analysis; Style critique
- H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Methods and apparatuses for performing context completion to messages in a session are provided in the present disclosure. A message may be obtained. It may be detected that there exists context ellipsis in the message. It may be determined whether the message is retained in the current domain of the session. In response to determining that the message is retained in the current domain, a complementary text for recovering the context ellipsis may be selected in the current domain. A completed message may be generated based on the message and the complementary text.

## Claims

### Claim 1

A method for performing context completion to messages in a session, comprising
obtaining a message;
detecting that there exists context ellipsis in the message;
determining whether the message is retained in the current domain of the session;
in response to determining that the message is retained in the current domain, selecting, in the current domain, a complementary text for recovering the context ellipsis; and
generating a completed message based on the message and the complemental y text.

### Claim 2 (depends on 1)

The method of claim 1; wherein the detecting that there exists context ellipsis in the message comprises:
extracting a topic and/or predicate from the message; and
determining that there exists topic ellipsis or predicate ellipsis in the message based on a result of the extracting.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
adding the extracted topic and/or predicate into a domain topic flow and/or domain predicate flow of the current domain respectively.

### Claim 4 (depends on 2)

The method of claim 2, further comprising:
determining session level information associated with the extracted topic and/or predicate; and
adding the session level information into a domain topic flow and/or domain predicate flow of the current domain respectively.

### Claim 5 (depends on 2)

The method of claim 2, wherein,
the extracting is performed based at least on a pre-established knowledge graph.

### Claim 6 (depends on 2)

The method of claim 2, wherein the determining whether the message is retained in the current domain of the session comprises:
in response to determining that there exists the topic ellipsis in the message, determining whether the message is retained in the current domain through a domain retaining classifier.

### Claim 7 (depends on 6)

The method of claim 6, wherein,
the domain retaining classifier is based on a recurrent neural network (RNN) model, and
the domain retaining classifier is obtained, through knowledge distillation, from a classifier which is based on a bidirectional encoder representations from transformers (BERT) model.

### Claim 8 (depends on 2)

The method of claim 2, wherein the determining whether the message is retained in the current domain of the session comprises:
in response to determining that there exists the predicate ellipsis in the message, determining whether the message is retained in the current domain based on a topic included in the message.

### Claim 9 (depends on 3)

The method of claim 3, wherein the selecting a complementary text comprises:
in response to determining that there exists the topic ellipsis in the message, selecting the complementary text from the domain topic flow; or
in response to determining that there exists the predicate ellipsis in the message, selecting the complementary text from the domain predicate flow.

### Claim 10 (depends on 3)

The method of claim 3, wherein the selecting a complementary text comprises:
selecting the complementary text from a plurality of candidate texts in the domain topic flow and/or domain predicate flow, through at least one of convolutional neural network (CNN)-based ranking and regression-based ranking.

### Claim 11 (depends on 10)

The method of claim 10, wherein,
the CNN-based ranking is performed according to at least one of the following information of a candidate text:
text similarity between the candidate text and the message,
pointwise mutual information (PMI) between the candidate text and the message,
PMI between the candidate text and a synonymous representation of the message, and
PMI between a synonymous representation of the candidate text and the message; and
the regression-based ranking is performed according to at least one of the following features of a candidate text:
a frequency that the candidate text has occurred in the session,
whether the candidate text occurred in the last message,
whether the candidate text occurred in the last response,
the number of turns having passed since the candidate text occurred last time in the session,
PMI between the candidate text and the message, and
text similarity between the candidate text and the message.

### Claim 12 (depends on 1)

The method of claim 1, wherein the generating a completed message comprises:
generating at least one candidate completed message through placing the complementary text at different positions of the message; and
selecting the completed message from the at least one candidate completed message through a language model.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
in response to determining that the message is not retained in the current domain, determining a domain corresponding to the message;
selecting, in the domain corresponding to the message, a complementary text for recovering the context ellipsis; and
generating a completed message based on the message and the complementary text.

### Claim 14

An apparatus for performing context completion to messages in a session, comprising:
a message obtaining module, for obtaining a message;
a context ellipsis detecting module, for detecting that there exists context ellipsis in the message;
a domain retaining determining module, for determining whether the message is retained in the current domain of the session;
a complementary text selecting module, for in response to determining that the message is retained in the current domain, selecting, in the current domain, a complementary text for recovering the context ellipsis; and
a completed message generating module, for generating a completed message based on the message and the complementary text.

### Claim 15

An apparatus for performing context completion to messages in a session, comprising:
at least one processor; and
a memory storing computer-executable instructions that, when executed, cause the at least one processor to:
obtain a message,
detect that there exists context ellipsis in the message,
determine whether the message is retained in the current domain of the session,
in response to determining that the message is retained in the current domain, select, in the current domain, a complementary text for recovering the context ellipsis, and
generate a completed message based on the message and the complementary text.
