# US 2023/0147359 A1 · Open input classifier with entailment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0147359 A1
- Document ID: US20230147359A1
- Status: Published application
- Kind: A1
- Application number: 17/984,694
- Filed: 2022-11-10
- Published: 2023-05-11
- Earliest claimed: 2021-11-10
- Assignees: Woebot Labs, Inc.
- Applicants: Woebot Labs, Inc.
- Inventors: Casey Sackett; Andi Molyneux; Joe Gallagher
- Web page: https://patentgrep.com/patent/US20230147359A1

## CPC

- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 40/35: Discourse or dialogue representation
- G06F 40/279: Recognition of textual entities
- G06F 40/30: Semantic analysis
- G06F 18/20: Analysing
- G06F 2218/10: by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A natural language processing combination classifier is disclosed, leveraging an entailment classifier and optionally at least one of a pattern matching classifier and a trained machine learning (ML) classifier. Each of the different types of classifiers can be used to identify different categories of matches. For example, a pattern matching (e.g., regular expression) classifier may identify exact matches, an entailment classifier can obtain context-specific classifications based on likelihood of entailment to comparison data, and the SML classifier can obtain potential matches based on large-scale supervised training. The comparison data for the entailment classifier can be generated from small datasets, and can be readily updated without the need for retraining any machine learning models. Different types of classifiers can be processed using different logic to provide a user with the most appropriate response given a circumstance and given the user’s open input.

## Claims

### Claim 1

A system, comprising:
one or more data processors; and
a non-transitory computer-readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to perform operations including:
receiving an input phrase associated with a topic;
accessing entailment comparison data, wherein the entailment comparison data includes a plurality of example inputs associated with a plurality of topics, wherein the plurality of topics includes the topic;
generating classification data for the input phrase, wherein generating the classification data includes applying an entailment classifier to the input phrase using the entailment comparison data, wherein the generated classification data is indicative of the topic; and
generating a response based at least in part on the generated classification data.

### Claim 2 (depends on 1)

The system of claim 1, wherein the operations further comprise presenting an input phrase prompt, wherein receiving the input phrase is in response to presenting the input phrase prompt, and wherein presenting the input phrase prompt includes presenting an input field for free text.

### Claim 3 (depends on 1)

The system of claim 1, wherein applying the entailment classifier to the input phrase using the entailment comparison data includes:
generating, for each of the plurality of example inputs of the entailment comparison data, an entailment value indicative of whether the input phrase i) implies the respective example input; ii) contradicts the respective example input; or iii) is neutral with respect to the respective example input;
generating, for each entailment value, a confidence score;
identifying a set of one or more relevant example inputs from the plurality of example inputs based at least in part on the entailment values and the confidence scores;
identifying a set of one or more matching topics from the plurality of topics based at least in part on the set of one or more relevant examples inputs; and
generating entailment classification data based at least in part on the set of one or more matching topics.

### Claim 4 (depends on 1)

The system of claim 1, wherein applying the entailment classifier to the input phrase using the entailment comparison data includes:
generating, for each of the plurality of example inputs of the entailment comparison data, an entailment value indicative of whether the input phrase i) implies the respective example input; ii) contradicts the respective example input; or iii) is neutral with respect to the respective example input;
generating, for each entailment value, a confidence score; and
determining that the confidence score for each entailment value is below a threshold score; and
wherein generating the classification data further includes applying a supervised machine learning (SML) classifier to the input phrase to generate SML classification data in response to determining that the confidence score for each entailment value is below the threshold score, and wherein the generated classification data is based at least in part on the SML classification data.

### Claim 5 (depends on 1)

The system of claim 1, wherein generating the classification data further includes applying the input phrase to at least one of a pattern matching classifier and a supervised machine learning (SML) classifier.

### Claim 6 (depends on 5)

The system of claim 5, wherein generating the classification data includes:
applying the pattern matching classifier to the input phrase to generate pattern matching classification data based on one or more preset patterns; and
determining that the pattern matching classification data is indicative that the input phrase does not match the one or more preset patterns;
wherein applying the entailment classifier to the input phrase generates entailment classification data, wherein applying the entailment classifier to the input phrase occurs in response to determining that the pattern matching classification data is indicative that the input phrase does not match the one or more preset patterns, wherein the entailment classification data is indicative of the topic, and wherein the generated classification data is based at least in part on the entailment classification data.

### Claim 7 (depends on 5)

The system of claim 5, wherein generating the classification data includes:
generating entailment classification data in response to applying the entailment classifier to the input phrase;
generating at least one of pattern matching classification data and SML classification data using the pattern matching classifier and SML classifier, respectively; and
identifying the topic using the entailment classification data and the at least one of the pattern matching classification data and the SML classification data.

### Claim 8 (depends on 7)

The system of claim 7, wherein identifying the topic using the entailment classification data and the at least one of the pattern matching classification data and the SML classification data includes:
determining one or more possible topics using first data, wherein the first data is selected from the group consisting of the entailment classification data and the at least one of the pattern matching classification data and the SML classification data; and
selecting the topic from the one or more possible topics using second data, wherein the second data is selected from the group consisting of the entailment classification data and the at least one of the pattern matching classification data and the SML classification data, and wherein the second data is different from the first data.

### Claim 9 (depends on 5)

The system of claim 5, wherein the SML classifier is a bidirectional encoder representations from transformers (BERT) classifier.

### Claim 10 (depends on 1)

The system of claim 1, wherein generating the classification data further includes applying the input phrase to at least one of a pattern matching classifier and a supervised machine learning (SML) classifier to generate pattern matching classification data and SML classification data, respectively; and wherein accessing the comparison entailment data is based at least in part on at least one of the pattern matching classification data and the SML classification data.

### Claim 11 (depends on 1)

The system of claim 1, wherein accessing the entailment comparison data includes selecting the entailment comparison data from a plurality of potential entailment comparison datasets, wherein each potential entailment comparison dataset is associated with a unique subject matter.

### Claim 12 (depends on 1)

The system of claim 1, wherein each of the plurality of topics of the entailment comparison data is associated with psychological therapy, and wherein generating the response facilitates delivery of psychological therapy to a user.

### Claim 13 (depends on 12)

The system of claim 12, wherein generating the classification data further includes applying the input phrase a supervised machine learning (SML) classifier that has been trained using training data that is not associated with psychological therapy.

### Claim 14

A computer-implemented method, comprising:
executed on the one or more data processors, cause the one or more data processors to perform operations including:
receiving an input phrase associated with a topic;
accessing entailment comparison data, wherein the entailment comparison data includes a plurality of example inputs associated with a plurality of topics, wherein the plurality of topics includes the topic;
generating classification data for the input phrase, wherein generating the classification data includes applying an entailment classifier to the input phrase using the entailment comparison data, wherein the generated classification data is indicative of the topic; and
generating a response based at least in part on the generated classification data.

### Claim 15 (depends on 14)

The method of claim 14, further comprising presenting an input phrase prompt, wherein receiving the input phrase is in response to presenting the input phrase prompt, and wherein presenting the input phrase prompt includes presenting an input field for free text.

### Claim 16 (depends on 14)

The method of claim 14, wherein applying the entailment classifier to the input phrase using the entailment comparison data includes:
generating, for each of the plurality of example inputs of the entailment comparison data, an entailment value indicative of whether the input phrase i) implies the respective example input; ii) contradicts the respective example input; or iii) is neutral with respect to the respective example input;
generating, for each entailment value, a confidence score;
identifying a set of one or more relevant example inputs from the plurality of example inputs based at least in part on the entailment values and the confidence scores;
identifying a set of one or more matching topics from the plurality of topics based at least in part on the set of one or more relevant examples inputs; and
generating entailment classification data based at least in part on the set of one or more matching topics.

### Claim 17 (depends on 14)

The method of claim 14, wherein applying the entailment classifier to the input phrase using the entailment comparison data includes:
generating, for each of the plurality of example inputs of the entailment comparison data, an entailment value indicative of whether the input phrase i) implies the respective example input; ii) contradicts the respective example input; or iii) is neutral with respect to the respective example input;
generating, for each entailment value, a confidence score; and
determining that the confidence score for each entailment value is below a threshold score; and
wherein generating the classification data further includes applying a supervised machine learning (SML) classifier to the input phrase to generate SML classification data in response to determining that the confidence score for each entailment value is below the threshold score, and wherein the generated classification data is based at least in part on the SML classification data.

### Claim 18 (depends on 14)

The method of claim 14, wherein generating the classification data further includes applying the input phrase to at least one of a pattern matching classifier and a supervised machine learning (SML) classifier.

### Claim 19 (depends on 18)

The method of claim 18, wherein generating the classification data includes:
applying the pattern matching classifier to the input phrase to generate pattern matching classification data based on one or more preset patterns; and
determining that the pattern matching classification data is indicative that the input phrase does not match the one or more preset patterns;
wherein applying the entailment classifier to the input phrase generates entailment classification data, wherein applying the entailment classifier to the input phrase occurs in response to determining that the pattern matching classification data is indicative that the input phrase does not match the one or more preset patterns, wherein the entailment classification data is indicative of the topic, and wherein the generated classification data is based at least in part on the entailment classification data.

### Claim 20 (depends on 18)

The method of claim 18, wherein generating the classification data includes:
generating entailment classification data in response to applying the entailment classifier to the input phrase;
generating at least one of pattern matching classification data and SML classification data using the pattern matching classifier and SML classifier, respectively; and
identifying the topic using the entailment classification data and the at least one of the pattern matching classification data and the SML classification data.

### Claim 21 (depends on 20)

The method of claim 20, wherein identifying the topic using the entailment classification data and the at least one of the pattern matching classification data and the SML classification data includes:
determining one or more possible topics using first data, wherein the first data is selected from the group consisting of the entailment classification data and the at least one of the pattern matching classification data and the SML classification data; and
selecting the topic from the one or more possible topics using second data, wherein the second data is selected from the group consisting of the entailment classification data and the at least one of the pattern matching classification data and the SML classification data, and wherein the second data is different from the first data.

### Claim 22 (depends on 18)

The method of claim 18, wherein the SML classifier is a bidirectional encoder representations from transformers (BERT) classifier.

### Claim 23 (depends on 14)

The method of claim 14, wherein generating the classification data further includes applying the input phrase to at least one of a pattern matching classifier and a supervised machine learning (SML) classifier to generate pattern matching classification data and SML classification data, respectively; and wherein accessing the comparison entailment data is based at least in part on at least one of the pattern matching classification data and the SML classification data.

### Claim 24 (depends on 14)

The method of claim 14, wherein accessing the entailment comparison data includes selecting the entailment comparison data from a plurality of potential entailment comparison datasets, wherein each potential entailment comparison dataset is associated with a unique subject matter.

### Claim 25 (depends on 14)

The method of claim 14, wherein each of the plurality of topics of the entailment comparison data is associated with psychological therapy, and wherein generating the response facilitates delivery of psychological therapy to a user.

### Claim 26 (depends on 25)

The method of claim 25, wherein generating the classification data further includes applying the input phrase a supervised machine learning (SML) classifier that has been trained using training data that is not associated with psychological therapy.

### Claim 27

A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to perform operations including:
receiving an input phrase associated with a topic;
accessing entailment comparison data, wherein the entailment comparison data includes a plurality of example inputs associated with a plurality of topics, wherein the plurality of topics includes the topic;
generating classification data for the input phrase, wherein generating the classification data includes applying an entailment classifier to the input phrase using the entailment comparison data, wherein the generated classification data is indicative of the topic; and
generating a response based at least in part on the generated classification data.

### Claim 28 (depends on 27)

The computer-program product of claim 27, wherein the operations further comprise presenting an input phrase prompt, wherein receiving the input phrase is in response to presenting the input phrase prompt, and wherein presenting the input phrase prompt includes presenting an input field for free text.

### Claim 29 (depends on 27)

The computer-program product of claim 27, wherein applying the entailment classifier to the input phrase using the entailment comparison data includes:
generating, for each of the plurality of example inputs of the entailment comparison data, an entailment value indicative of whether the input phrase i) implies the respective example input; ii) contradicts the respective example input; or iii) is neutral with respect to the respective example input;
generating, for each entailment value, a confidence score;
identifying a set of one or more relevant example inputs from the plurality of example inputs based at least in part on the entailment values and the confidence scores;
identifying a set of one or more matching topics from the plurality of topics based at least in part on the set of one or more relevant examples inputs; and
generating entailment classification data based at least in part on the set of one or more matching topics.

### Claim 30 (depends on 27)

The computer-program product of claim 27, wherein applying the entailment classifier to the input phrase using the entailment comparison data includes:
generating, for each of the plurality of example inputs of the entailment comparison data, an entailment value indicative of whether the input phrase i) implies the respective example input; ii) contradicts the respective example input; or iii) is neutral with respect to the respective example input;
generating, for each entailment value, a confidence score; and
determining that the confidence score for each entailment value is below a threshold score; and
wherein generating the classification data further includes applying a supervised machine learning (SML) classifier to the input phrase to generate SML classification data in response to determining that the confidence score for each entailment value is below the threshold score, and wherein the generated classification data is based at least in part on the SML classification data.