# US 2021/0240917 A1 · Producing explainable rules via deep learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240917 A1
- Document ID: US20210240917A1
- Status: Published application
- Kind: A1
- Application number: 16/780,721
- Filed: 2020-02-03
- Published: 2021-08-05
- Earliest claimed: 2020-02-03
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Prithviraj Sen; Siddhartha Brahma; Yunyao Li; Laura Chiticariu; Rajasekar Krishnamurthy; Shivakumar Vaithyanathan; Marina Danilevsky Hailpern
- Web page: https://patentgrep.com/patent/US20210240917A1

## CPC

- G06F 40/56: Natural language generation
- G06F 18/2155: {characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling}
- G06F 18/24133: {Distances to prototypes}
- G06F 40/16: Automatic learning of transformation rules, e.g. from examples
- G06F 40/216: using statistical methods
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 40/30: Semantic analysis
- G06F 40/44: Statistical methods, e.g. probability models
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/048: Activation functions
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- G06N 20/00: Machine learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method according to one embodiment includes receiving, at a deep neural network (DNN), a plurality of sentences each having an associated label; training the DNN, utilizing the plurality of sentences and associated labels; and producing a linguistic expression (LE) utilizing the trained DNN.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving, at a deep neural network (DNN), a plurality of sentences each having an associated label;
training the DNN, utilizing the plurality of sentences and associated labels; and
producing a linguistic expression (LE) utilizing the trained DNN.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the DNN is refined via iterative training, where:
a subset of the plurality of sentences is input into the DNN,
each sentence within the subset is propagated through all layers of the DNN to determine a predicted label for the sentence,
the predicted label for the sentence is compared to the associated label for the sentence to determine a computed error,
the computed error is back-propagated through the DNN, and
the DNN is adjusted to minimize the computed error.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the DNN includes a plurality of intermediate layers, where each of the plurality of intermediate layers corresponds to a condition within the linguistic expression used to perform a determination of a label.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein each condition within a predictive model is derived from one of a plurality of intermediate layers within the DNN.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein when training of the DNN is complete, each intermediate layer of the DNN constitutes a condition for the linguistic expression.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the linguistic expression takes a sentence as input and produces the associated label as output.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the linguistic expression includes a human-readable model, such that the linguistic expression includes a word-based rule model that includes one or more conditions and that is understandable by a human user.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising applying the linguistic expression to one or more sentences that do not have an associated label, utilizing the DNN.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein for each of the one or more sentences that does not have an associated label, in response to determining that each of the one or more conditions of the linguistic expression is met, the associated label is assigned to a sentence.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein for each of the one or more sentences that does not have an associated label, in response to determining that one or more conditions of the linguistic expression is not met, an absence of the associated label is assigned to a sentence.

### Claim 11

A computer program product for producing explainable rules via deep learning, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:
receiving, utilizing the processor at a deep neural network (DNN), a plurality of sentences each having an associated label;
training the DNN, utilizing the processor and the plurality of sentences and associated labels; and
producing a linguistic expression (LE) utilizing the trained DNN and the processor.

### Claim 12 (depends on 11)

The computer program product of claim 11, wherein the DNN is refined via iterative training, where:
a subset of the plurality of sentences is input into the DNN,
each sentence within the subset is propagated through all layers of the DNN to determine a predicted label for the sentence,
the predicted label for the sentence is compared to the associated label for the sentence to determine a computed error,
the computed error is back-propagated through the DNN, and
the DNN is adjusted to minimize the computed error.

### Claim 13 (depends on 11)

The computer program product of claim 11, wherein the DNN includes a plurality of intermediate layers, where each of the plurality of intermediate layers corresponds to a condition within the linguistic expression used to perform a determination of a label.

### Claim 14 (depends on 11)

The computer program product of claim 11, wherein each condition within a predictive model is derived from one of a plurality of intermediate layers within the DNN.

### Claim 15 (depends on 11)

The computer program product of claim 11, wherein when training of the DNN is complete, each intermediate layer of the DNN constitutes a condition for the linguistic expression.

### Claim 16 (depends on 11)

The computer program product of claim 11, wherein the linguistic expression takes a sentence as input and produces the associated label as output.

### Claim 17 (depends on 11)

The computer program product of claim 11, wherein the linguistic expression includes a human-readable model, such that the linguistic expression includes a word-based rule model that includes one or more conditions and that is understandable by a human user.

### Claim 18 (depends on 11)

The computer program product of claim 11, further comprising applying the linguistic expression to one or more sentences that do not have an associated label, utilizing the DNN.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein for each of the one or more sentences that does not have an associated label, in response to determining that each of the one or more conditions of the linguistic expression is met, the associated label is assigned to a sentence.

### Claim 20 (depends on 18)

The computer program product of claim 18, wherein for each of the one or more sentences that does not have an associated label, in response to determining that one or more conditions of the linguistic expression is not met, an absence of the associated label is assigned to a sentence.

### Claim 21

A system, comprising:
a processor; and
logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to:
receive, at a deep neural network (DNN), a plurality of sentences each having an associated label;
train the DNN, utilizing the plurality of sentences and associated labels; and
produce a linguistic expression (LE) utilizing the trained DNN.

### Claim 22

A computer-implemented method, comprising:
identifying a collection of sentences having labels;
inputting the sentences into a deep neural network (DNN) having several layers, the DNN analyzing the sentences and producing linguistic expressions (LEs) corresponding to the sentences; and
in view of input from a human expert, modifying the linguistic expressions, wherein the modified linguistic expressions form a predictive model that, given an unlabeled sentence, predicts one or more labels for the unlabeled sentence.

### Claim 23 (depends on 22)

The computer-implemented method of claim 22, wherein the predictive model undergoes iterative refinement in view of its quality.

### Claim 24 (depends on 22)

The computer-implemented method of claim 22, further comprising:
(a) choosing a subset of sentences and their corresponding labels from the collection of sentences;
(b) for each sentence in the subset, feeding the sentence as input to the DNN and propagating it through all of the DNN's layers until its predicted label is obtained as output from the DNN, at which point the predicted label is compared with its correct label, and a corresponding error is computed;
(c) backpropagating the computed error to update conditions included in the linguistic expressions; and
repeating steps (b) and (c) for one or more additional subsets in the collection of sentences.

### Claim 25 (depends on 22)

The computer-implemented method of claim 22, wherein the several layers of the DNN include an intermediate layer for each condition in the linguistic expressions.