# US 12,191,007 B2 · Human-in-the-loop interactive model training

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,191,007 B2
- Document ID: US12191007B2
- Status: Granted
- Kind: B2
- Application number: 16/618,656
- Filed: 2017-09-29
- Published: 2021-11-18
- Granted: 2025-01-07
- Earliest claimed: 2017-08-30
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Kai Chen; Eyal Oren; Hector Yee; James Wilson; Alvin Rajkomar; Michaela Hardt
- Web page: https://patentgrep.com/patent/US12191007B2

## CPC

- G06N 20/20: Ensemble learning
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Example embodiments relate to a method for training a predictive model from data. The method includes defining a multitude of predicates as binary functions operating on time sequences of the features or logical operations on the time sequences of the features. The method also includes iteratively training a boosting model by generating a number of new random predicates, scoring all the new random predicates by weighted information gain with respect to a class label associated with a prediction of the boosting model, selecting a number of the new random predicates with the highest weighted information gain and adding them to the boosting model, computing weights for all the predicates in the boosting model, removing one or more of the selected new predicates with the highest information gain from the boosting model in response to input from an operator. The method may include repeating the prior steps a plurality of times.

## Claims

### Claim 1

A computer-implemented method of training a predictive model from data comprising a multitude of features, each feature associated with a real value and a time component, comprising the steps of executing the following instructions in a processor of the computer:
a) defining a multitude of predicates as binary functions operating on time sequences of the features or logical operations on the time sequences of the features;
b) iteratively training a boosting model by performing the following:
1) Generating a number of new random predicates as binary functions operating on at least one of (i) time sequences of the features or (ii) logical operations on the time sequences of the features;
2) Scoring all the new random predicates by weighted information gain with respect to a class label associated with a prediction of the boosting model;
3) Selecting, from the new random predicates, a number of the new random predicates that are the highest with respect to their weighted information gain scores and adding them to the boosting model;
4) Computing weights for all the predicates in the boosting model;
5) Removing one or more of the selected number of the new random predicates from the boosting model in response to input from an operator; and
6) Repeating the performance of steps 1, 2, 3, 4 and 5 a plurality of times and thereby generating a final iteratively trained boosting model.

### Claim 2 (depends on 1)

The method of claim 1, further comprising the step of c) evaluating the final iteratively trained boosting model.

### Claim 3 (depends on 2)

The method of claim 2, wherein the evaluation step (c) comprises evaluating the final iteratively trained boosting model for at least one of accuracy, complexity, or trustworthiness.

### Claim 4 (depends on 1)

The method of claim 1, wherein the data is in a tuple format of the type {X, xi, ti} where X is the name of feature, xi is a real value of the feature and ti is a time component for the real value xi, and wherein the predicates are defined as binary functions operating on at least one of (i) sequences of tuples or (ii) logical operations on sequences of the tuples.

### Claim 5 (depends on 4)

The method of claim 4, wherein the sequences of tuples are defined by time periods selected from the group consisting of 1 or more days, 1 or more hours, 1 or more minutes, or 1 or more months.

### Claim 6 (depends on 1)

The method of claim 1, wherein the data comprises electronic health record data for a multitude of patients.

### Claim 7 (depends on 1)

The method of claim 1, wherein the method further comprises the step of dividing the predicates into groups based on understandability, namely a first group of relatively more human understandable predicates and a second group of relatively less human understandable predicates and wherein the new random predicates are selected from the first group.

### Claim 8 (depends on 7)

The method of claim 7, wherein the data comprises electronic health record data for a multitude of patients, and wherein the set of predicates are represented in a manner to show the subject matter or source within the electronic health record data of the predicates.

### Claim 9 (depends on 8)

The method of claim 8, wherein the predicates comprise an existence predicate returning a result of 0 or 1 depending on whether a feature exists in the electronic health record data for a given patient in the multitude of patients; and a counts predicate returning a result of 0 or 1 depending on the number of counts of a feature in the electronic health record data for a given patient in the multitude of patients relative to a numeric parameter C.

### Claim 10 (depends on 1)

The method of claim 1, wherein step b) 5) further comprises the step of graphically representing the predicates currently in the boosting model and providing the operator with the ability to remove one or more of the predicates.

### Claim 11 (depends on 10)

The method of claim 10, further comprising the step of graphically representing the weights computed for each of the predicates in step b) 4).

### Claim 12 (depends on 1)

The method of claim 1, further comprising the step of graphically representing a set of predicates added to the boosting model after each of the iterations of step b) 6).

### Claim 13 (depends on 1)

The method of claim 1, wherein step b) further comprises the step of providing the operator with the ability to define a predicate during model training.

### Claim 14 (depends on 1)

The method of claim 1, wherein step b) further comprises the step of removing redundant predicates.

### Claim 15 (depends on 1)

The method of claim 1, further comprising the step of ranking the predicates selected in step b) 3).

### Claim 16 (depends on 1)

The method of claim 1, further comprising the step of generating statistics of predicates in the boosting model and presenting them to the operator.

### Claim 17 (depends on 1)

The method of claim 1, wherein in step b) 5) the one or more predicates are removed which are not causally related to the prediction of the boosting model.

### Claim 18

A computer-implemented method of training a predictive model from electronic health record data for a multitude of patients, the data comprising a multitude of features, each feature associated with real values and a time component, wherein the data is in a tuple format of the type {X, xi, ti} where X is the name of feature, xi is a real value of the feature and ti is a time component for the real value xi, comprising the steps of implementing the following instructions in a processor of the computer:
a) defining a multitude of predicates as at least one of (i) binary functions operating on sequences of the tuples or (ii) logical operations on the sequences of the tuples;
b) dividing the multitude of predicates into groups based on understandability, namely a first group of relatively more human understandable predicates and a second group of relatively less human understandable predicates;
c) iteratively training a boosting model by performing the following:
1) Generating a number of new random predicates from the first group of predicates as binary functions operating on at least one of (i) sequences of the tuples or (ii) logical operations on the sequences of the tuples;
2) Scoring all the new random predicates by weighted information gain with respect to a class label associated with a prediction of the boosting model;
3) Selecting, from the new random predicates, a number of the new random predicates that are the highest with respect to their weighted information gain scores and adding them to the boosting model;
4) Computing weights for all the predicates in the boosting model;
5) Removing one or more of the selected number of the new random predicates from the boosting model in response to input from an operator; and
6) Repeating the performance of steps 1, 2, 3, 4 and 5 a plurality of times and thereby generating a final iteratively trained boosting model.

### Claim 19 (depends on 18)

The method of claim 18, further comprising the step d) of evaluating the final iteratively trained boosting model.

### Claim 20

A workstation for providing operator input into iteratively training a boosting model, wherein the workstation comprises an interface and a processor, and wherein the processor is configured to perform operations comprising:
1) Generating a number of new random predicates as binary functions operating on at least one of (i) time sequences of input features or (ii) logical operations on the time sequences of the input features;
2) Scoring all the new random predicates by weighted information gain with respect to a class label associated with a prediction of the boosting model;
3) Selecting, from the new random predicates, a number of the new random predicates that are the highest with respect to their weighted information gain scores and adding them to the boosting model;
4) Computing weights for all the predicates in the boosting model;
5) Removing one or more of the selected number of the new random predicates from the boosting model in response to input from an operator, wherein the input is received by way of the interface; and
6) Repeating the performance of steps 1, 2, 3, 4 and 5 a plurality of times and thereby generating a final iteratively trained boosting model.

### Claim 21 (depends on 20)

The workstation of claim 20, wherein predicates are defined as at least one of (i) binary functions operating on sequences of features having both a real value component and a time component or (ii) logical operations on sequences of the features.

### Claim 22 (depends on 20)

The workstation of claim 20, wherein the operations further comprise the step of graphically representing on the interface a set of predicates added to the boosting model after each of the iterations of step 6).

### Claim 23 (depends on 20)

The workstation of claim 20, wherein the step (i) of automatically generating the plurality of additional predicates comprises:
(a) generating candidate predicates by a pseudo-random algorithm;
(b) scoring the candidate predicates for weighted information gain in the boosting model; and
(c) selecting the additional predicates from the candidate predicates based on the scores.

### Claim 24 (depends on 23)

The workstation of claim 23, wherein the output of each predicate is a binary value.

### Claim 25 (depends on 23)

The workstation of claim 23, wherein each sample in the training data is formatted as a plurality of data items having a tuple format of the type {X, xi, ti}, wherein xi indicates the value of feature X at a time ti, and i labels the tuple of the sample, and wherein each predicate is defined as a function performed on a plurality of data items of the sample.

### Claim 26 (depends on 23)

The workstation according to claim 23, wherein the training data comprises electronic health record data for a plurality of patients.

### Claim 27 (depends on 23)

The workstation of claim 23, wherein each predicate is a function of a part of the sample relating to a single corresponding one of the features.

### Claim 28 (depends on 23)

The workstation according to claim 23, wherein the features are each associated with a corresponding one of a set of human understandable categories or groups, and step (iii) of displaying a plurality of the set of predicates includes displaying grouped together the predicates which are functions of data relating to features of each category or group.

### Claim 29 (depends on 23)

The workstation of claim 23, wherein step (iii) of displaying a plurality of the set of predicates includes displaying a respective weight value of the boosting model.

### Claim 30 (depends on 23)

The workstation of claim 23, further comprising evaluating the accuracy of the boosting model in predicting the label using a validation sub-set of the training data.

### Claim 31 (depends on 23)

The workstation of claim 23, wherein the rejected one or more of the updated set of predicates are not causally related to a prediction of the boosting model.

### Claim 32

A computer-implemented method of generating a predictive model from training data, the predictive model being for predicting a label based on input data which, for each of a plurality of features X, indicates a value x of the feature at each of a plurality of times, and the training data comprising a plurality of samples, each sample indicating the value of one or more of the features at each of one of more times and a corresponding label; the method comprising implementing the following steps as instructions with a processor:
defining a set of predicates, each predicate being a function which generates an output when applied to time sequences of the features or logical combinations of the time sequences of the features;
generating a boosting model, the boosting model receiving as input the respective outputs of each of the set of predicates when applied to the samples of the training data; and
performing a plurality of times, the sequence of steps of:
(i) automatically generating a plurality of additional predicates as binary functions operating on at least one of (i) time sequences of the features or (ii) logical operations on the time sequences of the features;
(ii) adding the plurality of additional predicates to predicates already in the boosting model to form an updated set of predicates;
(iii) displaying a plurality of the updated set of predicates; and
(iv) receiving data input rejecting one or more of the updated set of predicates; and
(v) removing the rejected one or more predicates from the updated set of predicates.

### Claim 33 (depends on 32)

The method of claim 32, wherein the additional predicates comprise at least one of existence predicates which are each indicative of a specific feature taking a value in a specific range at least one time, and count predicates which are each indicative of a specific feature taking a value in a specific range at more than, less than, or equal to a specific number of times C.