# US 2023/0056772 A1 · Influence function in machine learning for interpretation of lengthy and noisy documents

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0056772 A1
- Document ID: US20230056772A1
- Status: Published application
- Kind: A1
- Application number: 17/408,944
- Filed: 2021-08-23
- Published: 2023-02-23
- Earliest claimed: 2021-08-23
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Yada Zhu; Wei Zhang; Guangnan Ye; Xiaodong Cui
- Web page: https://patentgrep.com/patent/US20230056772A1

## CPC

- G06Q 30/0202: Market predictions or forecasting for commercial activities
- G06F 40/30: Semantic analysis
- G06N 3/08: Learning methods
- G06Q 30/0201: Market modelling; Market analysis; Collecting market data
- G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A neural network to predict a future indicator of a given entity can be trained based on historical earnings call data and historical market data. An earnings call transcript can be received, from which to predict the future indicator. Preprocessing can be performed using a natural language processing (NLP) technique to select candidate sentences from the earnings call transcript. For a candidate sentence in the candidate sentences, and using the trained neural network, a sentence gradient can be determined, which is indicative of sensitivity of the trained neural network to the candidate sentence. Based on the determined sentence gradient associated with each of the candidate sentences, an explanation of the trained neural network's predicted future indicator can be provided.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving historical earnings call data;
receiving historical market data;
training a neural network to predict a future indicator of a given entity based on the historical earnings call data and the historical market data;
receiving an earnings call transcript, from which to predict the future indicator;
preprocessing the earnings call transcript using a natural language processing (NLP) technique to select candidate sentences from the earnings call transcript;
for a candidate sentence in the candidate sentences and using the trained neural network,
determining a sentence gradient indicative of sensitivity of the trained neural network to the candidate sentence; and
based on the determined sentence gradient associated with each of the candidate sentences, providing an explanation of the trained neural network's predicted future indicator.

### Claim 2 (depends on 1)

The method of claim 1, wherein the preprocessing the earnings call transcript using a natural language processing (NLP) technique to select candidate sentences from the earnings call transcript, includes:
performing a term frequency-inverse document frequency (TF-IDF) analytics on the historical earnings call data, the TF-IDF analytics extracting keywords from the historical earnings call data, the keywords extracted according to an industry category associated with the given entity;
for each sentence in the earnings call transcript,
computing a distance measurement based on comparing words in the sentence with the keywords, and ranking the sentences based on the computed distance measurement; and
selecting as the candidate sentences, top k-ranked sentences, wherein k is predefined.

### Claim 3 (depends on 1)

The method of claim 1, wherein the determining a sentence gradient includes performing an influence function (IF) that measures pairwise sensitivity between the candidate sentence input to the neural network and the predicted future indicator output by the neural network.

### Claim 4 (depends on 1)

The method of claim 1, wherein the determining a sentence gradient includes tracing how a loss on a point changes during a training process of the neural network responsive to the candidate sentence being utilized.

### Claim 5 (depends on 1)

The method of claim 1, further including causing the candidate sentences to be presented on a graphical user interface.

### Claim 6 (depends on 5)

The method of claim 5, further including causing the earnings call transcript to be presented with the candidate sentences highlighted on the graphical user interface.

### Claim 7 (depends on 6)

The method of claim 6, further including causing the graphical user interface to allow a user to select a candidate sentence from the presented candidate sentences and to further highlight sentences appearing before and after the selected candidate sentence.

### Claim 8 (depends on 1)

The method of claim 1, wherein the entity and the future indicator to predict are input by a user.

### Claim 9 (depends on 1)

The method of claim 1, wherein the candidate sentences form a summarization of the earnings call transcript.

### Claim 10

A system comprising:
a processor;
a memory device coupled with the processor;
the processor configured to at least:
receive historical earnings call data;
receive historical market data;
train a neural network to predict a future indicator of a given entity based on the historical earnings call data and the historical market data;
receive an earnings call transcript, from which to predict the future indicator;
preprocess the earnings call transcript using a natural language processing (NLP) technique to select candidate sentences from the earnings call transcript;
for a candidate sentence in the candidate sentences and using the trained neural network, determine a sentence gradient indicative of sensitivity of the trained neural network to the candidate sentence; and
based on the determined sentence gradient associated with each of the candidate sentences, provide an explanation of the trained neural network's predicted future indicator.

### Claim 11 (depends on 10)

The system of claim 10, wherein in preprocessing the earnings call transcript using a natural language processing (NLP) technique to select candidate sentences from the earnings call transcript, the processor is configured to:
perform a term frequency-inverse document frequency (TF-IDF) analytics on the historical earnings call data, the TF-IDF analytics extracting keywords from the historical earnings call data, the keywords extracted according to an industry category associated with the given entity;
for each sentence in the earnings call transcript,
compute a distance measurement based on comparing words in the sentence with the keywords, and rank the sentences based on the computed distance measurement; and
select as the candidate sentences, top k-ranked sentences, wherein k is predefined.

### Claim 12 (depends on 10)

The system of claim 10, wherein the determining a sentence gradient includes performing an influence function (IF) that measures pairwise sensitivity between the candidate sentence input to the neural network and the predicted future indicator output by the neural network.

### Claim 13 (depends on 10)

The system of claim 10, wherein the determining a sentence gradient includes tracing how a loss on a point changes during a training process of the neural network responsive to the candidate sentence being utilized.

### Claim 14 (depends on 10)

The system of claim 10, wherein the processor is further configured to cause the candidate sentences to be presented on a graphical user interface with associated importance scores based on the sentence gradient.

### Claim 15 (depends on 14)

The system of claim 14, wherein the processor is further configured to cause the earnings call transcript to be presented with the candidate sentences highlighted on the graphical user interface.

### Claim 16 (depends on 15)

The system of claim 15, wherein the processor is further configured to cause the graphical user interface to allow a user to select a candidate sentence from the presented candidate sentences and to highlight sentences appearing before and after the selected candidate sentence on the presented earnings call transcript.

### Claim 17

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a device to cause the device to:
receive historical earnings call data;
receive historical market data;
train a neural network to predict a future indicator of a given entity based on the historical earnings call data and the historical market data;
receive an earnings call transcript, from which to predict the future indicator;
preprocess the earnings call transcript using a natural language processing (NLP) technique to select candidate sentences from the earnings call transcript;
for a candidate sentence in the candidate sentences and using the trained neural network,
determine a sentence gradient indicative of sensitivity of the trained neural network to the candidate sentence; and
based on the determined sentence gradient associated with each of the candidate sentences, provide an explanation of the trained neural network's predicted future indicator.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein in preprocessing the earnings call transcript using a natural language processing (NLP) technique to select candidate sentences from the earnings call transcript, the device is further caused to:
perform a term frequency-inverse document frequency (TF-IDF) analytics on the historical earnings call data, the TF-IDF analytics extracting keywords from the historical earnings call data, the keywords extracted according to an industry category associated with the given entity;
for each sentence in the earnings call transcript,
compute a distance measurement based on comparing words in the sentence with the keywords, and ranking the sentences based on the computed distance measurement; and
select as the candidate sentences, top k-ranked sentences, wherein k is predefined.

### Claim 19 (depends on 17)

The computer program product of claim 17, wherein the determining a sentence gradient includes performing an influence function (IF) that measures pairwise sensitivity between the candidate sentence input to the neural network and the predicted future indicator output by the neural network.

### Claim 20 (depends on 17)

The computer program product of claim 17, wherein the determining a sentence gradient includes tracing how a loss on a point changes during a training process of the neural network responsive to the candidate sentence being utilized.