# US 2019/0378048 A1 · Time-factored performance prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378048 A1
- Document ID: US20190378048A1
- Status: Published application
- Kind: A1
- Application number: 16/004,096
- Filed: 2018-06-08
- Published: 2019-12-12
- Earliest claimed: 2018-06-08
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Mayank SHRIVASTAVA; Hui ZHOU; Pushpraj SHUKLA; Emre Hamit KOK; Sonal Prakash MANE; Dimitrios BRISIMITZIS
- Web page: https://patentgrep.com/patent/US20190378048A1

## CPC

- G06N 20/00: Machine learning
- G06F 16/90335: {Query processing}
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Training query intents are allocated for multiple training entities into training time intervals in a time series based on a corresponding query intent time for each training query intent. Training performance results for the multiple training entities are allocated into the training time intervals in the time series based on a corresponding performance time of each training performance result. A machine learning model for a training milestone of the time series is trained based on the training query intents allocated to a training time interval prior to the training milestone and the training performance results allocated to a training time interval after the training milestone. Target performance for the target entity for an interval after a target milestone in the time series is predicted by inputting to the trained machine learning model target query intents allocated to the target entity in a target time interval before the target milestone.

## Claims

### Claim 1

A method of predicting performance of a target entity using a machine learning model, the method comprising:
allocating training query intents generated from a query-URL click graph for multiple training entities into training time intervals in a time series based on a corresponding query intent time for each training query intent, the training time intervals in the time series being separated by training milestones in the time series;
allocating training performance results for the multiple training entities into the training time intervals in the time series based on a corresponding performance time of each training performance result;
training the machine learning model for a training milestone of the time series based on the training query intents allocated to a training time interval prior to the training milestone and the training performance results allocated to a training time interval after the training milestone;
allocating target query intents generated from the query-URL click graph for the target entity into target time intervals based on a corresponding query intent time for each target query intent, the target time intervals in the time series being separated by target milestones in the time series; and
generating a prediction of a target performance result for the target entity for an interval after a target milestone in the time series by inputting to the trained machine learning model target query intents allocated to the target entity in a target time interval before the target milestone.

### Claim 2 (depends on 1)

The method of claim 1 wherein the training query intents are generated from the query-URL click graph by a generating operation comprising:
selecting the multiple training entities that satisfy a training entity selection condition relative to the target entity;
extracting from the query-URL click graph training queries associated with the multiple training entities;
embedding the training queries extracted from the query-URL click graph into multidimensional vectors; and
clustering the multidimensional vectors into multiple groups, the multidimensional vectors of a first group satisfying a similarity condition for the first group better than the multidimensional vectors of a second group, each group of multidimensional records constituting one of the training query intents.

### Claim 3 (depends on 1)

The method of claim 1 wherein the target query intents are generated from the query-URL click graph by a generating operation comprising:
extracting from the query-URL click graph target queries associated with the target entity;
embedding the target queries extracted from the query-URL click graph into multidimensional vectors; and
clustering the multidimensional vectors into multiple groups, the multidimensional vectors of a first group satisfying a similarity condition for the first group better than the multidimensional vectors of a second group, each group of multidimensional records constituting one of the target query intents.

### Claim 4 (depends on 1)

The method of claim 1 wherein the training query intents are generated from a from a query-URL click graph by a generating operation comprising:
distributing the training query intents in the time series based on time values associated with the training queries.

### Claim 5 (depends on 1)

The method of claim 1 wherein the operations of allocating training query intents and allocating training performance results are based on metadata extracted from an entity knowledge graph.

### Claim 6 (depends on 1)

The method of claim 1 wherein the training operation comprises:
training the machine learning model for a different training milestone of the time series based on the allocated training query intents allocated to a different training interval prior to the training milestone and the allocated training performance results allocated to a different training interval after the training milestone.

### Claim 7 (depends on 1)

The method of claim 1 wherein the training operation comprises:
training the machine learning model using metadata extracted from an entity knowledge graph.

### Claim 8 (depends on 1)

The method of claim 1 wherein the generating operation comprises:
generating a different prediction of a different target performance result for the target entity for a different target time interval after a different target milestone in the time series by inputting to the trained machine learning model target query intents for the target entity allocated to a different time interval before the different target milestone.

### Claim 9

A computing device for predicting performance of a target entity, the computing device comprising:
a time-factored aggregator configured to allocate training query intents generated from a query-URL click graph for multiple training entities into training time intervals in a time series based on a corresponding query intent time for each training query intent, the training time intervals in the time series being separated by training milestones in the time series, and to allocate training performance results for the multiple training entities into the training time intervals in the time series based on a corresponding performance time of each training performance result; and
a machine learning model configured to receive the allocated training query intents and allocated training performance results, the machine learning model being trained for one or more of the training milestones of the time series based on the training query intents allocated to a training time interval prior to the training milestone and the training performance results allocated to a training time interval after the training milestone, the machine learning model being further configured to generate a prediction of a target performance result for the target entity for an interval after a target milestone in the time series by inputting to the trained machine learning model target query intents allocated to the target entity in a target time interval before the target milestone.

### Claim 10 (depends on 9)

The computing device of claim 9 further comprising:
a query intent generator configured to receive and analyze the query-URL click graph and to generate the training query intents by selecting the multiple training entities that satisfy a training entity selection condition relative to the target entity, extracting from the query-URL click graph training queries associated with the multiple training entities, embedding the training queries extracted from the query-URL click graph into multidimensional vectors, and clustering the multidimensional vectors into multiple groups, the multidimensional vectors of a first group satisfying a similarity condition for the first group better than the multidimensional vectors of a second group, each group of multidimensional records constituting one of the training query intents.

### Claim 11 (depends on 9)

The computing device of claim 9 further comprising:
a query intent generator configured to receive and analyze the query-URL click graph and to generate the training query intents by extracting from the query-URL click graph target queries associated with the target entity, embedding the target queries extracted from the query-URL click graph into multidimensional vectors, and clustering the multidimensional vectors into multiple groups, the multidimensional vectors of a first group satisfying a similarity condition for the first group better than the multidimensional vectors of a second group, each group of multidimensional records constituting one of the target query intents.

### Claim 12 (depends on 9)

The computing device of claim 9 wherein the training query intents and training performance results are allocated based on metadata extracted from an entity knowledge graph, and the machine learning model is further trained the machine learning model using metadata extracted from an entity knowledge graph.

### Claim 13 (depends on 9)

The computing device of claim 9 wherein the machine learning model is further trained for a different training milestone of the time series based on the allocated training query intents allocated to a different training interval prior to the training milestone and the allocated training performance results allocated to a different training interval after the training milestone.

### Claim 14 (depends on 9)

The computing device of claim 9 wherein the machine learning model is configured to further generate a different prediction of a different target performance result for the target entity for a different target time interval after a different target milestone in the time series by inputting to the trained machine learning model target query intents for the target entity allocated to a different time interval before the different target milestone.

### Claim 15

One or more tangible processor-readable storage media of a tangible article of manufacture encoding processor-executable instructions for executing on an electronic computing system a process of predicting performance of a target entity using a machine learning model, the process comprising:
allocating training query intents generated from a query-URL click graph for multiple training entities into training time intervals in a time series based on a corresponding query intent time for each training query intent, the training time intervals in the time series being separated by training milestones in the time series;
allocating training performance results for the multiple training entities into the training time intervals in the time series based on a corresponding performance time of each training performance result;
training the machine learning model for a training milestone of the time series based on the training query intents allocated to a training time interval prior to the training milestone and the training performance results allocated to a training time interval after the training milestone;
allocating target query intents generated from the query-URL click graph for the target entity into target time intervals based on a corresponding query intent time for each target query intent, the target time intervals in the time series being separated by target milestones in the time series; and
generating a prediction of a target performance result for the target entity for an interval after a target milestone in the time series by inputting to the trained machine learning model target query intents allocated to the target entity in a target time interval before the target milestone.

### Claim 16 (depends on 15)

The one or more tangible processor-readable storage media of claim 15 wherein the training query intents are generated from the query-URL click graph by a generating operation comprising:
selecting the multiple training entities that satisfy a training entity selection condition relative to the target entity;
extracting from the query-URL click graph training queries associated with the multiple training entities;
embedding the training queries extracted from the query-URL click graph into multidimensional vectors; and
clustering the multidimensional vectors into multiple groups, the multidimensional vectors of a first group satisfying a similarity condition for the first group better than the multidimensional vectors of a second group, each group of multidimensional records constituting one of the training query intents.

### Claim 17 (depends on 15)

The one or more tangible processor-readable storage media of claim 15 wherein the operations of allocating training query intents and allocating training performance results are based on metadata extracted from an entity knowledge graph.

### Claim 18 (depends on 15)

The one or more tangible processor-readable storage media of claim 15 wherein the training operation comprises:
training the machine learning model for a different training milestone of the time series based on the allocated training query intents allocated to a different training interval prior to the training milestone and the allocated training performance results allocated to a different training interval after the training milestone.

### Claim 19 (depends on 15)

The one or more tangible processor-readable storage media of claim 15 wherein the training operation comprises:
training the machine learning model using metadata extracted from an entity knowledge graph.

### Claim 20 (depends on 15)

The one or more tangible processor-readable storage media of claim 15 wherein the generating operation comprises:
generating a different prediction of a different target performance result for the target entity for a different target time interval after a different target milestone in the time series by inputting to the trained machine learning model target query intents for the target entity allocated to a different time interval before the different target milestone.
