# US 2020/0402013 A1 · Predicting successful outcomes

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0402013 A1
- Document ID: US20200402013A1
- Status: Published application
- Kind: A1
- Application number: 16/445,897
- Filed: 2019-06-19
- Published: 2020-12-24
- Earliest claimed: 2019-06-19
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Man N. Yeung; Sandeep Tiwari; Sumedha K. Swamy; Nitin Lakhotia
- Web page: https://patentgrep.com/patent/US20200402013A1

## CPC

- G06Q 10/1053: Employment or hiring
- G06F 18/2155: {characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06N 20/20: Ensemble learning
- G06N 3/08: Learning methods
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The disclosed embodiments provide a system for predicting successful outcomes. During operation, the system determines interaction features characterizing interaction between a moderator of a job and one or more applicants for the job. Next, the system applies a machine learning model to the interaction features to produce a score representing a likelihood of a positive outcome for the job. The system then applies a threshold to the score to generate a predicted outcome for the job. Finally, the system outputs the predicted outcome in association with the job.

## Claims

### Claim 1

A method, comprising:
determining interaction features characterizing interaction between a moderator of a job and one or more applicants for the job;
applying, by one or more computer systems, a machine learning model to the interaction features to produce a score representing a likelihood of a positive outcome for the job;
applying, by the one or more computer systems, a threshold to the score to generate a predicted outcome for the job; and
outputting the predicted outcome in association with the job.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating labels for additional jobs based on outcomes received for the jobs over a period; and
inputting additional features for the additional jobs with the labels as training data for the machine learning model.

### Claim 3 (depends on 2)

The method of claim 2, wherein inputting the additional features for the additional jobs with the labels as the training data for the machine learning model comprises:
generating multiple versions of the machine learning model from different subsets of the training data;
for each version of the machine learning model, determining a performance of the version based on a remainder of the training data that was not used to train the version; and
selecting a final version of the machine learning model with the best performance for use in predicting outcomes for jobs.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
selecting the machine learning model to match a job segment of the job.

### Claim 5 (depends on 4)

The method of claim 4, wherein the job segment comprises at least one of:
paid jobs;
free jobs;
onsite sources for jobs; and
offsite sources for jobs.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
applying the machine learning model to job features for the job to produce the score.

### Claim 7 (depends on 6)

The method of claim 6, wherein the job features comprise at least one of:
a location;
a function;
an industry;
a seniority;
a title;
a skill;
a salary;
a company segment; and
a payment model for the job.

### Claim 8 (depends on 1)

The method of claim 1, wherein determining the interaction features comprises:
determining, based on a hierarchy of the interaction features, a highest-ranked interaction feature with a non-zero value for the job; and
converting remaining interaction features for the job that are below the highest-ranked feature in the hierarchy to zero values.

### Claim 9 (depends on 8)

The method of claim 8, wherein the hierarchy of the interaction features comprises:
a first rank for a first number of applicants messaged by the moderator;
a second rank that is below the first rank for a second number of applicants with resumes viewed by the moderator; and
a third rank that is below the second rank for a third number of applicants with profiles viewed by the moderator.

### Claim 10 (depends on 8)

The method of claim 8, wherein the hierarchy of the interaction features comprises:
a first rank for a first number of qualified applicants for the job; and
a second rank that is below the first rank for a second number of non-qualified applicants for the job.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
selecting the job for use in generating the predicted outcome after the job has been posted for a pre-specified period.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
generating a recommendation for controlling delivery of the job based on the predicted outcome.

### Claim 13 (depends on 12)

The method of claim 12, wherein the recommendation comprises at least one of:
an adjustment to subsequent delivery of the job; and
a budget for the job.

### Claim 14

A system, comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
determine interaction features characterizing interaction between a moderator of a job and one or more applicants for the job;
apply a machine learning model to the interaction features and job features for the job to produce a score representing a likelihood of a positive outcome for the job;
apply a threshold to the score to generate a predicted outcome for the job; and
output the predicted outcome in association with the job.

### Claim 15 (depends on 14)

The system of claim 14, wherein the memory further stores instructions that, when executed by the one or more processors, cause the system to:
generate labels for additional jobs based on outcomes received for the jobs over a period; and
input additional features for the additional jobs with the labels as training data for the machine learning model.

### Claim 16 (depends on 15)

The system of claim 15, wherein inputting the additional features for the additional jobs with the labels as the training data for the machine learning model comprises:
generating multiple versions of the machine learning model from different subsets of the training data;
for each version of the machine learning model, determining a performance of the version based on a remainder of the training data that was not used to train the version; and
selecting a final version of the machine learning model with the best performance for use in predicting outcomes for jobs.

### Claim 17 (depends on 14)

The system of claim 14, wherein determining the interaction features comprises:
determining, based on a hierarchy of the interaction features, a highest-ranked interaction feature with a non-zero value for the job; and
converting remaining interaction features for the job that are below the highest-ranked feature in the hierarchy to zero values.

### Claim 18 (depends on 14)

The system of claim 14, wherein the interaction features comprise at least one of:
a first number of applicants messaged by the moderator;
a second number of applicants with resumes viewed by the moderator;
a third number of applicants with profiles viewed by the moderator a fourth number of qualified applicants for the job; and
a fifth number of non-qualified applicants for the job.

### Claim 19 (depends on 14)

The system of claim 14, wherein the job features comprise at least one of:
a location;
a function;
an industry;
a seniority;
a title;
a skill;
a salary;
a company segment; and
a payment model for the job.

### Claim 20

A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:
determining interaction features characterizing interaction between a moderator of a job and one or more applicants for the job;
applying a machine learning model to the interaction features to produce a score representing a likelihood of a positive outcome for the job;
applying a threshold to the score to generate a predicted outcome for the job; and
outputting the predicted outcome in association with the job.