# US 2022/0036248 A1 · System and Method for Ensemble Expert Diversification via Bidding

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036248 A1
- Document ID: US20220036248A1
- Status: Published application
- Kind: A1
- Application number: 16/944,415
- Filed: 2020-07-31
- Published: 2022-02-03
- Earliest claimed: 2020-07-31
- Assignees: Oath Inc.
- Applicants: Oath Inc.
- Inventors: Gal Lalouche; Ran Wolff
- Web page: https://patentgrep.com/patent/US20220036248A1

## CPC

- G06N 20/20: Ensemble learning
- G06N 5/043: Distributed expert systems; Blackboards
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present teaching relates to method, system, medium, and implementations for machine learning. A check is performed on a level of available bidding currency for bidding a training sample that is used to train a model via machine learning. A bid in an amount within the available bidding currency is sent, to a source of the training sample, for the training sample. The training sample is received from the source when the bid is successful. A prediction is then generated in accordance with the training sample based on one or more parameters associated with the model and is sent to the source.

## Claims

### Claim 1

A method implemented on at least one machine including at least one processor, memory, and communication platform capable of connecting to a network for machine learning, the method comprising:
checking a level of available bidding currency for bidding a training sample used for training a model via machine learning;
sending, to a source of the training sample, a bid for the training sample in an amount within the available bidding currency;
receiving the training sample from the source when the bid is successful; and
sending, to the source, a prediction generated in accordance with the training sample based on one or more parameters associated with the model.

### Claim 2 (depends on 1)

The method of claim 1, wherein the source determines how to distribute the training sample to achieve expert diversification.

### Claim 3 (depends on 1)

The method of claim 1, wherein the available bidding currency is
initialized by the source; and
is deducted by the amount upon sending the bid to the source.

### Claim 4 (depends on 1)

The method of claim 1, further comprising
computing, with respect to the prediction, a metric characterizing the prediction; and
sending, to the source, the metric together with the prediction to facilitate the source to determine whether a ground truth label for the training sample is to be provided.

### Claim 5 (depends on 4)

The method of claim 4, wherein the metric includes a confidence score indicative of a level of confidence in the prediction.

### Claim 6 (depends on 1)

The method of claim 1, further comprising
receiving, from the source, a ground truth label corresponding to the training sample; and
updating the one or more parameters of the model based on the prediction and the ground truth label.

### Claim 7 (depends on 6)

The method of claim 6, wherein the step of updating comprises:
computing a discrepancy between the prediction and the ground truth label;
determining an adjustment to the one or more parameters of the model based on the discrepancy; and
adjusting the one or more parameters in accordance with the adjustment.

### Claim 8

Machine readable non-transitory medium having information recorded thereon for machine learning, wherein the information, once read by the machine, causes the machine to perform:
checking a level of available bidding currency for bidding a training sample used for training a model via machine learning;
sending, to a source of the training sample, a bid for the training sample in an amount within the available bidding currency;
receiving the training sample from the source when the bid is successful; and
sending, to the source, a prediction generated in accordance with the training sample based on one or more parameters associated with the model.

### Claim 9 (depends on 8)

The medium of claim 8, wherein the source determines how to distribute the training sample to achieve expert diversification.

### Claim 10 (depends on 8)

The medium of claim 8, wherein the available bidding currency is
initialized by the source; and
is deducted by the amount upon sending the bid to the source.

### Claim 11 (depends on 8)

The medium of claim 8, wherein the information, when read by the machine, further causes the machine to perform:
computing, with respect to the prediction, a metric characterizing the prediction; and
sending, to the source, the metric together with the prediction to facilitate the source to determine whether a ground truth label for the training sample is to be provided.

### Claim 12 (depends on 11)

The medium of claim 11, wherein the metric includes a confidence score indicative of a level of confidence in the prediction.

### Claim 13 (depends on 1)

The medium of claim 1, wherein the information, once read by the machine, further causes the machine to perform:
receiving, from the source, a ground truth label corresponding to the training sample; and
updating the one or more parameters of the model based on the prediction and the ground truth label.

### Claim 14 (depends on 13)

The medium of claim 13, wherein the step of updating comprises:
computing a discrepancy between the prediction and the ground truth label;
determining an adjustment to the one or more parameters of the model based on the discrepancy; and
adjusting the one or more parameters in accordance with the adjustment.

### Claim 15

A system for machine learning, comprising:
a currency assessment unit configured for checking a level of available bidding currency for bidding a training sample used for training a model via machine learning;
a training data bidding unit configured for sending, to a source of the training sample, a bid for the training sample in an amount within the available bidding currency;
a training data processing unit configured for receiving the training sample from the source when the bid is successful; and
a training unit configured for sending, to the source, a prediction generated in accordance with the training sample based on one or more parameters associated with the model.

### Claim 16 (depends on 15)

The system of claim 15, wherein the source determines how to distribute the training sample to achieve expert diversification.

### Claim 17 (depends on 15)

The system of claim 15, wherein the available bidding currency is
initialized by the source; and
is deducted by the amount upon sending the bid to the source.

### Claim 18 (depends on 15)

The system of claim 15, further comprising a confidence assessment unit configured for:
computing, with respect to the prediction, a metric characterizing the prediction; and
sending, to the source, the metric together with the prediction to facilitate the source to determine whether a ground truth label for the training sample is to be provided, wherein the metric includes a confidence score indicative of a level of confidence in the prediction.

### Claim 19 (depends on 15)

The system of claim 15, wherein the training unit is further configured for:
receiving, from the source, a ground truth label corresponding to the training sample; and
updating the one or more parameters of the model based on the prediction and the ground truth label.

### Claim 20 (depends on 19)

The system of claim 19, wherein the step of updating comprises:
computing a discrepancy between the prediction and the ground truth label;
determining an adjustment to the one or more parameters of the model based on the discrepancy; and
adjusting the one or more parameters in accordance with the adjustment.
