US 2021/0241040 A1 · Systems and Methods for Ground Truth Dataset Curation

Abstract

Systems and methods of the present disclosure enable ground truth dataset creation using processing devices configured to receive at least one user profile of at least one user and user-generated labels for each sample of at least one sample, where the user-generated labels are associated with the at least one user profile. A most likely label for each sample is produced using a weighting of each user-generated label according to user attributes in each user profile associated with each user-generated label. Each most likely label is recorded for each sample in a ground-truth dataset associated with each sample to identify a ground truth of the at least one sample, and each most likely label and each sample is provided to a machine learning model to train the machine learning model according to a difference between a predicted label for each sample and each most likely label.

Claims20

Dependent claims

A method comprising:

receiving, by at least one processor, user-generated labels for each sample of at least one sample;
wherein the user-generated labels are associated with at least one user profile of at least one user;
receiving, by the at least one processor, the at least one user profile;
producing, by the at least one processor, a most likely label for each sample based at least in part on a weighting of each user-generated label of the user-generated labels according to user attributes represented in each user profile of the at least one user profile associated with each user-generated label;
recording, by the at least one processor, each most likely label for each sample in a ground-truth dataset associated with each sample to identify a ground truth of the at least one sample;
and
providing, by the at least one processor, each most likely label and each sample to a machine learning model to train the machine learning model according to a difference between a predicted label for each sample and each most likely label.

4 more elements

A system comprising:

at least one processor in communication with at least one memory configured to access instructions in the at least one memory that cause the at least one processor to perform steps to:
receive user-generated labels for each medical sample of at least one sample;
wherein the user-generated labels are associated with at least one user profile of at least one user;
receive the at least one user profile;
produce a most likely label for each medical sample based at least in part on a weighting of each user-generated label of the user-generated labels according to user attributes represented in each user profile of the at least one user profile associated with each user-generated label;
and
record each most likely label for each medical sample in a ground-truth dataset associated with each medical sample to identify a ground truth of the at least one sample.

4 more elements

Dependencies are inferred from the claim text.

Systems and Methods for Ground Truth Dataset Curation

Life of this patent

Systems and Methods for Ground Truth Dataset Curation

Life of this patent

Abstract

Claims20