Loading patent…
OwnerTorsten SCHOLAK, Lee ZAMPARO +4inventors
InventorsTorsten SCHOLAK, Lee ZAMPARO, Hector PALACIOS, Kamil LEGAULT, Pierre-André NOËL, Krzysztof MAJEWSKI
Estimating the term…
A method and system for improving quality of a dataset for which a labeling task is to be completed. A loop is repeated comprising: inferring, for each of the labeler identifiers in the dataset, an estimated proficiency value; inferring a predicted uncertainty value of correctness of the label for at least a subset of the raw data items; and receiving a trusted evaluation value of correctness for one or more labels of the subset of the raw data items for which the predicted uncertainty is inferred. The loop is repeated until the highest predicted uncertainty value in the dataset is below a threshold value.
A method for improving quality of a dataset for which a labeling task is to be completed, the dataset comprising raw data items and, for each data item, one or more labels representing answers to the labeling task and, for each label, an associated labeler identifier, the method comprising:
A labeling system configured for improving quality of a dataset, stored in a storage system, for which a task is to be completed, the dataset comprising raw data items and, for each data item, one or more labels representing an answer to the task and, for each label, an associated labeler, the labeling system comprising:
Dependencies are inferred from the claim text.