US 2022/0092354 A1 · METHOD AND SYSTEM FOR GENERATING LABELED DATASET USING A TRAINING DATA RECOMMENDER TECHNIQUE

Abstract

This disclosure relates generally to a method and system for generating labelled dataset using a training data recommender technique. Recommender systems face major challenges in handling dynamic data on machine learning paradigms thereby rendering inaccurate unlabeled dataset. The method of the present disclosure is based on a training data recommender technique suitably constructed with a newly defined parameter such as the labelled data prediction threshold to determine the adequate amount of labelled training data required for training the one or more machine learning models. The method processes the received unlabeled dataset for labelling the unlabeled dataset based on a labelled data prediction threshold which is determined using a trained training data recommender technique. This labelling data threshold leads to a significant reduction in training time while performing the one or more machine learning models and thus recommender systems to quickly adapt disruptions thereby decreasing the reduction factor.

Claims9

Dependent claims

A processor implemented method for generating labeled dataset using a training data recommender technique, comprising:

receiving, by a labeling function generator, via one or more hardware processors, (i) an unlabeled dataset, and (ii) a labelled dataset comprising a training data and a test data;
extracting, via the one or more hardware processors, a plurality of feature subsets from the labeled dataset;
feeding, via the one or more hardware processors, the plurality of feature subsets extracted from the labeled dataset to a one or more machine learning models;
generating, via the one or more hardware processors, a plurality of labelling functions for the labelled dataset using the one or more trained machine learning models;
executing, via the one or more hardware processors, the plurality of labelling functions for processing the unlabeled dataset to generate a sparse matrix;
constructing, by a snorkel, via the one or more hardware processors, a generative model for the sparse matrix to label the unlabelled dataset;
and
training, the one or more machine learning models, via the one or more hardware processors, with required amount of labelled dataset for labelling the unlabeled dataset based on a labelled data prediction threshold which is determined using a training data recommender technique.

5 more elements

A system (100), for generating labeled dataset using a training data recommender technique comprising:

a memory (102) storing instructions;
one or more communication interfaces (106);
and
one or more hardware processors (104) coupled to the memory (102) via the one or more communication interfaces (106), wherein the one or more hardware processors (104) are configured by the instructions to:
receive, by a labeling function generator, (i) an unlabeled dataset, and (ii) a labelled dataset comprising a training data and a test data;
extract, a plurality of feature subsets from the labeled dataset;
feed, the plurality of feature subsets extracted from the labeled dataset to a one or more machine learning models;
generate, a plurality of labelling functions for the labelled dataset using the one or more trained machine learning models;
execute, the plurality of labelling functions for processing the unlabeled dataset to generate a sparse matrix;
construct, by a snorkel, a generative model for the sparse matrix to label the unlabelled dataset;
and
train, the one or more machine learning models with required amount of labelled dataset for labelling the unlabeled dataset based on a labelled data prediction threshold which is determined using a training data recommender technique.

9 more elements

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors perform actions comprising:

receiving, by a labeling function generator, (i) an unlabeled dataset, and (ii) a labelled dataset comprising a training data and a test data;
extracting, a plurality of feature subsets from the labeled dataset;
feed, the plurality of feature subsets extracted from the labeled dataset to a one or more machine learning models;
generating, a plurality of labelling functions for the labelled dataset using the one or more trained machine learning models;
executing, the plurality of labelling functions for processing the unlabeled dataset to generate a sparse matrix;
constructing, by a snorkel, a generative model for the sparse matrix to label the unlabelled dataset;
and
training, the one or more machine learning models with required amount of labelled dataset for labelling the unlabeled dataset based on a labelled data prediction threshold which is determined using a training data recommender technique.

5 more elements

Dependencies are inferred from the claim text.

Method and system for generating labeled dataset using a training data recommender technique

Life of this patent

Method and system for generating labeled dataset using a training data recommender technique

Life of this patent

Abstract

Claims9