Methods and systems for automated document classification with partially labeled data using semi-supervised learning

A non-transitory machine-readable medium containing instructions that, when executed by a machine, cause the machine to:

• sort a document collection comprising a plurality of electronic documents into a plurality of categories;
• train a classifier corresponding to a category of the plurality of categories to output a probability that an electronic document associated with the category is confidential;
• and
• output the probability that the electronic document associated with the category is confidential using the classifier;
• wherein the instructions causing the machine to train the classifier cause the machine to:
• determine that a cardinality of a set of nonconfidential samples in a training set is not above a pipeline threshold but is at least one;
• preprocess each sample in the training set to extract raw text from each sample to thereby form a group of confidential samples that are preprocessed, a group of nonconfidential samples that are preprocessed, and a group of unlabeled samples that are preprocessed;
• generate based on the determining that the cardinality is not above the pipeline threshold, respective first and second finalized training groups using the preprocessed confidential samples and a nonconfidential sample set, the nonconfidential sample set derived at least in part from the group of preprocessed unlabeled samples based on predicted probabilities that one or more of the preprocessed unlabeled samples behave similarly to a subsample of the preprocessed confidential samples according to a similarity threshold;
• train the classifier via a first pipeline using the first finalized training group, the first finalized training group generated based on a first portion of the group of confidential samples and a second portion of the group of unlabeled samples, and via a second pipeline using the second finalized training group, the second finalized training group generated based on the first portion of the group of confidential samples and a third portion of a group having the group of unlabeled samples and the group of nonconfidential samples;
• and
• classify, by the classifier, unlabeled documents as confidential using the probability, the probability calculated based on the training set having nonconfidential samples not above the pipeline threshold.

A computing device comprising:

• a memory containing a machine-readable medium containing instructions for performing a method of classifying electronic documents;
• and
• a processor coupled to the memory, the processor configured to execute the instructions to cause the processor to:
• sort a document collection comprising a plurality of electronic documents into a plurality of categories;
• train a classifier corresponding to a category of the plurality of categories to output a probability that an electronic document associated with the category belongs to a selected type using a training set, wherein in training the classifier the processor is configured to:
• determine that a cardinality of a set of negative samples in the training set is zero;
• preprocess each sample in the training set to extract raw text from each sample to thereby form a group of positive samples that are preprocessed and a group of unlabeled samples that are preprocessed;
• generate, based on the determining that the cardinality is zero, a finalized training group using the preprocessed positive samples and a negative sample set, the negative sample set derived at least in part from the group of preprocessed unlabeled samples based on predicted probabilities that one or more of the preprocessed unlabeled samples behave similarly to a subsample of the preprocessed positive samples according to a similarity threshold;
• train the classifier via a first pipeline that uses semi-supervised learning, wherein the training is performed using the finalized training group generated based on a portion of the group of positive samples and a portion of a group of unlabeled samples;
• determine the probability that the electronic document associated with the category belongs to the selected type using the classifier;
• and
• classify unlabeled documents as belonging to the selected type using the probability, the probability calculated based on the training set having zero negative samples.