# US 2021/0279566 A1 · Contrastive Neural Network Training in an Active Learning Environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279566 A1 - Document ID: US20210279566A1 - Status: Published application - Kind: A1 - Application number: 16/809,319 - Filed: 2020-03-04 - Published: 2021-09-09 - Earliest claimed: 2020-03-04 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Chen Lin; Hongtan Sun; John Rofrano; Maja Vukovic - Web page: https://patentgrep.com/patent/US20210279566A1 ## CPC - G06N 3/08: Learning methods - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06F 18/23: Clustering techniques - G06F 18/2321: using statistics or function optimisation, e.g. modelling of probability density functions - G06F 18/24137: {Distances to cluster centroïds} - G06F 18/285: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/09: Supervised learning - G06N 3/091: Active learning - G06N 3/096: Transfer learning - G06N 20/00: Machine learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Embodiments relate to a system, program product, and method for training a contrastive neural network (CNN) in an active learning environment. A neural network is pre-trained with labeled data of a historical dataset. The CNN is trained for the new dataset by applying the new dataset and contrasting the new dataset against the historical dataset to extract novel patterns. Features novel to the new dataset are learned, including updating weights of the knowledge operator. The borrowed knowledge operator weights are combined with the updated knowledge operator weights. The CNN is leveraged to predict one or more labels for the new dataset as output data. ## Claims ### Claim 1 A computer system comprising: a processor operatively coupled to memory; and an artificial intelligence (AI) platform, in communication with the processor, the AI platform having machine learning (ML) tools to train a contrastive neural network (CNN), the tools comprising: a pre-training manager to feed a historical dataset into a neural network, the pre-training manager to select the neural network and pre-train the selected neural network with labeled data in the historical dataset, including the pre-training manager to train weights of layers of the selected neural network; a training manager, operatively coupled to the pre-training manager, to train the CNN for a new dataset, including: borrow weights of a knowledge operator from the pre-trained neural network; learn features novel to the new dataset, including update weights of the knowledge operator; and combine the borrowed weights with the updated weights; and a prediction manager, operatively coupled to the training manager, to leverage the CNN to predict one or more labels for the dataset as output data. ### Claim 2 (depends on 1) The computer system of claim 1, further comprising the training manager to identify and extract one or more novel patterns in the new dataset and absent from the historical dataset prior to training the CNN, and to de-noise the new dataset, wherein de-noising includes masking irrelevant information while preserving the novel patterns. ### Claim 3 (depends on 2) The computer system of claim 2, further comprising the training manager to organize the identified one or more novel patterns in the de-noised new dataset into two or more clusters, and to selectively evaluate the two or more clusters for prediction accuracy. ### Claim 4 (depends on 3) The computer system of claim 3, further comprising the training manager to dynamically select the cluster from which samples of labeling will be drawn for annotation based on a characteristic associated with the prediction accuracy of the cluster. ### Claim 5 (depends on 4) The computer system of claim 4, further comprising the training manager to update the prediction accuracy of the cluster responsive to the annotation. ### Claim 6 (depends on 5) The computer system of claim 5, further comprising the training manager to leverage the updated prediction accuracy as a characteristic for the dynamic cluster selection. ### Claim 7 (depends on 1) The computer system of claim 1, wherein the trained CNN is a new machine learning classifier adaptively modified from the neural network to output one or more labels for the dataset as a predicted data annotation. ### Claim 8 A computer program product to train a contrastive neural network (CNN) in an active learning environment, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by a processor to: feed a historical dataset into a neural network, including the program code to select the neural network and pre-train the selected neural network with labeled data in the historical dataset, the pre-training including training weights of layers of the selected neural network; train the CNN for a new dataset, including the program code to transfer knowledge from the trained weights of the selected neural network, including: borrow weights of a knowledge operator from the pre-trained neural network as a regularizer; learn features novel to the new dataset, including update weights of the knowledge operator; and combine the borrowed weights with the updated weights; and leverage the CNN to predict one or more labels for the dataset as output data. ### Claim 9 (depends on 8) The computer program product of claim 8, further comprising the program code to identify and extract one or more novel patterns in the new dataset and absent from the historical dataset prior to training the contrastive neural network, and to de-noise the new dataset, wherein de-noising includes masking irrelevant information while preserving the novel patterns. ### Claim 10 (depends on 9) The computer program product of claim 9, further comprising the program code to organize the identified one or more novel patterns in the de-noised new dataset into two or more clusters, and to selectively evaluate the two or more clusters for prediction accuracy. ### Claim 11 (depends on 10) The computer program product of claim 10, further comprising the program code to dynamically select the cluster from which samples of labeling will be drawn for annotation based on a characteristic associated with the prediction accuracy of the cluster. ### Claim 12 (depends on 11) The computer program product of claim 11, further comprising the program code to update the prediction accuracy of the cluster responsive to the annotation. ### Claim 13 (depends on 12) The computer program product of claim 12, further comprising the program code to leverage the updated prediction accuracy as a characteristic for the dynamic cluster selection. ### Claim 14 (depends on 8) The computer program product of claim 8, the trained CNN is a new machine learning classifier adaptively modified from the neural network to output one or more labels for the dataset as a predicted data annotation. ### Claim 15 A method comprising: feeding a historical dataset into a neural network, including selecting the neural network and pre-training the selected neural network with labeled data in the historical dataset, the pre-training including training weights of layers of the selected neural network; training a contrastive neural network (CNN) in an active learning environment for a new dataset while transferring knowledge from the trained weights of the selected neural network, including: borrowing weights of a knowledge operator from the pre-trained neural network; learning features novel to the new dataset, including updating weights of the knowledge operator; and combining the borrowed knowledge operator weights with the updated weights the CNN predicting one or more labels for the new data as output data. ### Claim 16 (depends on 15) The method of claim 15, further comprising identifying and extracting one or more novel patterns in the new dataset and absent from the historical dataset prior to training the contrastive neural network, and de-noising the new dataset, wherein de-noising includes masking irrelevant information while preserving the novel patterns. ### Claim 17 (depends on 16) The method of claim 16, further comprising organizing the identified one or more novel patterns in the de-noised new dataset into two or more clusters, and selectively evaluating the two or more clusters for prediction accuracy. ### Claim 18 (depends on 17) The method of claim 17, further comprising dynamically selecting the cluster from which samples of labeling will be drawn for annotation based on a characteristic associated with the prediction accuracy of the cluster. ### Claim 19 (depends on 18) The method of claim 18, further comprising updating the prediction accuracy of the cluster responsive to the annotation and leveraging the updated prediction accuracy as a characteristic for the dynamic cluster selection. ### Claim 20 (depends on 15) The method of claim 15, the trained CNN is a new machine learning classifier adaptively modified from the neural network to output one or more labels for the dataset as a predicted data annotation.