TARGETED GENERATIVE PRE-TRAINED TRANSFORMERS ("GPTs")

A method for creating targeted, generative, pre-trained, transformer models (“targeted models”) from a generative, pre-trained, transformer model (“pre-trained model”), the method comprising:

• receiving a pre-trained model, said pre-trained model comprising a plurality of weighted neurons organized within a plurality of layers;
• pruning the pre-trained model for a specific discipline, the pruning comprising:
• identifying a plurality of training data elements pertaining to the specific discipline;
• processing the plurality of training data elements through the pre-trained model;
• during the processing, flagging each neuron affected by the plurality of training data elements;
• and
• removing one or more neurons from the plurality of neurons, said one or more neurons being unflagged;
• and
• creating a pruned copy of the pre-trained model, said pruned copy being a targeted model for the specific discipline, said pruned copy comprising the flagged neurons and associated weights, said pruned copy absent a portion of the pre-trained model's neurons which are unaffected during the processing.

A system for creating a targeted, generative, pre-trained, transformer model, the model comprising:

• a processor, the processor is operable to:
• receive a pre-trained model, the pre-trained model comprising a plurality of weighted neurons organized within a plurality of layers;
• receive an instruction to prune the pre-trained model for a predetermined field;
• identify a first set of one or more training data elements corresponding to the predetermined field;
• freeze the weights of the neurons of the pre-trained model;
• disable the first set of one or more training data elements from changing the weights of the neurons included in the pre-trained model;
• process the first set of one or more training data elements through the pre-trained model;
• during the process, highlight a subset of neurons within the pre-trained model, said subset of neurons affected during the process of the first set of one or more training data elements;
• create a pruned, targeted, pre-trained model for the predetermined field, said pruned, targeted, pre-trained model comprising the highlighted neurons and associated weights, said pruned, targeted, pre-trained model absent a portion of the plurality of neurons which are unaffected during the process;
• tune the pruned, targeted, pre-trained model by processing a second set of one or more training data elements that correspond to the predetermined field;
• and
• rebuild and regenerate neurons, at the pruned, targeted, pre-trained model, during process of the second set of one or more training data elements.