Systems and methods for text classification using label modular prompts

A non-transitory machine-readable medium comprising a plurality of machine-readable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform a method comprising:

• receiving, via a data interface, an input text sequence;
• determining, by a label modular prompt generator having a plurality of parameters, a set of class labels of interest from a set of possible class labels associated with the input text sequence;
• generating, by the label modular prompt generator, a plurality of label prompts based on the set of class labels of interest, wherein the generating the plurality of label prompts includes:
• concatenating a first class label and a sequence of soft tokens that are generated based on representations associated with the first class label into a first label prompt,
• wherein the soft tokens are tunable using the plurality of parameters of the label modular prompt generator;
• providing, by the label modular prompt generator, an input of the input text sequence prepended with the plurality of label prompts to a pretrained language model;
• and
• generating, by the pretrained language model, a task output in response to the input text sequence.

A system, comprising:

• a non-transitory memory;
• and
• one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform a method comprising:
• receiving, via a data interface, an input text sequence;
• determining, by a label modular prompt generator having a plurality of parameters, a set of class labels of interest from a set of possible class labels associated with the input text sequence;
• generating, by the label modular prompt generator, a plurality of label prompts based on the set of class labels of interest, wherein the generating the plurality of label prompts includes:
• concatenating a first class label and a sequence of soft tokens that are generated based on representations associated with the first class label into a first label prompt,
• wherein the soft tokens are tunable using the plurality of parameters of the label modular prompt generator;
• providing, by the label modular prompt generator, an input of the input text sequence prepended with the plurality of label prompts to a pretrained language model;
• and
• generating, by the pretrained language model, a task output in response to the input text sequence.