Intent detection

A method of training a neural network, comprising:

• modifying at least one token of a text phrase to obtain a modified text phrase;
• encoding the text phrase and the modified text phrase using an encoder to obtain a hidden representation of the text phrase and a modified hidden representation of the modified text phrase;
• training the encoder in a first training phase using a first contrastive learning loss, wherein the first contrastive loss is based on an unlabeled positive sample pair including the hidden representation and the modified hidden representation;
• and
• training the encoder in a second training phase following the first training phase using a second contrastive learning loss, wherein the second contrastive loss is based on a labeled positive sample pair including a first labeled hidden representation having a ground truth label and a second labeled hidden representation having the same ground truth label.

An apparatus for natural language processing, comprising:

• an encoder configured to encode a text phrase to obtain a hidden representation;
• a classification network configured to predict a label for the text phrase network based on the hidden representation;
• a pre-training component configured to train the encoder in a first training phase using a first contrastive learning loss, wherein the first contrastive learning loss is based on an unlabeled positive sample pair including the hidden representation and a modified hidden representation;
• and
• a fine-tuning component configured to train the encoder in a second training phase following the first training phase using a second contrastive learning loss, wherein the second contrastive learning loss is based on a labeled positive sample pair including a first labeled hidden representation with a ground truth label and a second labeled hidden representation having the same ground truth label.