Self contrastive decorrelation based training of machine learning models

A computer-implemented method comprising:

• training a machine learning model, the training comprising:
• performing a first encoding operation on a sentence including a plurality of text, the first encoding operation comprising generating a first vector representation of the sentence, the first vector representation including first numerical elements representing the plurality of text of the sentence, wherein the first vector representation is generated using a first dropout ratio that modifies, in a vector domain, the sentence;
• performing a second encoding operation on the sentence, the second encoding operation comprising generating a second vector representation of the sentence, wherein the second vector representation is generated using a second dropout ratio that modifies, in the vector domain, the sentence;
• mapping the first vector representation to a first high dimensional vector representation, and further mapping the second vector representation to a second high dimensional vector representation;
• decorrelating the first high dimensional vector representation and the second high dimensional vector representation;
• wherein a trained machine learning model comprises the decorrelated first high dimensional vector representation and the decorrelated second high dimensional vector representation.

A system comprising:

• at least one processor;
• and
• at least one memory including code which when executed by the at least one processor causes operations comprising:
• training a machine learning model, the training comprising:
• performing a first encoding operation on a sentence including a plurality of text, the first encoding operation comprising generating a first vector representation of the sentence, the first vector representation including first numerical elements representing the plurality of text of the sentence, wherein the first vector representation is generated using a first dropout ratio that modifies, in a vector domain, the sentence;
• performing a second encoding operation on the sentence, the second encoding operation comprising generating a second vector representation of the sentence, wherein the second vector representation is generated using a second dropout ratio that modifies, in the vector domain, the sentence;
• mapping the first vector representation to a first high dimensional vector representation, and further mapping the second vector representation to a second high dimensional vector representation;
• decorrelating the first high dimensional vector representation and the second high dimensional vector representation;
• wherein a trained machine learning model comprises the decorrelated first high dimensional vector representation and the decorrelated second high dimensional vector representation.

A non-transitory computer-readable storage medium including code which when executed by at least one processor causes operations comprising:

• training a machine learning model, the training comprising:
• performing a first encoding operation on a sentence including a plurality of text, the first encoding operation comprising generating a first vector representation of the sentence, the first vector representation including first numerical elements representing the plurality of text of the sentence, wherein the first vector representation is generated using a first dropout ratio that modifies, in a vector domain, the sentence;
• performing a second encoding operation on the sentence, the second encoding operation comprising generating a second vector representation of the sentence, wherein the second vector representation is generated using a second dropout ratio that modifies, in the vector domain, the sentence;
• mapping the first vector representation to a first high dimensional vector representation, and further mapping the second vector representation to a second high dimensional vector representation;
• decorrelating the first high dimensional vector representation and the second high dimensional vector representation;
• wherein a trained machine learning model comprises the decorrelated first high dimensional vector representation and the decorrelated second high dimensional vector representation.