US 2023/0401387 A1Published application

Domain adaptation of machine learning models for classification

AssigneeSalesforce, Inc.

InventorsZachary Alexander, Shubham Mehrotra

Life of this patent

Estimating the term…

Loading patent…

Abstract

The system receives a base machine learning model trained using a generic dataset. For example, the base machine learning model may be an off-the-shelf machine learning based model. The base machine learning model is trained to receive an input and generate a feature vector representing the input. The input may be a natural language expression, an image, or any other type of input. The system receives a domain specific training dataset based on known categories for input values. The system determines an orthogonal transformation for reducing the dimensions of the base machine learning model using on the domain specific training dataset. The system applies the orthogonal transformation to the base machine learning model to obtain a domain specific machine learning model. The system uses the domain specific machine learning model for processing inputs, for example, in a production environment.

Claims20

Dependent claims

A method for determining intent of natural language expressions used in chatbot conversations, the method comprising:

receiving a base machine learning model trained using a generic dataset, the base machine learning model trained to receive a natural language expression and generate a feature vector representing the natural language expression;
receiving a domain specific training dataset based on known intents for a chatbot;
determining an orthogonal transformation for reducing dimensions of the base machine learning model, the orthogonal transformation determined based on the domain specific training dataset;
applying the orthogonal transformation to the base machine learning model to obtain a domain specific machine learning model;
receiving an input natural language expression used in a chatbot conversation;
executing the domain specific machine learning model based on the received natural language expression to obtain a feature vector representing the input natural language expression;
comparing the feature vector representing the input natural language expression to one or more stored feature vectors previously mapped to intents;
determining an intent for input natural language expression based on an intent of a matching stored feature vector;
and
taking an action based on the determined intent.
determining an orthogonal transformation for reducing dimensions of the base machine learning model, the orthogonal transformation determined based on the domain specific training dataset;
applying the orthogonal transformation to the base machine learning model to obtain a domain specific machine learning model;
receiving an input natural language expression used in a chatbot conversation;
executing the domain specific machine learning model based on the received natural language expression to obtain a feature vector representing the input natural language expression;
comparing the feature vector representing the input natural language expression to one or more stored feature vectors previously mapped to intents;
determining an intent for input natural language expression based on an intent of a matching stored feature vector;
and
taking an action based on the determined intent.

15 more elements

A method for transforming a machine learning model, the method comprising:

receiving a base machine learning model trained using a generic dataset, the base machine learning model configured to receive an input value and generate a feature vector representing the input value;
receiving a domain specific training dataset based on known categories of values;
determining an orthogonal transformation for reducing dimensions of the base machine learning model, the determining comprising:
initializing the orthogonal transformation;
generating a set of triplets based on the domain specific training dataset, wherein a triplet includes an anchor data value, a positive value that has same category as the anchor value, and a negative value that has different category compared to the anchor value;
and
modifying the orthogonal transformation to minimize a loss across the set of triplets;
applying the orthogonal transformation to the base machine learning model to obtain a domain specific machine learning model;
using the domain specific machine learning model to determine a category for a new input value.

6 more elements

A non-transitory computer-readable storage medium storing instructions that when executed by one or more computer processors, causes the one or more computer processors to perform steps comprising:

receiving a base machine learning model trained using a generic dataset, the base machine learning model trained to receive a natural language expression and generate a feature vector representing the natural language expression;
receiving a domain specific training dataset based on known intents for a chatbot;
processors to perform steps comprising:
initializing the orthogonal transformation based on random values;
generating a set of triplets based on the domain specific training dataset, wherein a triplet includes an anchor data point, a positive data point that has same intent as the anchor data, and a negative data point that has different intent comparted to the anchor data;
and
modifying the orthogonal transformation to minimize a loss value across the set of triplets.

4 more elements

Dependencies are inferred from the claim text.