Text-to-visual machine learning embedding techniques
AssigneeAdobe Inc.
InventorsPranav Vineet Aggarwal, Zhe Lin, Baldo Antonio Faieta, Saeid Motiian
Life of this patent
Estimating the term…
Abstract
Text-to-visual machine learning embedding techniques are described that overcome the challenges of conventional techniques in a variety of ways. These techniques include use of query-based training data which may expand availability and types of training data usable to train a model. Generation of negative digital image samples is also described that may increase accuracy in training the model using machine learning. A loss function is also described that also supports increased accuracy and computational efficiency by losses separately, e.g., between positive or negative sample embeddings a text embedding.
Claims20
In a digital medium machine learning model training environment, a method implemented by a computing device, the method comprising:
- receiving, by the computing device, a plurality of text queries used to initiate a plurality of digital image searches;
- generating, by the computing device, a plurality of filtered text queries by filtering stop words from the plurality of text queries;
- generating, by the computing device, a training dataset based on the plurality of filtered text queries and a plurality of digital images generated by a plurality of digital image searches, the training dataset including:
- a positive digital image sample located using a first respective said filtered text query;
- and
- a negative digital image using a second respective said filtered text query that shares at least one item of text with the first respective said filtered text query and does not share at least one other item of text with the first respective said filtered text query;
- training, by the computing device, a model using machine learning based on a loss function using the training dataset;
- and
- generating, by the computing device, a subsequent search result using the model.
In a digital medium machine learning model training environment, a system comprising:
- a search module implemented at least partially in hardware to generate a plurality of digital images generated by a plurality of digital image searches;
- a training data generation module implemented at least partially in hardware to generate a training dataset, the training data generation module including:
- a positive sample generation module configured to select a positive digital image sample from the plurality of digital images;
- and
- a negative sample generation module configured to:
- generate filtered text by filtering text associated with the plurality of digital images;
- generate a subset of the plurality of digital images including a digital image from the plurality of digital images associated with at least one item of filtered text also included with filtered text associated with the positive digital image sample and does not share at least one other item of filtered text with the positive digital image;
- and
- select a negative digital image sample from the subset;
- a machine-learning training module implemented at least partially in hardware to train a model using a loss function as part of machine learning based on the training dataset.
In a digital medium machine learning model training environment, a method implemented by a computing device, the method comprising:
- receiving, by the computing device, a plurality of digital images and a plurality of text associated with the plurality of digital images, respectively;
- generating, by the computing device, a training dataset based on the plurality of digital images and the plurality of text, the training dataset having a first training dataset that is query based and a second training dataset that is title based and including a positive digital image sample, text of the plurality of text associated with the positive digital image sample, and a negative digital image sample;
- training, by the computing device, a model using machine learning based on a loss function using the training dataset, the training including:
- generating a text embedding from the text, a positive image embedding from the positive digital image sample, and a negative image embedding from the negative digital image sample;
- and
- determining, using the loss function, a loss between the text embedding and the positive image embedding for the first training dataset separately from a loss between the text embedding and the negative image embedding of the second training dataset.
Dependencies are inferred from the claim text.