US 11,455,485 B2Granted

Content prediction based on pixel-based vectors

AssigneeAdobe Inc.

InventorsMichele Saad, Lauren Dest

Life of this patent

Estimating the term…

Loading patent…

Abstract

Methods and systems disclosed herein relate generally to systems and methods for predicting content based on vector data structures generated from image pixels. A content-prediction application accesses a color palette having two or more color-palette categories and selects a first color of the color palette. The content-prediction application generates a first vector based on a set of pixel values that represent the first color of the color palette. The content-prediction application determines a distance metric between the first vector and a second vector, in which the second vector is identified by applying a convolutional neural network model on an image depicting an item that includes a second color. In response to determining that the distance metric is less than a predetermined threshold, the content-prediction application selects the content item corresponding to the second vector.

Claims17

Dependent claims

A method comprising:

accessing a color palette that includes two or more color-palette categories;
selecting a first color of a color-palette category of the two or more color-palette categories;
identifying a set of pixel values that characterize the selected first color;
generating, based on the set of pixel values of the selected first color, a first vector that represents the first color at a first location within a color-embedding space;
identifying a second vector specifying a second location within the color-embedding space, wherein the second vector is identified by applying a convolutional neural network model on an image depicting an item that includes a second color, wherein applying the convolutional neural network model to the image to generate the second vector includes:
applying an convolutional layer of the convolutional neural network model to a set of pixels of the image to generate a first set of values that represent the set of pixels, the first set of values including a first number of values;
applying an embedding layer of the convolutional neural network model to the first number of values to generate a second set of values that represent the set of pixels of the image, wherein the second set of values includes a second number of values that is less than the first number;
and
generating the second vector that includes the second set of values, wherein the second set of values indicate location coordinates of the second location within the color-embedding space;
determining that a distance metric between the first vector and the second vector is less than a predetermined threshold, wherein the predetermined threshold indicates a degree of color similarity between the first vector and second vector;
selecting, based on the distance metric being less than the predetermined threshold, the item that includes the second color;
and
generating, for output on a user interface, an item catalog that includes the selected item.

10 more elements

A system comprising:

one or more processors;
and
a memory having stored thereon instructions that, upon execution by the one or more processors, cause the one or more processors to:
access a color palette that includes two or more color-palette categories;
select a first color of a color-palette category of the two or more color-palette categories;
identify a set of pixel values that characterize the selected first color;
generate, based on the set of pixel values of the selected first color, a first vector that represents the first color at a first location within a color-embedding space;
identify a second vector specifying a second location within the color-embedding space, wherein the second vector is identified by applying a convolutional neural network model on an image depicting an item that includes a second color, wherein applying the convolutional neural network model to the image to generate the second vector includes:
applying an convolutional layer of the convolutional neural network model to a set of pixels of the image to generate a first set of values that represent the set of pixels, the first set of values including a first number of values;
applying an embedding layer of the convolutional neural network model to the first number of values to generate a second set of values that represent the set of pixels of the image, wherein the second set of values includes a second number of values that is less than the first number;
and
generating the second vector that includes the second set of values, wherein the second set of values indicate location coordinates of the second location within the color-embedding space;
determine that a distance metric between the first vector and the second vector is less than a predetermined threshold, wherein the predetermined threshold indicates a degree of color similarity between the first vector and second vector;
select, based on the distance metric being less than the predetermined threshold, the item that includes the second color;
and
generate, for output on a user interface, an item catalog that includes the selected item.

13 more elements

A computer program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause one or more data processors to perform actions including:

accessing a color palette that includes two or more color-palette categories;
selecting a first color of a color-palette category of the two or more color-palette categories;
identifying a set of pixel values that characterize the selected first color;
generating, based on the set of pixel values of the selected first color, a first vector that represents the first color at a first location within a color-embedding space;
identifying a second vector specifying a second location within the color-embedding space, wherein the second vector is identified by applying a convolutional neural network model on an image depicting an item that includes a second color, wherein applying the convolutional neural network model to the image to generate the second vector includes:
applying an convolutional layer of the convolutional neural network model to a set of pixels of the image to generate a first set of values that represent the set of pixels, the first set of values including a first number of values;
applying an embedding layer of the convolutional neural network model to the first number of values to generate a second set of values that represent the set of pixels of the image, wherein the second set of values includes a second number of values that is less than the first number;
and
generating the second vector that includes the second set of values, wherein the second set of values indicate location coordinates of the second location within the color-embedding space;
determining that a distance metric between the first vector and the second vector is less than a predetermined threshold, wherein the predetermined threshold indicates a degree of color similarity between the first vector and second vector;
selecting, based on the distance metric being less than the predetermined threshold, the item that includes the second color;
and
generating, for output on a user interface, an item catalog that includes the selected item.

10 more elements

Dependencies are inferred from the claim text.