# US 2021/0342928 A1 · Arranging information describing items within a page maintained in an online system based on an interaction with a link to the page

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342928 A1
- Document ID: US20210342928A1
- Status: Published application
- Kind: A1
- Application number: 16/864,729
- Filed: 2020-05-01
- Published: 2021-11-04
- Earliest claimed: 2020-05-01
- Assignees: Facebook, Inc.
- Applicants: Facebook, Inc.
- Inventors: Steve Morin; Xuewei Ouyang; Martin Mroz; Anuj Madan
- Web page: https://patentgrep.com/patent/US20210342928A1

## CPC

- G06Q 30/0631: {Recommending goods or services}
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06Q 10/40: Business processes related to social networking or social networking services
- G06Q 30/0277: {Online advertisement}
- G06Q 30/0623: {by investigating goods or services}
- G06Q 30/0643: {graphically representing goods, e.g. 3D product representation}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An online system receives information describing (an) item(s) associated with an entity and a content item including an image. The online system accesses and applies a trained machine-learning model to predict a probability that the content item includes an image of an item associated with the entity. If the probability is at least a threshold probability, a link to a page associated with the item that includes a set of the information describing the item(s) is added to the content item by the online system. Responsive to receiving an interaction with the link from a user presented with the content item, the online system determines a measure of similarity between the item and each additional item based on the information describing the item(s), arranges the set of the information describing the item(s) within the page based on the measure(s) of similarity, and sends the page for display to the user.

## Claims

### Claim 1

A method comprising:
receiving information describing one or more items associated with an entity having a presence on an online system, the information comprising one or more images of each of the one or more items;
receiving a content item to be presented to one or more viewing users of the online system, the content item comprising an image;
accessing a trained machine-learning model, the trained machine-learning model trained based at least in part on the information describing the one or more items;
applying the trained machine-learning model to a set of attributes of the content item to predict a probability that the content item comprises an image of an item associated with the entity;
determining whether the predicted probability is at least a threshold probability;
responsive to determining that the predicted probability is at least the threshold probability, adding, to the content item, a link to a page associated with the item maintained in the online system, wherein the page comprises a set of the information describing the one or more items;
determining an opportunity to present content to a viewing user of the online system;
sending the content item comprising the link for display to the viewing user;
responsive to receiving an interaction with the link from the viewing user, determining a measure of similarity between the item and each additional item of the one or more items, the measure of similarity determined based at least in part on the information describing the one or more items;
arranging the set of the information describing the one or more items within the page based at least in part on the measure of similarity between the item and each additional item of the one or more items; and
sending the page for display to the viewing user.

### Claim 2 (depends on 1)

The method of claim 1, wherein the information describing the one or more items comprises one or more of: a color associated with an item of the one or more items, a style associated with an item of the one or more items, a genre associated with an item of the one or more items, a material associated with an item of the one or more items, a type associated with an item of the one or more items, a brand associated with an item of the one or more items, an artist associated with an item of the one or more items, a version associated with an item of the one or more items, a model associated with an item of the one or more items, a set of specifications associated with an item of the one or more items, and a price associated with an item of the one or more items.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining the measure of similarity between the item and each additional item of the one or more items comprises:
for each of the one or more items, generating an embedding in an embedding space based at least in part on a set of the information describing a corresponding item;
identifying an embedding corresponding to the item; and
for each additional item of the one or more items:
identifying an additional embedding corresponding to the additional item,
determining a distance between the embedding corresponding to the item and the additional embedding corresponding to the additional item, and
determining the measure of similarity between the item and the additional item based at least in part on the distance.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining the measure of similarity between the item and each additional item of the one or more items comprises:
determining a set of metadata associated with each of the one or more items based at least in part on the information describing the one or more items;
generating a hierarchy of categories associated with the one or more items based at least in part on the set of metadata associated with each of the one or more items, the hierarchy of categories comprising a plurality of nodes and one or more edges, wherein each of the plurality of nodes represents a category and each of the one or more edges connecting a pair of the plurality of nodes represents a relationship between a plurality of categories represented by the pair of nodes;
identifying a node corresponding to a category of the hierarchy of categories associated with the item based at least in part on a set of the information describing the item; and
for each additional item of the one or more items:
identifying an additional node corresponding to an additional category of the hierarchy of categories associated with the additional item based at least in part on a set of the information describing a corresponding item,
determining a distance between the node corresponding to the category of the hierarchy of categories associated with the item and the additional node corresponding to the additional category of the hierarchy of categories associated with the additional item, and
determining the measure of similarity between the item and the additional item based at least in part on the distance.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
responsive to receiving the interaction with the link from the viewing user, identifying a set of the one or more items having at least a threshold measure of similarity to the item based at least in part on the measure of similarity between the item and each additional item of the one or more items; and
including information describing the item and the set of the one or more items having at least the threshold measure of similarity to the item in the page.

### Claim 6 (depends on 5)

The method of claim 5, wherein identifying the set of the one or more items having at least the threshold measure of similarity to the item comprises:
for each of the one or more items, generating an embedding in an embedding space based at least in part on a set of the information describing a corresponding item;
identifying an embedding corresponding to the item;
identifying a set of embeddings within a threshold distance of the embedding corresponding to the item; and
identifying the set of the one or more items having at least the threshold measure of similarity to the item, wherein the set of the one or more items correspond to the set of embeddings.

### Claim 7 (depends on 5)

The method of claim 5, wherein identifying the set of the one or more items having at least the threshold measure of similarity to the item comprises:
determining a set of metadata associated with each of the one or more items based at least in part on the information describing the one or more items;
generating a hierarchy of categories associated with the one or more items based at least in part on the set of metadata associated with each of the one or more items, the hierarchy of categories comprising a plurality of nodes and one or more edges, wherein each of the plurality of nodes represents a category and each of the one or more edges connecting a pair of the plurality of nodes represents a relationship between a plurality of categories represented by the pair of nodes;
identifying a node corresponding to a category of the hierarchy of categories associated with the item based at least in part on a set of the information describing the item;
for each additional item of the one or more items, identifying an additional node corresponding to an additional category of the hierarchy of categories associated with the additional item based at least in part on a set of the information describing a corresponding item;
identifying a set of nodes within a threshold distance of the node corresponding to the category of the hierarchy of categories associated with the item; and
identifying the set of the one or more items having at least the threshold measure of similarity to the item, wherein the set of the one or more items correspond to the set of nodes.

### Claim 8 (depends on 1)

The method of claim 1, wherein the set of attributes of the content item comprises one or more selected from the group consisting of: a set of pixel values associated with the image comprising the content item, a tag comprising the content item, a link comprising the content item, a caption comprising the content item, a location associated with the content item, a comment on the content item, an audience associated with the content item, an inferred relationship between the content item and a topic, and any combination thereof.

### Claim 9 (depends on 1)

The method of claim 1, wherein the content item is received from the entity.

### Claim 10 (depends on 1)

The method of claim 1, wherein the content item is received from a content-providing user of the online system other than the entity.

### Claim 11

A computer program product comprising a computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:
receive information describing one or more items associated with an entity having a presence on an online system, the information comprising one or more images of each of the one or more items;
receive a content item to be presented to one or more viewing users of the online system, the content item comprising an image;
access a trained machine-learning model, the trained machine-learning model trained based at least in part on the information describing the one or more items;
apply the trained machine-learning model to a set of attributes of the content item to predict a probability that the content item comprises an image of an item associated with the entity;
determine whether the predicted probability is at least a threshold probability;
responsive to determining that the predicted probability is at least the threshold probability, add, to the content item, a link to a page associated with the item maintained in the online system, wherein the page comprises a set of the information describing the one or more items;
determine an opportunity to present content to a viewing user of the online system;
send the content item comprising the link for display to the viewing user;
responsive to receiving an interaction with the link from the viewing user, determine a measure of similarity between the item and each additional item of the one or more items, the measure of similarity determined based at least in part on the information describing the one or more items;
arrange the set of the information describing the one or more items within the page based at least in part on the measure of similarity between the item and each additional item of the one or more items; and
send the page for display to the viewing user.

### Claim 12 (depends on 11)

The computer program product of claim 11, wherein the information describing the one or more items comprises one or more of: a color associated with an item of the one or more items, a style associated with an item of the one or more items, a genre associated with an item of the one or more items, a material associated with an item of the one or more items, a type associated with an item of the one or more items, a brand associated with an item of the one or more items, an artist associated with an item of the one or more items, a version associated with an item of the one or more items, a model associated with an item of the one or more items, a set of specifications associated with an item of the one or more items, and a price associated with an item of the one or more items.

### Claim 13 (depends on 11)

The computer program product of claim 11, wherein determine the measure of similarity between the item and each additional item of the one or more items comprises:
for each of the one or more items, generate an embedding in an embedding space based at least in part on a set of the information describing a corresponding item;
identify an embedding corresponding to the item; and
for each additional item of the one or more items:
identify an additional embedding corresponding to the additional item,
determine a distance between the embedding corresponding to the item and the additional embedding corresponding to the additional item, and
determine the measure of similarity between the item and the additional item based at least in part on the distance.

### Claim 14 (depends on 11)

The computer program product of claim 11, wherein determine the measure of similarity between the item and each additional item of the one or more items comprises:
determine a set of metadata associated with each of the one or more items based at least in part on the information describing the one or more items;
generate a hierarchy of categories associated with the one or more items based at least in part on the set of metadata associated with each of the one or more items, the hierarchy of categories comprising a plurality of nodes and one or more edges, wherein each of the plurality of nodes represents a category and each of the one or more edges connecting a pair of the plurality of nodes represents a relationship between a plurality of categories represented by the pair of nodes;
identify a node corresponding to a category of the hierarchy of categories associated with the item based at least in part on a set of the information describing the item; and
for each additional item of the one or more items:
identify an additional node corresponding to an additional category of the hierarchy of categories associated with the additional item based at least in part on a set of the information describing a corresponding item,
determine a distance between the node corresponding to the category of the hierarchy of categories associated with the item and the additional node corresponding to the additional category of the hierarchy of categories associated with the additional item, and
determine the measure of similarity between the item and the additional item based at least in part on the distance.

### Claim 15 (depends on 11)

The computer program product of claim 11, wherein the computer-readable storage medium further has instructions encoded thereon that, when executed by the processor, cause the processor to:
responsive to receiving the interaction with the link from the viewing user, identify a set of the one or more items having at least a threshold measure of similarity to the item based at least in part on the measure of similarity between the item and each additional item of the one or more items; and
include information describing the item and the set of the one or more items having at least the threshold measure of similarity to the item in the page.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein identify the set of the one or more items having at least the threshold measure of similarity to the item comprises:
for each of the one or more items, generate an embedding in an embedding space based at least in part on a set of the information describing a corresponding item;
identify an embedding corresponding to the item;
identify a set of embeddings within a threshold distance of the embedding corresponding to the item; and
identify the set of the one or more items having at least the threshold measure of similarity to the item, wherein the set of the one or more items correspond to the set of embeddings.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein identify the set of the one or more items having at least the threshold measure of similarity to the item comprises:
determine a set of metadata associated with each of the one or more items based at least in part on the information describing the one or more items;
generate a hierarchy of categories associated with the one or more items based at least in part on the set of metadata associated with each of the one or more items, the hierarchy of categories comprising a plurality of nodes and one or more edges, wherein each of the plurality of nodes represents a category and each of the one or more edges connecting a pair of the plurality of nodes represents a relationship between a plurality of categories represented by the pair of nodes;
identify a node corresponding to a category of the hierarchy of categories associated with the item based at least in part on a set of the information describing the item;
for each additional item of the one or more items, identify an additional node corresponding to an additional category of the hierarchy of categories associated with the additional item based at least in part on a set of the information describing a corresponding item;
identify a set of nodes within a threshold distance of the node corresponding to the category of the hierarchy of categories associated with the item; and
identify the set of the one or more items having at least the threshold measure of similarity to the item, wherein the set of the one or more items correspond to the set of nodes.

### Claim 18 (depends on 11)

The computer program product of claim 11, wherein the set of attributes of the content item comprises one or more selected from the group consisting of: a set of pixel values associated with the image comprising the content item, a tag comprising the content item, a link comprising the content item, a caption comprising the content item, a location associated with the content item, a comment on the content item, an audience associated with the content item, an inferred relationship between the content item and a topic, and any combination thereof

### Claim 19 (depends on 11)

The computer program product of claim 11, wherein the content item is received from the entity.

### Claim 20 (depends on 11)

The computer program product of claim 11, wherein the content item is received from a content-providing user of the online system other than the entity.
