# US 2023/0394295 A1 · Supervised contrastive learning for related content recommendation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0394295 A1
- Document ID: US20230394295A1
- Status: Published application
- Kind: A1
- Application number: 17/805,262
- Filed: 2022-06-03
- Published: 2023-12-07
- Earliest claimed: 2022-06-03
- Assignees: Microsofttechnology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Chu WANG; Rui XIA; Zhiting CHEN; Li HUANG; Kun WU; Andrew Lawrence STEWART; Hanmu ZUO; Meenaz MERCHANT; Kamal GINOTRA; Saurajit MUKHERJEE; Arun K. SACHETI; Tingting WANG
- Web page: https://patentgrep.com/patent/US20230394295A1

## CPC

- G06F 16/9536: Search customisation based on social or collaborative filtering
- G06N 3/08: Learning methods
- G06F 18/2137: based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 3/04: Architecture, e.g. interconnection topology
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Aspects of the present disclosure relate to a providing related content recommendations in response to a user search query by supervising the training of pair embeddings using contrastive learning and pairwise co-click signals. The approach combines a two tower model architecture with a cascaded multilayer perceptron model to enable the adoption of variable combinations of input features and more representative learned pair embeddings. The learned embeddings undergo supervised contrastive loss training to generate a related content recommendation model, which is subsequently evaluated using both online and offline metrics. The related content recommendation model can provide results to search queries that improve recommendation quality and increase user engagement, thereby ultimately enhancing long term user experience.

## Claims

### Claim 1

A system comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations comprising:
obtaining a search query;
generating a set of features for a search query tower and a related content tower, wherein the search query tower and the related content tower are each based on the obtained search query;
training the search query tower and the related content tower;
generating a trained feature vector for each of the search query tower and related content tower;
training an inner product based on the search query trained feature vector and the related content trained feature vector; and
generating, based on the trained inner product, a related content model for generating a set of recommended content based on a user search query.

### Claim 2 (depends on 1)

The system of claim 1, wherein training the search query tower and related content tower further comprises utilizing a cascaded multilayer perceptron model comprised of multiple layers.

### Claim 3 (depends on 2)

The system of claim 2, wherein the cascaded multilayer perceptron model layers comprises one or more of an expand layer or a bottleneck layer, and between the multilayer perceptron layers data scaling is performed.

### Claim 4 (depends on 3)

The system of claim 3, wherein data scaling comprises one or more of data standardization or data normalization including batch normalization, activation, and dropout.

### Claim 5 (depends on 2)

The system of claim 2, wherein a skip connection is utilized between an initial input layer and another multilayer perceptron layer of the cascaded multilayer perceptron model.

### Claim 6 (depends on 1)

The system of claim 1, wherein generating the set of features further comprises pre-processing the search query to generate a set of low-dimensional features using at least one of feature scaling, centering, or dimensionality reduction.

### Claim 7 (depends on 1)

The system of claim 1, wherein the inner product is trained using self-supervised representation learning employing contrastive loss by tracking pairwise co-click signals associated with a plurality of search queries.

### Claim 8

A method comprising:
receiving a search request including a search query for content;
generating, using a related content model including a two-tower cascaded multilayer perceptron model, a set of recommended content associated with both the search query and an instance of content that is responsive to the search query; and
providing, in response to the search request, the generated set of recommended content in association with the instance of content that is responsive to the search query.

### Claim 9 (depends on 8)

The method of claim 8, wherein:
the instance of content is a first instance of content; and
the related content model is trained using a set of co-click signals that includes:
a positive pair between a second instance of content and a first instance of recommended content; and
a negative pair between the second instance of content and a second instance of recommended content.

### Claim 10 (depends on 8)

The method of claim 8, wherein a first tower of the two-tower cascaded multilayer perceptron model is associated with a first content type and a second tower of the two-tower cascaded multilayer perceptron model is associated with a second content type.

### Claim 11 (depends on 10)

The method of claim 10, wherein the first content type is a text content type associated with the search query and the second content type is an image content type.

### Claim 12 (depends on 8)

The method of claim 8, wherein the search request further includes an indication of the instance of content that is responsive to the search query.

### Claim 13 (depends on 8)

The method of claim 8, further comprising identifying the instance of content that is responsive to the search query.

### Claim 14

A method comprising:
obtaining a search query;
generating a set of features for a search query tower and a related content tower, wherein the search query tower and the related content tower are each based on the obtained search query;
training the search query tower and the related content tower;
generating a trained feature vector for each of the search query tower and related content tower;
training an inner product based on the search query trained feature vector and the related content trained feature vector; and
generating, based on the trained inner product, a related content model for generating a set of recommended content based on a user search query.

### Claim 15 (depends on 14)

The method of claim 14, wherein training the search query tower and related content tower further comprises utilizing a cascaded multilayer perceptron model comprised of multiple layers.

### Claim 16 (depends on 15)

The method of claim 15, wherein the cascaded multilayer perceptron model layers comprises one or more of an expand layer or a bottleneck layer, and between the multilayer perceptron layers data scaling is performed.

### Claim 17 (depends on 16)

The method of claim 16, wherein data scaling comprises one or more of data standardization or data normalization including batch normalization, activation, and dropout.

### Claim 18 (depends on 15)

The method of claim 15, wherein a skip connection is utilized between an initial input layer and another multilayer perceptron layer of the cascaded multilayer perceptron model.

### Claim 19 (depends on 1)

The method of claim 1, wherein generating the set of features further comprises pre-processing the search query to generate a set of low-dimensional features using at least one of feature scaling, centering, or dimensionality reduction.

### Claim 20 (depends on 14)

The method of claim 14, wherein the inner product is trained using self-supervised representation learning employing contrastive loss by tracking pairwise co-click signals associated with a plurality of search queries.
