# US 2025/0156641 A1 · Two-tower neural network for content-audience relationship prediction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156641 A1
- Document ID: US20250156641A1
- Status: Published application
- Kind: A1
- Application number: 18/388,726
- Filed: 2023-11-10
- Published: 2025-05-15
- Earliest claimed: 2023-11-10
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Xueqian Tang; Lijun Peng; Jiarui Wang; Yi Zhang; Yi Wu; Arvind Subramaniam
- Web page: https://patentgrep.com/patent/US20250156641A1

## CPC

- G06N 3/045: Combinations of networks
- G06F 40/30: Semantic analysis
- G06N 20/00: Machine learning
- G06Q 10/42: Determination of affinities or common interests between users
- G06Q 30/0242: Determining effectiveness of advertisements
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0475: Generative networks
- G06N 3/048: Activation functions
- G06N 3/096: Transfer learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

In an example embodiment, a generator model such as a large language model (LLM) is leveraged to generate embeddings for both pieces of content and users. The embeddings map the pieces of content and the users into the same latent n-dimensional space. The embeddings are then fine-tuned using a two-tower deep neural network, with one of the towers representing users and the other tower representing content. The two-tower deep neural network is trained to optimize the embeddings over some shared goal, such as user engagement with content, and uses information such as user interactions with content in that process. A clustering technique, such as K-nearest neighbor (kNN) can then be used to identify a grouping of top user/content pairs based on similarity between users and content, as reflected in the embeddings. For a given piece of content, therefore, the top users from that cluster can then be recommended as an audience for the content.

## Claims

### Claim 1

A system comprising:
a non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processor, cause the system to perform operations comprising:
passing data about a first user to a first generator model to produce a user semantic embedding, the user semantic embedding being a first point in a latent n-dimensional space, the first point representing a semantic meaning of the data about the first user;
passing the data about a first piece of content to the first generator model to produce a content semantic embedding, the content semantic embedding being a second point in the latent n-dimensional space, the second point representing a semantic meaning of the data about the first piece of content; and
feeding the user semantic embedding and the content semantic embedding to a two-tower deep neural network trained to predict a probability of a first type of engagement between the first user and the first piece of content, the two-tower deep neural network fine-tuning the user semantic embedding into a user embedding and the content semantic embedding into a content embedding, each tower of the two-tower deep neural network trained using shared engagement data, the probability determined based on a geometric relationship between the user embedding and the content embedding.

### Claim 2 (depends on 1)

The system of claim 1, wherein the operations further comprise:
recommending the first piece of content be displayed to the first user in lieu of one or more other pieces of content to be displayed to the first user, based upon the predicted probability.

### Claim 3 (depends on 1)

The system of claim 1, wherein the operations further comprise:
recommending the first piece of content be displayed to the first user in lieu of the first piece of content being displayed to one or more users, based upon the predicted probability.

### Claim 4 (depends on 1)

The system of claim 1, wherein the shared engagement data includes information about interactions by users with landing pages of web sites associated with pieces of content.

### Claim 5 (depends on 1)

The system of claim 1, wherein the two-tower deep neural network is trained to predict both a probability of a first type of engagement between the first user and the first piece of content and a probability of a second type of engagement between the first user and the first piece of content by training the two-tower deep neural network using a multi-task training technique.

### Claim 6 (depends on 5)

The system of claim 5, wherein the training includes including a first matching layer for the first type of engagement and a second matching layer for the second type of engagement in the two-tower deep neural network.

### Claim 7 (depends on 1)

The system of claim 1, wherein the operations further comprise:
predicting an audience based on the predicted probability and on other predicted probabilities for other users interactions with the first piece of content;
causing display of the first piece of content to users in the predicted audience;
measuring interactions between the users in the predicted audience and the first piece of content; and
retraining the two-tower deep neural network based on the measured interactions.

### Claim 8 (depends on 1)

The system of claim 1, wherein the operations further comprise:
causing one or more pieces of content to be presented in a first campaign to a group of users based on the geometric relationship between user embeddings of the group of users and content embeddings for the one or more pieces of content;
measuring a success rate of the first campaign based on interactions by users in the first group of users with the presented one or more pieces of content; and
based on the success rate, presenting insights as to why the first campaign was or was not successful, using the user embeddings of the group of users and the content embeddings for the one or more pieces of content.

### Claim 9 (depends on 1)

The system of claim 1, using a second generator model to generate a summary from a landing page associated with the first piece of content; and
wherein the passing the data about the first piece of content to the first generator model includes passing the summary to the first generator model.

### Claim 10

A method comprising:
passing data about a first user to a first generator model to produce a user semantic embedding, the user semantic embedding being a first point in a latent n-dimensional space, the first point representing a semantic meaning of the data about the first user;
passing the data about a first piece of content to the first generator model to produce a content semantic embedding, the content semantic embedding being a second point in the latent n-dimensional space, the second point representing a semantic meaning of the data about the first piece of content; and
feeding the user semantic embedding and the content semantic embedding to a two-tower deep neural network trained to predict a probability of a first type of engagement between the first user and the first piece of content, the two-tower deep neural network fine-tuning the user semantic embedding into a user embedding and the content semantic embedding into a content embedding, each tower of the two-tower deep neural network trained using shared engagement data, the probability determined based on a geometric relationship between the user embedding and the content embedding.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
recommending the first piece of content be displayed to the first user in lieu of one or more other pieces of content to be displayed to the first user, based upon the predicted probability.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
recommending the first piece of content be displayed to the first user in lieu of the first piece of content being displayed to one or more users, based upon the predicted probability.

### Claim 13 (depends on 10)

The method of claim 10, wherein the shared engagement data includes information about interactions by users with landing pages of web sites associated with pieces of content.

### Claim 14 (depends on 10)

The method of claim 10, wherein the two-tower deep neural network is trained to predict both a probability of a first type of engagement between the first user and the first piece of content and a probability of a second type of engagement between the first user and the first piece of content by training the two-tower deep neural network using a multi-task training technique.

### Claim 15 (depends on 14)

The method of claim 14, wherein the training includes including a first matching layer for the first type of engagement and a second matching layer for the second type of engagement in the two-tower deep neural network.

### Claim 16 (depends on 10)

The method of claim 10, further comprising:
predicting an audience based on the predicted probability and on other predicted probabilities for other users' interactions with the first piece of content;
causing display of the first piece of content to users in the predicted audience;
measuring interactions between the users in the predicted audience and the first piece of content; and
retraining the two-tower deep neural network based on the measured interactions.

### Claim 17 (depends on 10)

The method of claim 10, further comprising:
causing one or more pieces of content to be presented in a first campaign to a group of users based on the geometric relationship between user embeddings of the group of users and content embeddings for the one or more pieces of content;
measuring a success rate of the first campaign based on interactions by users in the first group of users with the presented one or more pieces of content; and
based on the success rate, presenting insights as to why the first campaign was or was not successful, using the user embeddings of the group of users and the content embeddings for the one or more pieces of content.

### Claim 18 (depends on 10)

The method of claim 10, using a second generator model to generate a summary from a landing page associated with the first piece of content; and
wherein the passing the data about the first piece of content to the first generator model includes passing the summary to the first generator model.

### Claim 19

A non-transitory machine-readable storage medium comprising instructions which, when implemented by one or more machines, cause the one or more machines to perform operations comprising:
passing data about a first user to a first generator model) to produce a user semantic embedding, the user semantic embedding being a first point in a latent n-dimensional space, the first point representing a semantic meaning of the data about the first user;
passing the data about a first piece of content to the first generator model to produce a content semantic embedding, the content semantic embedding being a second point in the latent n-dimensional space, the second point representing a semantic meaning of the data about the first piece of content; and
feeding the user semantic embedding and the content semantic embedding to a two-tower deep neural network trained to predict a probability of a first type of engagement between the first user and the first piece of content, the two-tower deep neural network fine-tuning the user semantic embedding into a user embedding and the content semantic embedding into a content embedding, each tower of the two-tower deep neural network trained using shared engagement data, the probability determined based on a geometric relationship between the user embedding and the content embedding.

### Claim 20 (depends on 19)

The non-transitory machine-readable storage medium of claim 19, wherein the operations further comprise:
recommending the first piece of content be displayed to the first user in lieu of one or more other pieces of content to be displayed to the first user, based upon the predicted probability.
