# US 2025/0165544 A1 · Hierarchical representation learning of user interest

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165544 A1
- Document ID: US20250165544A1
- Status: Published application
- Kind: A1
- Application number: 18/839,396
- Filed: 2022-07-21
- Published: 2025-05-22
- Earliest claimed: 2021-09-26
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Linjun SHOU; Xingyao ZHANG; Ming GONG; Daxin JIANG
- Web page: https://patentgrep.com/patent/US20250165544A1

## CPC

- G06F 16/337: {Profile generation, learning or modification}
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06F 16/367: {Ontology}
- G06F 16/906: Clustering; Classification
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 20/00: Machine learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure proposes a method, apparatus and computer program product for hierarchical representation learning of user interest. A historical content item sequence of a user may be obtained. A topic and a text of each historical content item in the historical content item sequence may be identified, to obtain a topic sequence and a text sequence corresponding to the historical content item sequence. A comprehensive topic representation may be generated based on the topic sequence. A comprehensive text representation may be generated based on the text sequence. A user interest representation of the user may be generated based on the comprehensive topic representation and the comprehensive text representation.

## Claims

### Claim 1

A method for hierarchical representation learning of user interest, comprising: obtaining a historical content item sequence of a user; identifying a topic and a text of each historical content item in the historical content item sequence, to obtain a topic sequence and a text sequence corresponding to the historical content item sequence; generating a comprehensive topic representation based on the topic sequence; generating a comprehensive text representation based on the text sequence; and generating a user interest representation of the user based on the comprehensive topic representation and the comprehensive text representation.

### Claim 2 (depends on 1)

The method of claim 1, wherein the comprehensive topic representation and the comprehensive text representation have different information abstraction levels.

### Claim 3 (depends on 1)

The method of claim 1, wherein the generating a comprehensive topic representation comprises: generating a topic representation sequence corresponding to the topic sequence; construct a topic graph corresponding to the topic sequence; and generating the comprehensive topic representation based on the topic representation sequence and the topic graph.

### Claim 4 (depends on 3)

The method of claim 3, wherein the constructing a topic graph comprises: determining a plurality of topic categories included in the topic sequence; setting the plurality of topic categories into a plurality of nodes; determining a set of edges among the plurality of nodes; and combining the plurality of nodes and the set of edges into the topic graph.

### Claim 5 (depends on 4)

The method of claim 4, wherein the determining a set of edges comprises, for every two nodes in the plurality of nodes: determine whether there is a transition between two topic categories corresponding to the two nodes according to the topic sequence; in response to determining that there is a transition between the two topic categories, determining a transition direction of the transition and a number of transitions corresponding to the transition direction; and determining a direction and a number of edges existing between the two nodes based on the determined transition direction and the determined number of transitions.

### Claim 6 (depends on 1)

The method of claim 1, wherein the generating a comprehensive text representation comprises: generating a comprehensive text attention representation through an attention mechanism based on the text sequence, and the generating a user interest representation comprises: generating the user interest representation based on the comprehensive topic representation and the comprehensive text attention representation.

### Claim 7 (depends on 1)

The method of claim 1, wherein the generating a comprehensive text representation comprises: generating a comprehensive text capsule representation using a capsule network based at least on the text sequence, and the generating a user interest representation comprises: generating the user interest representation based on the comprehensive topic representation and the comprehensive text capsule representation.

### Claim 8 (depends on 1)

The method of claim 1, wherein the generating a comprehensive text representation comprises: generating a comprehensive text attention representation through an attention mechanism based on the text sequence; and generating a comprehensive text capsule representation using a capsule network based at least on the text sequence, and the generating a user interest representation comprises: generating the user interest representation based on the comprehensive topic representation, the comprehensive text attention representation, and the comprehensive text capsule representation.

### Claim 9 (depends on 8)

The method of claim 8, wherein the comprehensive text attention representation and the comprehensive text capsule representation have different information abstraction levels.

### Claim 10 (depends on 7)

The method of claim 7, wherein the generating a comprehensive text capsule representation comprises: generating an interest capsule representation using the capsule network based on the text sequence; generating a target content item representation of a target content item; and generating the comprehensive text capsule representation through an attention mechanism based on the interest capsule representation and the target content item representation.

### Claim 11 (depends on 1)

The method of claim 1, further comprising: predicting a click probability of the user clicking a target content item based on the user interest representation and a target content item representation of the target content item.

### Claim 12 (depends on 11)

The method of claim 11, wherein the click probability is output through a click probability predicting model, and a training of the click probability predicting model comprises: constructing a training dataset, the training dataset including a plurality of positive samples and a plurality of negative sample sets corresponding to the plurality of positive samples; generating a plurality of posterior click probabilities corresponding to the plurality of positive samples; generating a prediction loss based on the plurality of posterior click probabilities; and optimizing the click probability predicting model through minimizing the prediction loss.

### Claim 13 (depends on 12)

The method of claim 12, wherein the generating a plurality of posterior click probabilities comprises, for each positive sample: predicting a click probability of the positive sample corresponding to the positive sample; for each negative sample in a negative sample set corresponding to the positive sample, predicting a negative sample click probability corresponding to the negative sample, to obtain a negative sample click probability set corresponding to the negative sample set; and calculating a posterior click probability corresponding to the positive sample based on the positive sample click probability and the negative sample click probability set.

### Claim 14

An apparatus for hierarchical representation learning of user interest, comprising: at least one processor; and a memory storing computer-executable instructions that, when executed, cause the at least one processor to: obtain a historical content item sequence of a user, identify a topic and a text of each historical content item in the historical content item sequence, to obtain a topic sequence and a text sequence corresponding to the historical content item sequence, generate a comprehensive topic representation based on the topic sequence, generate a comprehensive text representation based on the text sequence, and generate a user interest representation of the user based on the comprehensive topic representation and the comprehensive text representation.

### Claim 15

A computer program product for hierarchical representation learning of user interest, comprising a computer program that is executed by at least one processor for: obtaining a historical content item sequence of a user; identifying a topic and a text of each historical content item in the historical content item sequence, to obtain a topic sequence and a text sequence corresponding to the historical content item sequence; generating a comprehensive topic representation based on the topic sequence; generating a comprehensive text representation based on the text sequence; and generating a user interest representation of the user based on the comprehensive topic representation and the comprehensive text representation
