# US 2024/0005170 A1 · Recommendation method, apparatus, electronic device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005170 A1
- Document ID: US20240005170A1
- Status: Published application
- Kind: A1
- Application number: 18/253,966
- Filed: 2021-11-17
- Published: 2024-01-04
- Earliest claimed: 2020-11-23
- Assignees: Zte Corporation
- Applicants: Zte Corporation
- Inventors: Yaofeng TU; Jin XU; Dezheng WANG; Sen FAN
- Web page: https://patentgrep.com/patent/US20240005170A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/092: Reinforcement learning
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06F 16/9537: Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
- G06N 3/098: Distributed learning, e.g. federated learning
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06Q 30/0201: Market modelling; Market analysis; Collecting market data
- G06Q 30/0271: {Personalized advertisement}
- G06Q 30/0631: {Recommending goods or services}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Disclosed are a recommendation method, an apparatus, an electronic device, and a storage medium. The recommendation method includes: generating an item recommendation rank for a user based on a policy network in response to a user request at moment t; obtaining an experience at the moment t based on a feedback result from the user on the item recommendation rank; aggregating the experience of each user at the moment t to a global experience pool; and obtaining an updated policy network based on experiences in the global experience pool.

## Claims

### Claim 1

A recommendation method, comprising:
generating an item recommendation rank for a user based on a policy network in response to a user request at moment t;
obtaining an experience at the moment t based on a feedback result from the user on the item recommendation rank;
aggregating the experience of each user at the moment t to a global experience pool; and
obtaining an updated policy network based on experiences in the global experience pool.

### Claim 2 (depends on 1)

The recommendation method according to claim 1, wherein the experience comprises: a state at the moment t, an initial score of an item at the moment t, a reward at the moment t, and a state at moment t+1;
wherein the initial score of the item at the moment t is obtained based on the state at the moment t and the policy network, and the reward at the moment t is obtained based on the feedback result from the user on the item recommendation rank.

### Claim 3 (depends on 2)

The recommendation method according to claim 2, wherein the generating the item recommendation rank for the user based on the policy network in response to the user request at the moment t comprises:
obtaining a state of the user at the moment t;
obtaining the initial score of the item at the moment t according to the state at the moment t and the policy network;
obtaining a weight of the item according to a matching degree between the user and the item; and
generating the item recommendation rank based on the initial score at the moment t and the weight of the item.

### Claim 4 (depends on 2)

The recommendation method according to claim 2, wherein the feedback result comprises: a primary click, a secondary click and a page number in a recommendation page where the item is located;
obtaining the reward at the moment t based on the feedback result from the user on the item recommendation rank comprises:
obtaining the reward at the moment t based on the primary click, the secondary click and a preset function;
wherein an independent variable of the preset function is the page number in the recommendation page where the item is located, and the preset function represents a variation tendency of the reward at the moment t with a variation of the page number in the recommendation page.

### Claim 5 (depends on 3)

The recommendation method according to claim 3, wherein before the generating the item recommendation rank for the user based on the policy network in response to the user request at the moment t, the recommendation method further comprises:
obtaining an entity feature vector based on a knowledge graph and a knowledge graph embedding model, wherein the knowledge graph is obtained based on user information and item information;
the state of the user at the moment t comprises: a user feature vector, an item feature vector, a user context feature vector; and
the obtaining the state of the user at the moment t comprises:
obtaining the user feature vector, the item feature vector, the user context feature vector at the moment t according to the entity feature vector.

### Claim 6 (depends on 1)

The recommendation method according to claim 1, wherein the obtaining the updated policy network based on the experiences in the global experience pool comprises:
sampling from the global experience pool to obtain an experience set; and
obtaining the updated policy network based on the experience set.

### Claim 7 (depends on 6)

The recommendation method according to claim 6, wherein the sampling from the global experience pool to obtain the experience set comprises:
obtaining a time difference error of each experience in the global experience pool;
obtaining a priority of each experience based on the time difference error of each experience; and
sampling from the global experience pool to obtain the experience set based on the priority of each experience;
wherein the time difference error is determined based on the reward at the moment t obtained by the feedback result from the user on the item recommendation rank and an expected reward obtained based on the state of the user at the moment t.

### Claim 8

(canceled)

### Claim 9 (depends on 1)

An electronic device, comprising:
at least one processor; and
a memory communicated with the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor to enable the at least one processor to perform the recommendation method according to claim 1.

### Claim 10 (depends on 1)

A non-transitory computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the recommendation method according to claim 1.
