# US 2020/0288205 A1 · Method, apparatus, electronic device, and storage medium for recommending multimedia resource

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0288205 A1
- Document ID: US20200288205A1
- Status: Published application
- Kind: A1
- Application number: 16/884,678
- Filed: 2020-05-27
- Published: 2020-09-10
- Earliest claimed: 2019-05-27
- Assignees: Beijing Dajia Internet Information Technology Co., Ltd.
- Applicants: Beijing Dajia Internet Information Technology Co., Ltd.
- Inventors: Wenkui DING; Yan LI
- Web page: https://patentgrep.com/patent/US20200288205A1

## CPC

- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- G06F 18/2113: by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
- G06F 18/251: {of input or preprocessed data}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/771: Feature selection, e.g. selecting representative features from a multi-dimensional feature space
- G06V 10/803: {of input or preprocessed data}
- G06V 10/82: using neural networks
- H04N 21/252: {Processing of multiple end-users' preferences to derive collaborative data}
- H04N 21/25891: {being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)}
- H04N 21/4668: {for recommending content, e.g. movies}
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

The present disclosure provides a method, an apparatus, an electronic device, and a storage medium for recommending multimedia resource, and relates to the field of machine learning. The method includes: acquiring features of the multimedia resource based on a convolutional neural network, where the convolutional neural network comprises N convolutional layers, where N is a positive integer; determining user interest information based on an identifier of a recommended user, where the user interest information is corresponding to the feature of each convolutional layer; determining a first feature matrix based on the convolution of convolution kernel and the feature, where the convolution kernel comprises the user interest information; generating user preference data based on the first feature matrix; and recommending the multimedia resource to the recommended user based on the N generated user preference data.

## Claims

### Claim 1

A method for recommending multimedia resource, comprising:
acquiring features of the multimedia resource based on a convolutional neural network, wherein the convolutional neural network comprises N convolutional layers, wherein N is a positive integer;
determining user interest information based on an identifier of a recommended user, wherein the user interest information is corresponding to the feature of each convolutional layer;
determining a first feature matrix based on the convolution of convolution kernel and the feature, wherein the convolution kernel comprises the user interest information;
generating user preference data based on the first feature matrix; and
recommending the multimedia resource to the recommended user based on the N generated user preference data.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
determining user attention information based on the identifier, wherein the user attention information matches with the features;
generating a user attention weight based on the user attention information and the feature; and
generating the user preference data based on the first feature matrix and the user attention weight.

### Claim 3 (depends on 2)

The method according to claim 2, wherein said generating the user attention weight comprises:
determining a second feature matrix based on the convolution of the convolution kernel and the feature, wherein the convolution kernel comprises the user attention information; and
obtaining the user attention weight by normalizing the second feature matrix.

### Claim 4 (depends on 2)

The method according to claim 2, wherein said generating the user preference data comprises:
reducing a dimension of the first feature matrix by a maximum pooling layer; and
determining the user preference data by weighting and fusing the dimension-reduced first feature matrix and the user attention weight.

### Claim 5 (depends on 1)

The method according to claim 1, wherein said recommending the multimedia resource to the recommended user comprises:
acquiring a target weight matrix, wherein the target weight matrix provides weights corresponding to the N generated user preference data;
weighting and fusing the N generated user preference data based on the target weight matrix; and
recommending the multimedia resource to the recommended user based on the weighted and fused user preference data.

### Claim 6 (depends on 1)

The method according to claim 1, wherein said determining the user interest information comprises:
determining the user interest information by vectorizing the identifier based on a first embedding layer, wherein the user interest information is a matrix with an m*m size, and m indicates a width of the convolution kernel used by the each convolutional layer.

### Claim 7 (depends on 2)

The method according to claim 2, wherein said determining the user attention information comprises:
determining the user attention information by vectorizing the identifier based on a second embedding layer;
wherein the user attention information is a matrix with an m*m size, and m indicates a width of the convolution kernel used by the each convolutional layer; and a second weight matrix of the second embedding layer is different from a first weight matrix of a first embedding layer.

### Claim 8 (depends on 3)

The method according to claim 3, wherein the second feature matrix is normalized by a following formula:
wi,s,t=exp(v_{i,s,t})/Σs′,t′exp(v_{i,s′,t′});
wherein wi,s,t refers to the user attention weight; v_{i, s, t} and v_{i, s′, t′} refer to two elements at different positions in the second feature matrix; and s′ and t′ are positive integers for traversing all elements in the second feature matrix.

### Claim 9 (depends on 5)

The method according to claim 5, wherein weighting and fusing the N generated user preference data by a following formula:
o
i
=
r
ij
*

o
ij
∑
j

r
ij
;
wherein oi refers to the fused user preference data; j ranges from 1 to N; oij refers to the user preference data corresponding to a feature output from a jth convolutional layer; rij refers to a weight corresponding to oij in the target weight matrix; the target weight matrix is (ri1, ri2, ri3, . . . , riL; and i refers to the identifier of the recommended user.

### Claim 10

An apparatus for recommending multimedia resource, comprising:
a processor; and
a memory, configured to store executable instructions of the processor,
wherein the processor is configured to read and execute the executable instructions to:
acquire features of the multimedia resource based on a convolutional neural network, wherein the convolutional neural network comprises N convolutional layers, wherein N is a positive integer;
determine user interest information based on an identifier of a recommended user, wherein the user interest information is corresponding to the feature of each convolutional layer;
determine a first feature matrix based on the convolution of convolution kernel and the feature, wherein the convolution kernel comprises the user interest information;
generate user preference data based on the first feature matrix; and
recommend the multimedia resource to the recommended user based on the N generated user preference data.

### Claim 11 (depends on 10)

The apparatus according to claim 10, wherein the processor is further configured to read and execute the executable instructions to:
determine user attention information based on the identifier, wherein the user attention information matches with the features;
generate a user attention weight based on the user attention information and the feature; and
generate the user preference data based on the first feature matrix and the user attention weight.

### Claim 12 (depends on 11)

The apparatus according to claim 11, wherein the processor is further configured to read and execute the executable instructions to:
determine a second feature matrix based on the convolution of the convolution kernel and the feature, wherein the convolution kernel comprises the user attention information; and
obtain the user attention weight by normalizing the second feature matrix.

### Claim 13 (depends on 11)

The apparatus according to claim 11, wherein the processor is further configured to read and execute the executable instructions to:
reduce a dimension of the first feature matrix by a maximum pooling layer; and
determine the user preference data by weighting and fusing the dimension-reduced first feature matrix and the user attention weight.

### Claim 14 (depends on 11)

The apparatus according to claim 11, wherein the processor is further configured to read and execute the executable instructions to:
acquire a target weight matrix, wherein the target weight matrix provides weights corresponding to the N generated user preference data;
weight and fuse the N generated user preference data based on the target weight matrix; and
recommend the multimedia resource to the recommended user based on the weighted and fused user preference data.

### Claim 15 (depends on 10)

The apparatus according to claim 10, wherein the processor is further configured to read and execute the executable instructions to:
determine the user interest information by vectorizing the identifier based on a first embedding layer; wherein the user interest information is a matrix with an m*m size, and m indicates a width of the convolution kernel used by the each convolutional layer.

### Claim 16 (depends on 11)

The apparatus according to claim 11, wherein the processor is further configured to read and execute the executable instructions to:
determine the user attention information by vectorizing the identifier based on a second embedding layer;
wherein the user attention information is a matrix with an m*m size, and m indicates a width of the convolution kernel used by the each convolutional layer; and a second weight matrix of the second embedding layer is different from a first weight matrix of a first embedding layer.

### Claim 17 (depends on 13)

The apparatus according to claim 13, wherein the second feature matrix is normalized by a following formula:
wi,s,t=exp(v_{i,s,t})/Σs′,t′exp(v_{i,s′,t′});
wherein wi,s,t refers to the user attention weight; v_{i, s, t} and v_{i, s′, t′} refer to two elements at different positions in the second feature matrix; and s′ and t′ are positive integers for traversing all elements in the second feature matrix.

### Claim 18 (depends on 14)

The apparatus according to claim 14, wherein the N generated user preference data are weighted and fused by a following formula:
o
i
=
r
ij
*

o
ij
∑
j

r
ij
;
wherein oi refers to the fused user preference data; j ranges from 1 to N; oij refers to the user preference data corresponding to a feature output from a jth convolutional layer; rij refers to a weight corresponding to oij in the target weight matrix; the target weight matrix is (ri1, ri2, ri3, . . . riL); and i refers to the identifier of the recommended user.

### Claim 19

A non-transitory computer readable storage medium, wherein instructions in the storage medium are executed by a processor of an electronic device to cause the electronic device to:
acquire features of the multimedia resource based on a convolutional neural network, wherein the convolutional neural network comprises N convolutional layers, wherein N is a positive integer;
determine user interest information based on an identifier of a recommended user, wherein the user interest information is corresponding to the feature of each convolutional layer;
determine a first feature matrix based on the convolution of convolution kernel and the feature, wherein the convolution kernel comprises the user interest information;
generate user preference data based on the first feature matrix; and
recommend the multimedia resource to the recommended user based on the N generated user preference data.