# US 2019/0318227 A1 · Recommendation system and method for estimating the elements of a multi-dimensional tensor on geometric domains from partial observations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318227 A1
- Document ID: US20190318227A1
- Status: Published application
- Kind: A1
- Application number: 15/952,984
- Filed: 2018-04-13
- Published: 2019-10-17
- Earliest claimed: 2018-04-13
- Assignees: Fabula Al Limited
- Applicants: Fabula Al Limited
- Inventors: Michael Bronstein; Federico Monti; Xavier Bresson
- Web page: https://patentgrep.com/patent/US20190318227A1

## CPC

- G06N 3/08: Learning methods
- G06F 16/24578: {using ranking}
- G06F 16/9535: Search customisation based on user profiles and personalisation
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and method for producing a recommendation of a plurality of items to a plurality of users are provided. A method for producing a recommendation of a plurality of items to a plurality of users can include: obtaining a subset of multi-dimensional tensor elements representing scores given to a subset of items by a subset of users; obtaining a plurality of geometric domains corresponding to a subset of the dimensions of said multi-dimensional tensor; computing multi-dimensional tensor features by applying at least a multi-domain intrinsic convolutional layer on the multi-dimensional tensor elements; computing a full set of multi-dimensional tensor elements from the multi-dimensional tensor features; and using said full set of multi-dimensional tensor elements to determine recommendation of said plurality of items to said plurality of users.

## Claims

### Claim 1

A computer implemented method for producing a recommendation of a plurality
of items to a plurality of users, comprising the steps of:
obtaining a subset of multi-dimensional tensor elements representing scores given to a subset of items by a subset of users;
obtaining a plurality of geometric domains corresponding to a subset of the dimensions of said multi-dimensional tensor;
computing multi-dimensional tensor features by applying at least a multi-domain intrinsic convolutional layer on the multi-dimensional tensor elements;
computing a full set of multi-dimensional tensor elements from the multi-dimensional tensor features;
using said full set of multi-dimensional tensor elements to determine recommendation of said plurality of items to said plurality of users.

### Claim 2 (depends on 1)

The method of claim 1, where each of the geometric domains is one of the following types:
a manifold;
a parametric surface;
an implicit surface;
a mesh;
a point cloud;
an undirected weighted or unweighted graph;
a directed weighted or unweighted graph;
and the geometric domains are all of the same type or of different types.

### Claim 3 (depends on 1)

The method of claim 1, where the multi-dimensional tensor is a two-dimensional matrix.

### Claim 4 (depends on 1)

The method of claim 1, where said step of obtaining a subset of multi-dimensional tensor elements comprising inputting said subset of multi-dimensional tensor elements.

### Claim 5 (depends on 1)

The method of claim 1, where said step of obtaining a plurality of geometric domains comprises inputting said plurality of geometric domains.

### Claim 6 (depends on 1)

The method of claim 1, where the step of computing the multi-dimensional tensor elements from the multi-dimensional tensor features further comprises the step of applying a neural network comprising at least a linear layer on the multi-dimensional tensor features.

### Claim 7 (depends on 1)

The method of claim 1, where the step of computing the multi-dimensional tensor elements from the multi-dimensional tensor features further comprises the steps of
obtaining the multi-dimensional tensor features;
computing a sequence of incremental updates of intermediate multi-dimensional tensor elements; and
using said sequence of incremental updates of intermediate multi-dimensional tensor elements for computing said multi-dimensional tensor elements.

### Claim 8 (depends on 7)

The method of claim 7, where the step of computing a sequence of incremental updates of the intermediate multi-dimensional tensor elements comprises applying at least one iteration of a recurrent process to the obtained multi-dimensional tensor features.

### Claim 9 (depends on 8)

The method of claim 8, where the recurrent process is implemented as one of the following:
a recurrent neural network;
a long-short term memory network.

### Claim 10 (depends on 7)

The method of claim 7, where the step of computing the multi-dimensional tensor elements comprises summing up the sequence of incremental updates of the intermediate multi-dimensional tensor elements.

### Claim 11 (depends on 1)

The method of claim 1, where the multi-dimensional tensor is given as a product of a plurality of factors.

### Claim 12 (depends on 11)

The method of claim 11, where the multi-dimensional tensor is a two-dimensional matrix given as a product of two factors.

### Claim 13 (depends on 11)

The method of claim 11, where the multi-dimensional tensor features comprise the factor features of each of the factors, and where the step of computing the multi-dimensional tensor features comprises applying at least an intrinsic convolutional layer to each of the factors to compute the factor features.

### Claim 14 (depends on 11)

The method of claim 11, where the step of computing the multi-dimensional tensor elements further comprises the steps of:
computing the elements of each of the factors, and
computing the product of said factors.

### Claim 15 (depends on 11)

The method of claim 11, where the multi-dimensional tensor features comprise the factor features of each of the factors, and where the step of computing the multi-dimensional tensor features comprises applying at least a multi-domain intrinsic convolutional layer to at least one of the factors to compute the factor features.

### Claim 16 (depends on 11)

The method of claim 11, where each of the factors has a corresponding geometric domain.

### Claim 17 (depends on 11)

The method of claim 11, where only a subset of the factors has corresponding geometric domains, and the remaining factors have no corresponding geometric domains.

### Claim 18 (depends on 17)

The method of claim 17, where the step of computing the factor elements comprises the steps of:
computing factor elements of the factors having corresponding geometric domains by applying at least an intrinsic convolutional layer to each of the said factors to compute the factor features.

### Claim 19 (depends on 13)

The method of claim 13, where the step of computing the multi-dimensional tensor elements further comprises the steps of:
for each of the multi-dimensional tensor factors obtaining the factor features;
computing a sequence of incremental updates of intermediate factor elements;
computing said factor elements using said sequence of incremental updates of intermediate factor elements;
computing the multi-dimensional tensor elements by using the factor elements of all the factors.

### Claim 20 (depends on 19)

The method of claim 19, where the step of computing a sequence of incremental updates of intermediate factor elements comprises applying at least one iteration of a recurrent process to the factor features.

### Claim 21 (depends on 20)

The method of claim 20, where the recurrent process is implemented as one of the following
a recurrent neural network;
a long-short term memory network.

### Claim 22 (depends on 19)

The method of claim 19, where the step of computing the factor elements comprises summing up the sequence of incremental updates of the intermediate factor elements.

### Claim 23 (depends on 1)

The method according to claim 1, further applying at least one of the following layers:
a linear layer, including outputting a weighted linear combination of input data;
a non-linear layer, including applying a non-linear function to input data;
a spatial pooling layer, including:
determining a subset of points on the geometric domain;
for each point of said subset, determining the neighbours on the geometric domain; and
computing an aggregation operation on input data over the neighbours for all the points of said subset;
a fully connected layer, including outputting a weighted linear combination of input data at all the points of the geometric domain;
a regularization layer,
wherein each layer has input data and output data and output data of one layer are given as input data to another layer.

### Claim 24 (depends on 23)

The method of claim 23, wherein two or more of said layers are applied in sequence, and the output data of one layer in the sequence is given as input data to a subsequent layer in the sequence.

### Claim 25 (depends on 23)

The method of claim 23, wherein the aggregation operation in a spatial pooling layer comprises one of the following:
maximum computation;
average computation;
weighted average computation;
average of squares computation.

### Claim 26 (depends on 1)

The method of claim 1, where the multi-domain intrinsic convolutional layer is one of the following:
a spectral multi-domain intrinsic convolutional layer;
a spectrum-free multi-domain intrinsic convolutional layer;
a spatial multi-domain intrinsic convolutional layer.

### Claim 27 (depends on 13)

The method of claim 13, wherein the intrinsic convolutional layer is one of the following:
spectral convolutional layer;
spectrum-free convolutional layer;
spatial convolutional layer.

### Claim 28 (depends on 23)

The method of claim 23, where the regularization layer comprises at least one of the following:
a drop out of an arbitrary percentage of layer variables;
a quadratic penalty of the variables.

### Claim 29 (depends on 1)

The method of claim 1, where for each of the dimensions of the multi-dimensional tensor, a geometric domain is provided as input.

### Claim 30 (depends on 1)

The method of claim 1, where the step of obtaining the geometric domains further comprises the steps of:
inputting geometric domains corresponding to a provided subset of the dimensions of the multi-dimensional tensor;
computing the geometric domains corresponding to the non-provided subset of the dimensions of the multi-dimensional tensor from the subset of multi-dimensional tensor elements.

### Claim 31 (depends on 30)

The method of claim 30, where the step of computing the geometric domain corresponding to the non-provided subset of the dimensions of the multi-dimensional tensor comprises the following steps, for each of the non-provided dimensions:
extracting multi-dimensional tensor elements along said dimension and representing them as vectors;
computing a metric between each pair of said vectors;
building a graph, whose edges are weighted according to the said metric.

### Claim 32 (depends on 30)

The method of claim 30, where the step of computing the geometric domain corresponding to the non-provided subset of the dimensions of the multi-dimensional tensor comprises the following steps, for each of the non-provided dimensions:
collecting multi-dimensional features representing the general behavior of entries of said dimension and represent them as vectors;
computing a metric between each pair of said vectors;
building a graph, whose edges are weighted according to the said metric.

### Claim 33 (depends on 23)

The method of claim 23, wherein more than one of said layers are applied and wherein parameters of the applied layers comprise one or more of the following:
weights and biases of the linear layers;
parameters of the multi-domain intrinsic convolutional layers, comprising one or more of the following:
spectral multipliers of multi-domain filters;
parameters of the spectrum-free multi-domain filter expansion;
parameters of the weighting functions used to compute the patch operators in the spatial multi-domain intrinsic convolutional layer;

### Claim 34 (depends on 23)

The method of claim 23, wherein more than one of said layers are applied and wherein parameters of the applied layers comprise one or more of the following:
elements of the factors having no corresponding geometric domains;
weights and biases of the linear layers;
parameters of the intrinsic convolutional layers, comprising one or more of the following:
spectral multipliers of filters;
parameters of the spectrum-free filter expansion;
parameters of the weighting functions used to compute the patch operators in the spatial intrinsic convolutional layer.

### Claim 35 (depends on 33)

The method of claim 33, wherein parameters of the applied layers are determined by minimizing a cost function by means of an optimization procedure.

### Claim 36 (depends on 35)

The method of claim 35, where the optimization procedure comprises minimizing one or more of the following:
the discrepancy between the input known multi-dimensional tensor elements and the corresponding subset of the computed multi-dimensional tensor elements;
a criterion of smoothness of multi-dimensional tensor elements;
a surrogate of the rank of the multi-dimensional tensor;
norms of the multi-dimensional tensor factors.

### Claim 37 (depends on 36)

The method of claim 36, where the criterion of smoothness is the Dirichlet norm on the respective on the respective geometric domains.

### Claim 38 (depends on 33)

The method of claim 33, wherein parameters of the applied layers further comprise parameters of the geometric domains.

### Claim 39 (depends on 38)

The method of claim 38, wherein the parameters of the geometric domains comprise the metrics of said geometric domains.

### Claim 40 (depends on 38)

The method of claim 38, wherein the at least one of the geometric domains is a graph and the parameters of said geometric domain are the edge weights of the graph.

### Claim 41 (depends on 40)

The method of claim 40, wherein the vertices of the graph are points in a feature space, and edge weights are computed by applying a parametric metric between pairs of points in said feature space, and the parameters of the geometric domain comprise the parameters of said parametric metric.

### Claim 42 (depends on 41)

The method of claim 41, wherein parameters of the applied layers and parameters of the geometric domains are determined by minimizing a cost function by means of an optimization procedure.

### Claim 43 (depends on 1)

The method of claim 1, where the step of using the full set of multi-dimensional tensor elements to output recommendation of a plurality of items to a plurality of users further comprising producing for each user a list of recommended items.

### Claim 44 (depends on 43)

The method of claim 43, the step of producing for each user a list of recommended items comprises at least one of the following:
sorting the subset of the multi-dimensional tensor elements corresponding to a user from the highest score to the lowest score;
extracting a subset of the highest scores;
outputting the items corresponding to the extracted highest scores.

### Claim 45

A computer system for producing a recommendation of a plurality of items to a plurality of users, the computer system including:
means to obtain:
at least a subset of the multi-dimensional tensor elements representing scores given to a subset of items by a subset of users;
a provided plurality of geometric domains corresponding to a subset of the dimensions of said multi-dimensional tensor;
means to compute:
multi-dimensional tensor features by applying at least a multi-domain intrinsic convolutional layer on the multi-dimensional tensor elements;
a full set of multi-dimensional tensor elements from the multi-dimensional tensor features;
a recommendation of said plurality of items to said plurality of users using said full set of multi-dimensional tensor elements; and
means to provide in output said recommendation of said plurality of items to said plurality of users.

### Claim 46 (depends on 45)

The system of claim 45, where each of the geometric domains is one of the following types:
a manifold;
a parametric surface;
an implicit surface;
a mesh;
a point cloud;
an undirected weighted or unweighted graph;
a directed weighted or unweighted graph;
and the geometric domains are all of the same type or of different types.

### Claim 47 (depends on 45)

The system of claim 45, where the multi-dimensional tensor is a two-dimensional matrix.

### Claim 48 (depends on 45)

The system of claim 45, where said means to obtain the geometric domains comprises means to take in input the geometric domains.

### Claim 49 (depends on 45)

The system of claim 45, where said means to obtain at least a subset of the multi-dimensional tensor elements comprises means to take in input only a subset of the multi-dimensional tensor elements, said means to obtain the geometric domains are configured to compute the geometric domains from said subset of the multi-dimensional tensor elements.

### Claim 50 (depends on 45)

The system of claim 45, where said means to compute the multi-dimensional tensor elements from the multi-dimensional tensor features are configured to apply a neural network comprising at least a linear layer on the multi-dimensional tensor features.

### Claim 51 (depends on 45)

The system of claim 45, where said means to compute the multi-dimensional tensor elements from the multi-dimensional tensor features are further configured to:
input the multi-dimensional tensor features;
compute a sequence of incremental updates of intermediate multi-dimensional tensor elements; and
use said sequence of incremental updates of intermediate multi-dimensional tensor elements for computing said multi-dimensional tensor elements.

### Claim 52 (depends on 51)

The system of claim 51, where said means to compute are further configured to apply at least one iteration of a recurrent process to the input multi-dimensional tensor features for computing said sequence of incremental updates of the multi-dimensional tensor.

### Claim 53 (depends on 52)

The system of claim 52, where said means to compute are configured to implement said recurrent process as:
a recurrent neural network;
a long-short term memory network.

### Claim 54 (depends on 51)

The system of claim 51, where said means to compute the multi-dimensional tensor elements are configured to sum the sequence of incremental updates of the multi-dimensional tensor elements.

### Claim 55 (depends on 45)

The system of claim 45, where the said means to obtain the multi-dimensional tensor are configured to receive the multi-dimensional tensor as a product of a plurality of factors.

### Claim 56 (depends on 55)

The system of claim 55, where the multi-dimensional tensor is a two-dimensional matrix given as a product of two factors.

### Claim 57 (depends on 55)

The system of claim 55, where the multi-dimensional tensor features comprise the factor features of each of the factors, and where the means to compute the multi-dimensional tensor features are configured to apply at least an intrinsic convolutional layer to each of the factors to compute the factor features.

### Claim 58 (depends on 55)

The system of claim 55, where the means to compute the multi-dimensional tensor elements are further configured to compute:
the elements of each of the factors, and
the product of said factors.

### Claim 59 (depends on 55)

The system of claim 55, where tensor features comprise the factor features of each of the factors, and where the means to compute the multi-dimensional tensor features are configured to apply at least a multi-domain intrinsic convolutional layer to at least one of the factors to compute the factor features.

### Claim 60 (depends on 55)

The system of claim 55, where each of the factors has a corresponding geometric domain.

### Claim 61 (depends on 55)

The system of claim 55, where only a subset of the factors has corresponding geometric domains, and the remaining factors have no corresponding geometric domains.

### Claim 62 (depends on 61)

The system of claim 61, where said means to compute the factor elements are configured to compute factor elements of the factors having corresponding geometric domains by applying at least an intrinsic convolutional layer to each of the said factors to compute the factor features.

### Claim 63 (depends on 57)

The system of claim 57, where said means to compute the multi-dimensional tensor elements are further configured to:
for each of the multi-dimensional tensor factors
inputting the factor features;
computing a sequence of incremental updates of intermediate factor elements;
computing said factor elements using said sequence of incremental updates of intermediate factor elements;
computing the multi-dimensional tensor elements by using the factor elements of all the factors.

### Claim 64 (depends on 63)

The system of claim 63, where the said means to compute the sequence of incremental updates of intermediate factor elements are configured to apply at least one iteration of a recurrent process to the factor features.

### Claim 65 (depends on 64)

The system of claim 64, where said means to compute the sequence of incremental updates of intermediate factor elements are configured to implement the recurrent process as one of the following:
a recurrent neural network;
a long-short term memory network.

### Claim 66 (depends on 63)

The system of claim 63, where said means to compute the factor elements are configured to sum the sequence of incremental updates of the intermediate factor elements.

### Claim 67 (depends on 45)

The system according to claim 45, wherein said means to compute said multi-dimensional tensor features are further configured to apply at least one of the following layers:
a linear layer, including outputting a weighted linear combination of input data;
a non-linear layer, including applying a non-linear function to input data;
a spatial pooling layer, by:
determining a subset of points on the geometric domain;
for each point of said subset, determining the neighbours on the geometric domain; and
computing an aggregation operation on input data over the neighbours for all the points of said subset;
a fully connected layer, including outputting a weighted linear combination of input data at all the points of the geometric domain;
a regularization layer,
wherein each layer has input data and output data and said means to computer are configured to give output data of one layer as input data to another layer.

### Claim 68 (depends on 67)

The system of claim 67, wherein said means to compute said multi-dimensional tensor features are configured to apply two or more of said layers in sequence, and to give the output data of one layer in the sequence as input data to a subsequent layer in the sequence.

### Claim 69 (depends on 67)

The system of claim 67, wherein the aggregation operation in a spatial pooling layer comprises one of the following:
maximum computation;
average computation;
weighted average computation;
average of squares computation.

### Claim 70 (depends on 45)

The system of claim 45, where the multi-domain intrinsic convolutional layer is one of the following:
a spectral multi-domain intrinsic convolutional layer;
a spectrum-free multi-domain intrinsic convolutional layer;
a spatial multi-domain intrinsic convolutional layer.

### Claim 71 (depends on 57)

The system of claim 57, wherein the intrinsic convolutional layer is one of the following:
spectral convolutional layer
spectrum-free convolutional layer
spatial convolutional layer.

### Claim 72 (depends on 67)

The system of claim 67, where the regularization layer comprises at least one of the following:
a drop out of an arbitrary percentage of layer variables;
a quadratic penalty of the variables

### Claim 73 (depends on 45)

The system of claim 45, where said means to obtain a plurality of geometric domains are configured to take in input a geometric domains for each of the dimensions of the multi-dimensional tensor.

### Claim 74 (depends on 45)

The system of claim 45, where said means to obtain a plurality of geometric domains are further configured to:
inputting geometric domains corresponding to a provided subset of the dimensions of the multi-dimensional tensor;
computing the geometric domains corresponding to the non-provided subset of the dimensions of the multi-dimensional tensor from the subset of multi-dimensional tensor elements.

### Claim 75 (depends on 74)

The system of claim 74, where said means to obtain a plurality of geometric domains are further configured, for each of the non-provided dimensions, to:
extracting multi-dimensional tensor elements along said dimension and representing them as vectors;
computing a metric between each pair of said vectors;
building a graph, whose edges are weighted according to the said metric.

### Claim 76 (depends on 74)

The system of claim 74, where said means to obtain a plurality of geometric domains are further configured, for each of the non-provided dimensions to:
collect multi-dimensional features representing the general behavior of entries of said dimension and represent them as vectors;
computing a metric between each pair of said vectors;
building a graph, whose edges are weighted according to the said metric.

### Claim 77 (depends on 67)

The system of claim 67, wherein said means to compute said multi-dimensional tensor features are configured to apply more than one of said layers and wherein parameters of the applied layers comprise one or more of the following:
weights and biases of the linear layers;
parameters of the multi-domain intrinsic convolutional layers, comprising one or more of the following:
spectral multipliers of multi-domain filters;
parameters of the spectrum-free multi-domain filter expansion;
parameters of the weighting functions used to compute the patch operators in the spatial multi-domain intrinsic convolutional layer;

### Claim 78 (depends on 67)

The system of claim 67, wherein said means to compute said multi-dimensional tensor features are configured to apply more than one of said layers and wherein parameters of the applied layers comprise one or more of the following:
elements of the factors having no corresponding geometric domains;
weights and biases of the linear layers;
parameters of the intrinsic convolutional layers, comprising one or more of the following:
spectral multipliers of filters;
parameters of the spectrum-free filter expansion;
parameters of the weighting functions used to compute the patch operators in the spatial intrinsic convolutional layer.

### Claim 79 (depends on 77)

The system of claim 77, wherein said means to compute said multi-dimensional tensor features are configured determine the parameters of the applied layers by minimizing a cost function by means of an optimization procedure.

### Claim 80 (depends on 79)

The system of claim 79, where the optimization procedure is implemented to minimize one or more of the following:
the discrepancy between the input known multi-dimensional tensor elements and the corresponding subset of the computed multi-dimensional tensor elements;
a criterion of smoothness of multi-dimensional tensor elements;
a surrogate of the rank of the multi-dimensional tensor;
norms of the multi-dimensional tensor factors.

### Claim 81 (depends on 80)

The system of claim 80, where the criterion of smoothness is the Dirichlet norm on the respective on the respective geometric domains.

### Claim 82 (depends on 77)

The system of claim 77, wherein the parameters of the applied layers further comprise parameters of the geometric domains.

### Claim 83 (depends on 82)

The system of claim 82, wherein the parameters of the geometric domains comprise the metrics of said geometric domains.

### Claim 84 (depends on 82)

The system of claim 82, wherein the at least one of the geometric domains is a graph and the parameters of said geometric domain are the edge weights of the graph.

### Claim 85 (depends on 84)

The system of claim 84, wherein the vertices of the graph are points in a feature space, and said means to compute said multi-dimensional tensor features are configured to compute the edge weights by applying a parametric metric between pairs of points in said feature space, and the parameters of the geometric domain comprise the parameters of said parametric metric.

### Claim 86 (depends on 85)

The system of claim 85, wherein said means to compute said multi-dimensional tensor features are configured to determine parameters of the applied layers and parameters of the geometric domains by minimizing a cost function by means of an optimization procedure.

### Claim 87 (depends on 45)

The system of claim 45, where said means to compute the full set of multi-dimensional tensor elements is further configured to produce for each user a list of recommended items.

### Claim 88 (depends on 87)

The system of claim 87, where the list of recommended items comprises at least one of the following:
sorting the subset of the multi-dimensional tensor elements corresponding to a user from the highest score to the lowest score;
extracting a subset of the highest scores;
outputting the items corresponding to the extracted highest scores.
