# US 2022/0068478 A1 · Scalable, accurate and reliable measure of variable dependence and independence, and utilization of the measure to train a neural network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0068478 A1
- Document ID: US20220068478A1
- Status: Published application
- Kind: A1
- Application number: 17/161,728
- Filed: 2021-01-29
- Published: 2022-03-03
- Earliest claimed: 2020-09-01
- Applicants: NEC Laboratories Europe GmbH
- Inventors: Shujian Yu; Francesco Alesiani
- Web page: https://patentgrep.com/patent/US20220068478A1

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G16B 5/20: Probabilistic models
- G16B 25/10: Gene or protein expression profiling; Expression-ratio estimation or normalisation
- G16B 40/20: Supervised data analysis
- G16H 20/00: ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY

## Abstract

A method for measuring dependence and independence of variables includes collecting data for the variables, computing sample Gram matrices for the variables, and computing element-wise products of the sample Gram matrices. Normalized mutual information of the variables is then computed using Renyi's α-order entropy function based on the element-wise products as an independence measure of the variables.

## Claims

### Claim 1

A method for measuring dependence and independence of variables, the method comprising:
collecting data for the variables;
computing sample Gram matrices for the variables;
computing element-wise products of the sample Gram matrices; and
computing normalized mutual information of the variables using Renyi's α-order entropy function based on the element-wise products as an independence measure of the variables.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the independence measure is automatically differentiable.

### Claim 3 (depends on 2)

The method according to claim 2, further comprising utilizing the independence measure as a loss function for training a neural network.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the variables are gene expressions, the method further comprising utilizing the independence measure to build a gene regulatory network (GRN) in which nodes represent levels of the gene expressions and edges represent interactions between genes.

### Claim 5 (depends on 4)

The method according to claim 4, further comprising utilizing the GRN to design a compound, classify biological samples, design a treatment or to identify a most informative set of genes for a particular disease.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the variables are signals, the method further comprising utilizing the independence measure for blind source separation to distinguish different sources of the signals.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the blind source separation is used to distinguish between and listen to multiple speakers, or to measure propagation delay of seismic waves for petroleum exploration.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the independence measure between variable is between [0,1] with 0 indicating complete independence and 1 indicating direct functional dependence.

### Claim 9 (depends on 1)

The method according to claim 1, wherein a normalization factor used to obtain the normalized mutual information is based on an upper bound of the mutual information using Shearer's inequality.

### Claim 10

A system for measuring dependence and independence of variables comprising one or more processors which, alone or in combination, are configured to provide for execution of the following steps:
collecting data for the variables;
computing sample Gram matrices for the variables;
computing element-wise products of the sample Gram matrices; and
computing normalized mutual information of the variables using Renyi's α-order entropy function based on the element-wise products as an independence measure of the variables.

### Claim 11 (depends on 10)

The system according to claim 10, wherein the independence measure is automatically differentiable, the system being further configured to utilize the independence measure as a loss function for training a neural network.

### Claim 12 (depends on 10)

The system according to claim 10, wherein the variables are gene expressions, the system being further configured to utilize the independence measure to build a gene regulatory network (GRN) in which nodes represent levels of the gene expressions and edges represent interactions between genes, the GRN being useable to design a compound, classify biological samples, design a treatment or to identify a most informative set of genes for a particular disease.

### Claim 13 (depends on 10)

The system according to claim 10, wherein the variables are signals, the system being further configured to utilize the independence measure for blind source separation to distinguish different sources of the source signals in order to distinguish between and listen to multiple speakers, or to measure propagation delay of seismic waves for petroleum exploration.

### Claim 14 (depends on 10)

The system according to claim 10, wherein the independence measure between variables is between [0,1] with 0 indicating complete independence and 1 indicating direct functional dependence.

### Claim 15

A tangible, non-transitory computer-readable medium having instructions thereon which, upon being executed by one or more processors, alone or in combination, provide for execution of the following steps:
collecting data for the variables;
computing sample Gram matrices for the variables;
computing element-wise products of the sample Gram matrices; and
computing normalized mutual information of the variables using Renyi's α-order entropy function based on the element-wise products as an independence measure of the variables.
