# US 10,320,752 B2 · Gradients over distributed datasets

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,320,752 B2
- Document ID: US10320752B2
- Status: Granted
- Kind: B2
- Application number: 15/521,409
- Filed: 2015-10-26
- Published: 2017-10-26
- Granted: 2019-06-11
- Earliest claimed: 2014-10-24
- Assignees: National ICT Australia Limited
- Applicants: National ICT Australia Limited
- Inventors: Stephen Hardy; Felix Lawrence; Daniel Visentin
- Web page: https://patentgrep.com/patent/US10320752B2

## CPC

- G16B 50/40: Encryption of genetic data
- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- G06Q 10/40: Business processes related to social networking or social networking services
- H04L 9/008: {involving homomorphic encryption}
- H04L 9/06: the encryption apparatus using shift registers or memories for block-wise {or stream} coding, e.g. DES systems {or RC4; Hash functions; Pseudorandom sequence generators}
- H04L 63/0414: {during transmission, i.e. party's identity is protected against eavesdropping, e.g. by using temporary identifiers, but is known to the other party or parties involved in the communication}
- G16B 40/00: ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding
- G16B 50/00: ICT programming tools or database systems specially adapted for bioinformatics
- G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

This disclosure relates to characterising data sets that are distributed as multiple data subsets over multiple computers such as by determining a gradient of an objective function. A computer determines a partial gradient of the objective function over a data subset stored on the computer and determines random data. The computer then determines an altered gradient by modifying the partial gradient based on the random data and encrypts the altered gradient such that one or more operations on the altered gradient can be performed based on the encrypted gradient and sends the encrypted gradient. Since the partial gradient is altered based on random data and encrypted it is difficult for another computer to calculate the data that is stored on the first computer. This is an advantage as it allows to preserve the privacy of the data stored on the first computer while still allowing to characterise the data set.

## Claims

### Claim 1

A method for determining a gradient of an objective function in a machine learning process to iteratively characterise a data set by a data model including model parameters, the dataset is distributed as multiple data subsets over multiple computer systems, the method comprising:
determining by a first computer system a partial gradient of the objective function over a first data subset stored on the first computer system, the objective function characterises an error between the data model and the first data subset, the partial gradient points towards a reduction in the error between the data model and the first data subset;
determining by the first computer system random data;
determining by the first computer system an altered gradient by modifying the partial gradient based on the random data;
encrypting by the first computer system the altered gradient to determine a first encrypted gradient such that one or more operations on the altered gradient can be performed based on the first encrypted gradient;
determining by the first computer system an output gradient based on the first encrypted gradient;
sending by the first computer system the output gradient to a receiving computer system.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving by the first computer system a second encrypted gradient of the objective function over a second data subset, the second data subset being stored on one or more second computer systems different to the first computer system;
wherein determining the output gradient comprises performing the one or more operations to combine the first encrypted gradient with the second encrypted gradient.

### Claim 3 (depends on 2)

The method of claim 2, wherein performing the one or more operations to combine the first encrypted gradient with the second encrypted gradient comprises adding the first encrypted gradient to the second encrypted gradient.

### Claim 4 (depends on 1)

The method of claim 1, wherein
determining random data comprises determining a random number; and
determining the altered gradient comprises multiplying the random number with the partial gradient or adding the random number to the partial gradient.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
encrypting the random data to determine first encrypted random data;
determining output random data based on the first encrypted random data; and
sending the output random data to the receiving computer system.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
receiving by the first computer system second encrypted random data;
wherein determining the output random data comprises performing the one or more operations to combine the first encrypted random data with the second random data.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the partial gradient is based on a regression model.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first data subset comprises training data for training a classifier and the training data comprises one or more samples and a label for each of the one or more samples.

### Claim 9 (depends on 8)

The method of claim 8, wherein the one or more samples comprise DNA related data.

### Claim 10 (depends on 1)

The method of claim 1, wherein determining the partial gradient comprises determining the partial gradient to extract principle components of the data set.

### Claim 11 (depends on 10)

The method of claim 10, wherein the first data subset comprises multiple images.

### Claim 12 (depends on 1)

The method of claim 1, wherein the first data subset comprises training data of a recommender system and determining the partial gradient comprises determining the partial gradient of the recommender system.

### Claim 13 (depends on 1)

The method of claim 1, wherein the data set comprises data from which an anomaly or outlier is to be detected and determining the partial gradient comprises determining the partial gradient of an anomaly or outlier detection system.

### Claim 14 (depends on 1)

The method of claim 1, wherein the first data subset or the second data subset or both consist of a single data record.

### Claim 15 (depends on 1)

The method of claim 1, wherein encrypting the altered gradient comprises using Paillier encryption.

### Claim 16

A non-transitory computer readable medium comprising computer-executable instructions stored thereon, that when executed by a processor, causes the processor to perform a method of determining a gradient of an objective function in a machine learning process to iteratively characterise a data set by a data model including model parameters, the dataset is distributed as multiple data subsets over multiple computer systems, the method comprising:
determining by a first computer system a partial gradient of the objective function over a first data subset stored on the first computer system, the objective function characterises an error between the data model and the first data subset, the partial gradient points towards a reduction in the error between the data model and the first data subset;
determining by the first computer system random data;
determining by the first computer system an altered gradient by modifying the partial gradient based on the random data;
encrypting by the first computer system the altered gradient to determine a first encrypted gradient such that one or more operations on the altered gradient can be performed based on the first encrypted gradient;
determining by the first computer system an output gradient based on the first encrypted gradient;
sending by the first computer system the output gradient to a receiving computer system.

### Claim 17

A computer system for determining a gradient of an objective function in a machine learning process to iteratively characterise a data set by a data model including model parameters, the dataset is distributed as multiple data subsets over multiple computer systems, the computer system comprising:
a datastore to store a first data subset;
a processor to
determine a partial gradient of the objective function over the first data subset, the objective function characterises an error between the data model and the first data subset, the partial gradient points towards a reduction in the error between the data model and the first data subset,
determine random data,
determine an altered gradient by modifying the partial gradient based on the random data,
encrypt the altered gradient to determine a first encrypted gradient such that one or more operations on the altered gradient can be performed based on the first encrypted gradient, and
determine an output gradient based on the first encrypted gradient; and
an output port to send the output gradient to a receiving computer system.