Gradients over distributed datasets

A non-transitory computer readable medium comprising computer-executable instructions stored thereon, that when executed by a processor, causes the processor to perform a method of determining a gradient of an objective function in a machine learning process to iteratively characterise a data set by a data model including model parameters, the dataset is distributed as multiple data subsets over multiple computer systems, the method comprising:

• determining by a first computer system a partial gradient of the objective function over a first data subset stored on the first computer system, the objective function characterises an error between the data model and the first data subset, the partial gradient points towards a reduction in the error between the data model and the first data subset;
• determining by the first computer system random data;
• determining by the first computer system an altered gradient by modifying the partial gradient based on the random data;
• encrypting by the first computer system the altered gradient to determine a first encrypted gradient such that one or more operations on the altered gradient can be performed based on the first encrypted gradient;
• determining by the first computer system an output gradient based on the first encrypted gradient;
• sending by the first computer system the output gradient to a receiving computer system.

A computer system for determining a gradient of an objective function in a machine learning process to iteratively characterise a data set by a data model including model parameters, the dataset is distributed as multiple data subsets over multiple computer systems, the computer system comprising:

• a datastore to store a first data subset;
• a processor to
• determine a partial gradient of the objective function over the first data subset, the objective function characterises an error between the data model and the first data subset, the partial gradient points towards a reduction in the error between the data model and the first data subset,
• determine random data,
• determine an altered gradient by modifying the partial gradient based on the random data,
• encrypt the altered gradient to determine a first encrypted gradient such that one or more operations on the altered gradient can be performed based on the first encrypted gradient, and
• determine an output gradient based on the first encrypted gradient;
• and
• an output port to send the output gradient to a receiving computer system.