# US 11,113,624 B2 · Distributed machine learning on heterogeneous data platforms > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,624 B2 - Document ID: US11113624B2 - Status: Granted - Kind: B2 - Application number: 15/648,121 - Filed: 2017-07-12 - Published: 2019-01-17 - Granted: 2021-09-07 - Earliest claimed: 2017-07-12 - Assignees: Sap SE - Applicants: Sap SE - Inventors: Chengyu Liu; Lian Yang; Xingtian Shi - Web page: https://patentgrep.com/patent/US11113624B2 ## CPC - G06N 20/00: Machine learning - G06F 9/46: Multiprogramming arrangements - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A distributed machine learning framework implemented with heterogeneous data platforms reduces data copying and exploits memory/computation resources of the different data platforms. A configuration component includes information to set up the system. A persistency component manages storage of data and a model trained by machine learning. A proxy repository includes predefined proxies for communication between heterogeneous data platform nodes and execution of the machine learning procedure. A machine learning execution component comprises three layers. A bottom work node layer within the data platform performs computations of the machine learning procedure. A middle server node layer comprising one server node per data platform, communicates with the work nodes to coordinate jobs on that data platform. An upper layer comprises a central server node communicating with server nodes and coordinating jobs of the different platforms. The system can extend to additional external data platforms and external machine learning libraries with predefined proxies. ## Claims ### Claim 1 A computer-implemented method comprising: sending to a first server node of a first data platform, a first request to collect a first partial result vector regarding a first parameter of a training set; receiving from the first server node, the first partial result vector computed by a first work node from data stored in the first data platform; sending to a second server node of a second data platform, a second request to collect a second partial result vector regarding a second parameter of the training set; receiving from the second server node, the second partial result vector computed by a second work node from data stored in the second data platform; aggregating the first partial result vector and the second partial result vector to create the result vector including an updated first parameter and an updated second parameter, wherein the first partial result vector and the second partial result vector are rows or columns in the result vector; sending back the result vector to the first server node and to the second server node for processing by the first server node and the second server node; receiving convergence data calculated by the first server node from the updated first parameter, and calculated by the second server node from the updated second parameter; a first in-memory database engine of a first in-memory database determining a convergence from the convergence data; and where the convergence is found, the first in-memory database engine storing in a persistence component of the first in-memory database, a model trained according to the result vector. ### Claim 2 (depends on 1) A method as in claim 1 further comprising: referencing training set partition information from a configuration component to determine a location of the first parameter in the first data platform. ### Claim 3 (depends on 1) A method as in claim 1 further comprising: referencing configuration information from a configuration component to determine a data format of the first data platform. ### Claim 4 (depends on 1) A method as in claim 1 further comprising: referencing a predetermined statement from a proxy repository to send the first request. ### Claim 5 (depends on 1) A method as in claim 1 wherein the first data platform comprises a second in-memory database. ### Claim 6 (depends on 5) A method as in claim 5 wherein the second data platform comprises other than an in-memory database. ### Claim 7 (depends on 1) A method as in claim 1 wherein the first partial result vector represents a gradient. ### Claim 8 (depends on 1) A method as in claim 1 wherein the first parameter and the second parameter comprise mutually disjoint sets. ### Claim 9 (depends on 1) A method as in claim 1 wherein the first parameter and the second parameter are based upon logistic regression. ### Claim 10 A non-transitory computer readable storage medium embodying a computer program for performing a method, said method comprising: referencing a proxy repository to obtain a first predetermined request statement; sending to a first server node of a first data platform, the first predetermined request statement to collect a first partial result vector regarding a first parameter of a training set; receiving from the first server node, the first partial result vector computed by a first work node from data stored in the first data platform; referencing the proxy repository to obtain a second predetermined request statement; sending to a second server node of a second data platform, the second predetermined request statement to collect a second partial result vector regarding a second parameter of the training set; receiving from the second server node, the second partial result vector computed by a second work node from data stored in the second data platform; aggregating the first partial result vector and the second partial result vector to create a result vector including an updated first parameter and an updated second parameter wherein the first partial result vector and the second partial result vector are rows or columns in the result vector; sending back the result vector to the first server node and to the second server node for processing by the first server node and the second server node; receiving convergence data calculated by the first server node from the updated first parameter, and calculated by the second server node from the updated second parameter; a first in-memory database engine of a first in-memory database determining convergence from the convergence data; and where the convergence is found, storing in a persistence component of the first in-memory database, a model trained according to the result vector. ### Claim 11 (depends on 10) A non-transitory computer readable storage medium as in claim 10 wherein the method further comprises: referencing training set partition information from a configuration component to determine a location of the first parameter in the first data platform. ### Claim 12 (depends on 10) A non-transitory computer readable storage medium as in claim 10 wherein the method further comprises: referencing configuration information from a configuration component to determine a data format of the first data platform. ### Claim 13 (depends on 10) A non-transitory computer readable storage medium as in claim 10 wherein the first partial result vector represents a gradient. ### Claim 14 (depends on 10) A non-transitory computer readable storage medium as in claim 10 wherein the first parameter and the second parameter comprise mutually disjoint sets. ### Claim 15 A computer system comprising: one or more processors; a software program, executable on said computer system, the software program configured to cause an engine of an in-memory database to: send to a first server node of a first data platform, a first request to collect a first partial result vector regarding a first parameter of a training set; receive from the first server node, the first partial result vector computed by a first work node from data stored in the first data platform; send to a second server node of a second data platform, a second request to collect a second partial result vector regarding a second parameter of the training set; receive from the second server node, the second partial result vector computed by a second work node from data stored in the second data platform; aggregate the first partial result vector and the second partial result vector to create a result vector including an updated first parameter and an updated second parameter, wherein the first partial result vector and the second partial result vector are rows or columns in the result vector; send back the result vector to the first server node and to the second server node for processing by the first server node and the second server node; receive convergence data calculated by the first server node from the updated first parameter, and calculated by the second server node from the updated second parameter; determine convergence from the convergence data; and where the convergence is found, store in the in memory database, a model trained according to the result vector. ### Claim 16 (depends on 15) A computer system as in claim 15 wherein the software program is further configured to cause the in-memory database engine to: reference training set partition information from a configuration component to determine a location of the first parameter in the first data platform. ### Claim 17 (depends on 16) A computer system as in claim 16 wherein the software program is further configured to cause the in-memory database engine to: reference configuration information from a configuration component to determine a data format of the first data platform. ### Claim 18 (depends on 16) A computer system as in claim 16 wherein the software program is further configured to cause the in-memory database engine to: reference a predetermined statement from a proxy repository to send the first request. ### Claim 19 (depends on 16) A computer system as in claim 16 wherein the first partial result vector represents a gradient.