# US 2019/0377547 A1 · Data grouping for efficient parallel processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0377547 A1 - Document ID: US20190377547A1 - Status: Published application - Kind: A1 - Application number: 16/001,165 - Filed: 2018-06-06 - Published: 2019-12-12 - Earliest claimed: 2018-06-06 - Assignees: Sap SE - Applicants: Sap SE - Inventors: Markus Kahn - Web page: https://patentgrep.com/patent/US20190377547A1 ## CPC - G06F 7/08: Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry (by merging two or more sets of carriers in ordered sequence G06F7/16) - G06F 16/2255: {Hash tables} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An improved process for distributing data objects and a process for reducing skew in groups of data objects to be processed in parallel are provided herein. A request for parallel processing of a plurality of data objects is received. One or more groups for distributing the data objects are generated. Hash value intervals for the one or more groups are determined. Hash values for the plurality of data objects are determined. The plurality of data objects are distributed into the one or more groups based on their respective hash values and the hash value intervals. The plurality of data objects are processed in parallel by the groups comprising the distributed data objects. The processing results of the plurality of data objects are provided in response to the request. ## Claims ### Claim 1 A system for improved data distribution for parallel processing, the system comprising: one or more memories; one or more processing units coupled to the one or more memories; and one or more computer readable storage media storing instructions that, when loaded into the one or more memories, cause the one or more processing units to perform data distribution operations for: receiving a plurality of data objects; calculating a total number of data objects; determining a total number of groups into which to distribute the data objects, based at least in part on the total number of data objects; calculating group hash values in parallel for the respective groups; sorting the group hash values; assigning the group hash values to the groups in sorted order; determining hash value intervals for the respective groups, based in part on their respective hash values and the hash value of the group immediately preceding; calculating data object hash values in parallel for the respective data objects in the plurality of data objects; distributing the plurality of data objects in parallel to the groups based on their respective data object hash values and the hash value intervals; and, providing the groups comprising the distributed plurality of data objects for processing the data objects in parallel by group. ### Claim 2 (depends on 1) The system of claim 1, wherein the data objects are records from a database. ### Claim 3 (depends on 1) The system of claim 1, wherein a given hash value interval for a given group ranges from a first hash value after the group hash value of the preceding group to the group hash value of the given group. ### Claim 4 (depends on 1) The system of claim 1, wherein distributing the plurality of data objects comprises storing a reference to a given data object in its given applicable group. ### Claim 5 (depends on 1) The system of claim 1, wherein distributing the plurality of data objects comprises comparing the hash value of a given data object to minimum hash values and maximum hash values of the hash value intervals. ### Claim 6 (depends on 1) The system of claim 1, wherein the total number of groups is calculated based at least in part on available computing resources. ### Claim 7 One or more non-transitory computer-readable storage media storing computer-executable instructions for causing a computing system to perform an improved method for distributing data objects, the method comprising: receiving a request for parallel processing of a plurality of data objects; generating one or more groups for distributing the data objects; determining hash value intervals for the one or more groups; determining hash values for the plurality of data objects; distributing the plurality of data objects into the one or more groups based on their respective hash values and the hash value intervals; processing the plurality of data objects in parallel by the groups comprising the distributed data objects; and, returning the processing results of the plurality of data objects in response to the request. ### Claim 8 (depends on 7) The one or more non-transitory computer-readable storage media of claim 7, wherein the data objects are records from a database. ### Claim 9 (depends on 7) The one or more non-transitory computer-readable storage media of claim 7, wherein a given hash value interval for a given group ranges from a first hash value after the group hash value of the preceding group to the group hash value of the given group. ### Claim 10 (depends on 7) The one or more non-transitory computer-readable storage media of claim 7, wherein distributing the plurality of data objects comprises storing a reference to a given data object in its given applicable group. ### Claim 11 (depends on 7) The one or more non-transitory computer-readable storage media of claim 7, wherein distributing the plurality of data objects comprises comparing the hash value of a given data object to minimum hash values and maximum hash values of the hash value intervals. ### Claim 12 (depends on 7) The one or more non-transitory computer-readable storage media of claim 7, wherein the total number of groups is calculated based at least in part on available computing resources. ### Claim 13 (depends on 7) The one or more non-transitory computer-readable storage media of claim 7, wherein the plurality of data objects is divided into data sets, and the data distribution method is performed separately and in parallel for each set. ### Claim 14 A method for reducing skew in groups of data objects to be processed in parallel, the method comprising: receiving a plurality of data objects; determining a total number of groups for distributing the data objects; generating value intervals for the groups using a discretized value generation function; for data objects of the plurality of data objects: assigning a value generated using the discretized value generation function to a given data object of the plurality of data objects, and distributing the given data object into a given group of the groups based on the value assigned to the given data object and the generated value interval for the given group; and, providing the groups having the distributed data objects. ### Claim 15 (depends on 14) The method of claim 14, wherein the data objects are records from a database. ### Claim 16 (depends on 14) The method of claim 14, wherein a given generated value interval for a given group ranges from a first value after the group generated value of the preceding group to the group generated value of the given group. ### Claim 17 (depends on 14) The method of claim 14, wherein distributing the plurality of data objects comprises storing a reference to a given data object in its given applicable group. ### Claim 18 (depends on 14) The method of claim 14, wherein distributing the plurality of data objects comprises comparing the generated value of a given data object to minimum values and maximum values of the generated value intervals. ### Claim 19 (depends on 14) The method of claim 14, wherein the total number of groups is calculated based at least in part on available computing resources. ### Claim 20 (depends on 14) The method of claim 14, wherein the plurality of data objects is divided into data sets, and the data distribution method is performed separately and in parallel for each set.