Cross-organization data instance matching

A system for two-stage data instance processing, the system comprising:

• one or more processors;
• a memory coupled to at least one of the one or more processors;
• a network interface coupling the one or more processors to processors of a plurality of organizations over a network;
• a set of computer program instructions stored in the memory and executed by at least one of the one or more processors in order to perform actions of:
• obtaining a set of data instances collected over the network from the plurality of organizations, wherein each of the data instances includes at least one record formed in an organization of the plurality of organizations that stores values of a plurality of attributes of the data instance;
• preliminarily dividing, in stage one, conflict detection using the one or more processors, the set of data instances into groups within the memory, wherein data instances with conflicting values for the same attribute are divided into different groups and stored on a remote server in a distributed cloud computing environment;
• subdividing, in stage two, data instances in each of the groups into clusters to increase precision of the dividing, the subdividing comprising constructing, by one or more processors, for each of the data instances in each of the groups, a feature vector based on the at least one record of the data instance for further processing, with the values of the plurality of attributes being transformed into binary values for increased efficiency of the processing and minimization of bandwidth and memory requirements;
• and
• creating a new electronic record having the transformed binary values of the plurality of attributes of the data instances in the cluster by combining the data instances in each cluster, the electronic record being updated on a local machine and stored on the remote server in the distributed cloud computing environment to minimize network bandwidth and local machine memory requirements for efficient iterative accessing and maintaining of the record.

A computer program product for data instance processing, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the program instructions being executable by one or more processor devices to cause the device to perform a method comprising:

• obtaining a set of data instances collected from a plurality of organizations over a network, wherein each of the data instances includes at least one record formed in an organization of the plurality of organizations that stores values of a plurality of attributes of the data instance;
• preliminarily dividing, in stage one, by conflict detection using at least one of the one or more processors, the set of data instances into groups, wherein data instances with conflicting values for the same attribute are divided into different groups and stored on a remote server in a distributed cloud computing environment;
• subdividing, in stage two, data instances in each of the groups into clusters to increase precision of the dividing, the subdividing comprising constructing, by one or more processors, for each of the data instances in each of the groups, a feature vector based on the at least one record of the data instance for further processing, with the values of the plurality of attributes being transformed into binary values for increased efficiency of the processing and minimization of bandwidth and memory requirements;
• and
• creating a new electronic record comprising the transformed binary values of the plurality of attributes of the data instances in the cluster by combining the data instances in each cluster, the electronic record being updated on a local machine and stored on the remote server in the distributed cloud computing environment to minimize network bandwidth and local machine memory requirements for efficient iterative accessing and maintaining of the record.