Accelerated large-scale similarity calculation

An electronic system comprising:

• one or more processing devices;
• and
• one or more non-transitory machine-readable storage devices storing instructions that are executable by the one or more processing devices to cause performance of operations comprising:
• receiving, at an entity correlation system, a query for data related to relationships among a set of entities;
• obtaining, by a first processing unit of the entity correlation system, data stored at a storage device, the data comprising information about multiple entities;
• generating, at the first processing unit, a plurality of data arrays using the data obtained from the data storage device, wherein each data array of the plurality of data arrays:
• i) comprises parameter values for a respective plurality of entities;
• and
• ii) is configured for processing at a respective computing cell of a second processing unit;
• providing, at the entity correlation system, at least two data arrays to the second processing unit, the second processing unit being configured to execute a correlation algorithm to concurrently process the at least two data arrays at respective computing cells of the second processing unit;
• computing, at the second processing unit, a correlation score for the at least two data arrays based on calculations performed at the respective computing cells using the correlation algorithm and the parameter values for the plurality of entities;
• determining, at the entity correlation system, relationships among entities of the at least two data arrays based on the computed correlation score, wherein the relationships indicate overlapping attributes that exist among at least a subset of the entities, and
• providing data indicating the determined relationships among the entities in response to receiving the query.