Clustering of structured and semi-structured data

A computerized system adapted for clustering data-subsets of one or more structured or semi-structured data sources, each data-subset comprising data-values, the system comprising at least one processing circuitry configured to:

• generate, for each data-subset in a group of data-subsets extracted from the one or more structured or semi-structured data sources, a respective vector indicative of characters' position distribution in data-values in the data-subset, thereby giving rise to a group of vectors;
• calculate, for each vector in the group of vectors, a respective proxy hash value;
• apply a first bitwise comparison and assigning different proxy hash values to respective blocks according to observed bitwise similarity between the proxy hash values;
• for each block, perform data-pattern matching of respective data-subsets of proxy hash values in the block, comprising:
• for at least one pair of proxy hash values in the block, apply at least one vector similarity test on respective vectors of the pair of proxy hash values;
• and in case the result of the at least one vector similarity test is greater than a first predefined threshold value, assign the respective data-subsets of the pair of proxy hash values to a common cluster, wherein the data-values in data-subsets assigned to the common cluster share similar data-patterns.

A non-transitory computer readable storage medium tangibly embodying a program of instructions that, when executed by a computer, causing the computer to perform a computerized method of clustering data-subsets (e.g. columns) of one or more structured or semi-structured data sources (e.g. tables), each data-subset comprising data-values, the method comprising using a processing circuitry for:

• generating, for each data-subset in a group of data-subsets extracted from the one or more structured or semi-structured data sources, a respective vector indicative of characters' position distribution in data-values in the data-subset, thereby giving rise to a group of vectors;
• calculating, for each vector in the group of vectors, a respective proxy hash value;
• applying a first bitwise comparison and assigning different proxy hash values to respective blocks according to observed bitwise similarity between the proxy hash values;
• for each block, performing data-pattern matching of respective data-subsets of proxy hash values in the block, comprising:
• for at least one pair of proxy hash values in the block, applying at least one vector similarity test on respective vectors of the pair of proxy hash values;
• and in case the result of the at least one vector similarity test is greater than a predefined threshold value, assigning the respective data-subsets of the pair of proxy hash values to a common cluster, wherein the data-values in data-subsets assigned to the common cluster share similar data-patterns.