Discovering contexts and structured data structures from hazy data

A method for storing data of an unknown data context and structure in a structured storage system, the method comprising:

• extracting, by a hardware processor and in real time, a list of key-value pairs from one or more inbound data blocks from one or more external data sources, wherein each key-value pair in the list of key-value pairs comprises a key that is a constant that defines a data set in an external data source of the external data sources and a value that is a variable that belongs to the data set;
• calculating, by the hardware processor, a similarity score for each key-value pair in the list of key-value pairs that is extracted from the one or more inbound data blocks based on one or more current existing data structure;
• determining, by the hardware processor, a highest similarity score from among the similarity score that is calculated for each key-value pair in the list of key-value pairs;
• determining, by the hardware processor, that the highest similarity score for the list does meet a predetermined similarity threshold;
• integrating, by the hardware processor, the list into a first current existing data structure of the one or more current existing data structures;
• and
• storing, by the hardware processor, the list of key-value pairs in the first current existing data structure in the structured storage system.

A method for storing data of an unknown data context and structure in a structured storage system, the method comprising:

• extracting, by a hardware processor and in real time, a list of key-value pairs from one or more inbound data blocks from one or more external data sources, wherein each key-value pair in the list of key-value pairs comprises a key that is a constant that defines a data set in an external data source of the external data sources and a value that is a variable that belongs to the data set;
• determining, by the hardware processor, that a data structure does not exist;
• creating, by the hardware processor, a new data structure to store the list of key-value pairs for a first inbound data block from the one or more inbound data blocks;
• and
• storing, by the hardware processor, the list of key-value pairs in the new data structure in the structured storage system as a table with each key of the key-value pair denoting a different column in the table.