Systems and methods for processing hierarchical, semi-structured, schema-less, polymorphic data

A computer implemented method for transforming source data and generating a fully specified data format, the method comprising:

• accessing, by at least one processor, specification of an existing source data repository, stored as at least part of a database in the memory of a computer system, the existing source data repository including polymorphic data;
• recursively scanning, by at least one processor, the entire existing source data repository to discover a full schema definition of all attributes within the existing source data repository, including across all source data attributes having polymorphic attributes, the recursive scanning including discovery of metadata for each attribute and each data type associated with the attribute recorded for a particular polymorphic attribute;
• and
• generating, by at least one processor automatically, transformation code subsequent to discovery, configured to transform the existing source data repository into a relational target format specified via the system, the transformation code based on the full schema definition of all the attributes within the existing source data repository, the transformation code when deployed defining a mapping for all instances and data types of all attributes in the existing source data repository and configured to output rows of data in one or more relational data objects.

A non-transitory computer readable medium comprising instructions that when executed cause at least one processor to perform a method for transforming source data and generating a fully specified data format, the method comprising:

• accessing specification of an existing source data repository, stored as at least part of a database in the memory of a computer system, the existing source data repository including polymorphic data;
• recursively scanning the entire existing source data repository to discover a full schema definition of all attributes within the existing source data repository, including across all source data attributes having polymorphic attributes, the recursive scanning including discovery of metadata for each attribute and each data type associated with the attribute recorded for a particular polymorphic attribute;
• and
• generating transformation code subsequent to discovery, configured to transform the existing source data repository into a relational target format specified via the system, the transformation code based on the full schema definition of all the attributes within the existing source data repository, the transformation code when deployed defining a mapping for all instances and data types of all attributes in the existing source data repository and configured to output rows of data in one or more relational data objects.