Domain data management services

A non-transitory computer-readable medium storing instructions that, when executed by at least one processor of a computing system, cause the computing system to perform operations, the operations comprising:

• registering a type of data file, wherein the registering the type of data file comprises:
• posting a file type and an identifier to a meta service, and
• storing metadata describing the type of data file, the metadata including a file storage service and a parser for the type of data file;
• receiving a first data file of the type of data file from a first domain, the first data file comprising raw data;
• storing the first data file using the file storage service;
• storing one or more access rules and a lineage of the first data file using a kernel storage service;
• parsing the first data file using the parser to generate a content from the raw data;
• storing the content using a content storage service, separately from the raw data;
• providing the first data file and the content to a search service for subsequent discovery by at least one of a file type, a file identifier, and a source type;
• automatically updating one or more second data files from one or more other domains based on the content of the first data file using the search service and the lineage;
• and
• enabling access to federated indices that provide reference to raw data files and parsed content stored by different domains, wherein:
• the parser is triggered in response to indexing the data file.

A computing system, comprising:

• one or more processors;
• and
• a memory system comprising one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations, the operations comprising:
• registering a type of data file, wherein the registering the type of data file comprises:
• posting a file type and an identifier to a meta service, and
• storing metadata describing the type of data file, the metadata including a file storage service and a parser for the type of data file;
• receiving a first data file of the type of data file from a first domain, the first data file comprising raw data;
• storing the first data file using the file storage service;
• storing one or more access rules and a lineage of the first data file using a kernel storage service;
• parsing the first data file using the parser to generate a content from the raw data;
• storing the content using a content storage service, separately from the raw data;
• providing the first data file and the content to a search service for subsequent discovery by at least one of a file type, a file identifier, and a source type;
• automatically updating one or more second data files from one or more other domains based on the content of the first data file using the search service and the lineage;
• and
• enabling access to federated indices that provide reference to raw data files and parsed content stored by different domains, wherein:
• the parser is triggered in response to indexing the data file.