Asynchronous document ingestion and enrichment system
AssigneeNoblis, Inc.
InventorsNathan DELLINGER, David PETERS
Life of this patent
Estimating the term…
Abstract
Provided are asynchronous data ingestion and enrichment systems and methods. The systems comprise a plurality of components (e.g., ingestion components, enrichment components, and/or publishing components). Instead of passing data from one component to another, the data is sent to a messaging queue that formats and hold the data until the subsequent component is ready to receive it. Additionally, each component comprises a central microservice and a plurality of instances, the central microservice configured to communicate with each instance of the plurality of instances.
Claims21
An asynchronous data ingestion and enrichment system comprising:
- a plurality of ingestion components, wherein each ingestion component of the plurality of ingestion components comprises an ingestion central microservice configured to manage a plurality of ingestion instances;
- a plurality of enrichment components, wherein each enrichment component of the plurality of enrichment components comprises an enrichment central micro service configured to manage a plurality of enrichment instances;
- a plurality of publishing components, wherein each publishing component of the plurality of publishing components comprises a publishing central microservice configured to manage a plurality of publishing instances;
- and
- an administrative service configured to:
- instruct each of the ingestion components to publish to one or more of a plurality of messaging queues,
- instruct each of the enrichment components to consume from one or more of the plurality of messaging queues and publish to one or more of the plurality of messaging queues, and
- instruct each of the publishing components to consume from one or more of the plurality of messaging queues.
A method for asynchronous data ingestion and enrichment system, the method performed at a system comprising:
- a plurality of ingestion components, wherein each ingestion component of the plurality of ingestion components comprises an ingestion central microservice configured to manage a plurality of ingestion instances;
- a plurality of enrichment components, wherein each enrichment component of the plurality of enrichment components comprises an enrichment central micro service configured to manage a plurality of enrichment instances;
- a plurality of publishing components, wherein each publishing component of the plurality of publishing components comprises a publishing central microservice configured to manage a plurality of publishing instances;
- and
- an administrative service;
- the method comprising:
- instructing, by the administrative service, each of the ingestion components to publish to one or more of a plurality of messaging queues,
- instructing, by the administrative service, each of the enrichment components to consume from one or more of the plurality of messaging queues and publish to one or more of the plurality of messaging queues, and
- instructing, by the administrative service, each of the publishing components to consume from one or more of the plurality of messaging queues.
A non-transitory computer-readable storage medium for asynchronous data ingestion and enrichment system, the non-transitory computer-readable storage medium storing instructions configured to be executed by one or more processors of a system comprising:
- a plurality of ingestion components, wherein each ingestion component of the plurality of ingestion components comprises an ingestion central microservice configured to manage a plurality of ingestion instances;
- a plurality of enrichment components, wherein each enrichment component of the plurality of enrichment components comprises an enrichment central micro service configured to manage a plurality of enrichment instances;
- a plurality of publishing components, wherein each publishing component of the plurality of publishing components comprises a publishing central microservice configured to manage a plurality of publishing instances;
- and
- an administrative service;
- the instructions causing the system to:
- instruct, by the administrative service, each of the ingestion components to publish to one or more of a plurality of messaging queues,
- instruct, by the administrative service, each of the enrichment components to consume from one or more of the plurality of messaging queues and publish to one or more of the plurality of messaging queues, and
- instruct, by the administrative service, each of the publishing components to consume from one or more of the plurality of messaging queues.
Dependencies are inferred from the claim text.