Managing complex service dependencies in a data integration system
AssigneeBaidu USA LLC
InventorsZhuang Li, Yifei Jiang, Dong Li, Fan Zhu, Qi Kong, Guang Yang, Jingao Wang
Life of this patent
Estimating the term…
Abstract
Described is a data integration technique and system for providing data in response to a request. The data integration system may overcome the limits of a fixed scheduling system by implementing a service dependency graph (SDG). In particular, the data integration system may represent the SDG as a directed acyclic graph (DAG). Accordingly, the system may leverage the nature of DAGs in order to derive an efficient topological ordering for calling one or more services. Thus, the data integration system may retrieve and compile data in an efficient and scalable manner.
Claims21
A non-transitory machine-readable medium storing instructions which, when executed by one or more processors of a computing device, cause the computing device to perform operations for efficiently retrieving data, the operations comprising:
- in response to a request for data, determining that retrieving the data requires calling one or more services to retrieve secondary data associated with the data;
- determining the services required to retrieve the secondary data by analyzing dependencies between the services, wherein the dependencies are stored as a directed acyclic graph (DAG) representation, wherein the DAG includes the services as nodes of the DAG and a dependency relationship between two services as a directed edge, wherein the services include at least one of a process or operation for determining or retrieving the secondary data;
- determining a sequence for calling the services based on a topological ordering of the services within the DAG, wherein, in response to determining that the topological ordering of the services comprises a service having a plurality of dependent services without any remaining dependent services, at least two of the plurality of dependent services without any remaining dependent services are called in parallel;
- and
- retrieving the secondary data in the determined sequence and compiling the secondary data to provide the data requested.
A computer-implemented method of efficiently providing data, comprising:
- in response to a request for data, determining that retrieving the data requires calling one or more services to retrieve secondary data associated with the data;
- determining the services required to retrieve the secondary data by analyzing dependencies between the services, wherein the dependencies are stored as a directed acyclic graph (DAG) representation, wherein the DAG includes the services as nodes of the DAG and a dependency relationship between two services as a directed edge, wherein the services include at least one of a process or operation for determining or retrieving the secondary data;
- determining a sequence for calling the services based on a topological ordering of the services within the DAG, wherein, in response to determining that the topological ordering of the services comprises a service having a plurality of dependent services without any remaining dependent services, the at least two of the plurality of dependent services without any remaining dependent services are called in parallel;
- and
- retrieving the secondary data in the determined sequence and compiling the secondary data to provide the data requested.
A data processing system, comprising:
- a processor;
- and
- a memory coupled to the processor, the memory storing instructions, which when executed by the processor, cause the processor to perform operations for efficiently retrieving data, the operations comprising:
- in response to a request for data, determining that retrieving the data requires calling one or more services to retrieve secondary data associated with the data,
- determining the services required to retrieve the secondary data by analyzing dependencies between the services, wherein the dependencies are stored as a directed acyclic graph (DAG) representation, wherein the DAG includes the services as nodes of the DAG and a dependency relationship between two services as a directed edge, wherein the services include at least one of a process or operation for determining or retrieving the secondary data,
- determining a sequence for calling the services based on a topological ordering of the services within the DAG, wherein, in response to determining that the topological ordering of the services comprises a service having a plurality of dependent services without any remaining dependent services, the at least two of the plurality of dependent services without any remaining dependent services are called in parallel, and
- retrieving the secondary data in the determined sequence and compiling the secondary data to provide the data requested.
Dependencies are inferred from the claim text.