Unified metrics computation platform
AssigneeMicrosoft Technology Licensing, LLC
InventorsKhai Tran, Harsh Shah, Maneesh Varshney
Life of this patent
Estimating the term…
Abstract
A unified metrics computation platform decouples user-facing query languages from backend execution engines with the help of an intermediate platform-agnostic language, based on relational algebra procedural query language. The user needs to only specify the computation logic at a single place. By leveraging the intermediate language representation, the unified metrics computation platform can execute the same computation logic on multiple execution engines. The unified metrics computation platform unifies batch, nearline and interactive computations by automatically converting existing user defined batch logic into nearline logic. The user is presented with a unified view of the batch and nearline computation results.
Claims20
A computer implemented method comprising:
- accessing an offline script configured to perform a data processing task, the offline script being in a scripting language;
- parsing the offline script and converting the offline script into a relational algebra plan;
- optimizing the relational algebra plan into a physical plan;
- using at least one processor, based on the physical plan, generating streaming code to perform the data processing task in a nearline environment, the streaming code corresponding to the offline script;
- and
- storing the streaming code;
- in response to a query with respect to the data processing task;
- accessing results produced by the offline script;
- accessing results produced by the streaming code;
- and
- causing a presentation on a display device of the results produced by the offline script and the results produced by the streaming code.
A system comprising:
- one or more processors;
- and
- a non-transitory computer readable storage medium comprising instructions that when executed by the one or processors cause the one or more processors to perform operations comprising:
- accessing an offline script configured to perform a data processing task, the offline script being in a scripting language;
- parsing the offline script and converting the offline script into a relational algebra plan;
- optimizing the relational algebra plan into a physical plan;
- based on the physical plan, generating streaming code to perform the data processing task in a nearline environment, the streaming code corresponding to the offline script;
- and
- storing the streaming code;
- in response to a query with respect to the data processing task;
- accessing results produced by the offline script;
- accessing results produced by the streaming code;
- and
- causing a presentation on a display device of the results produced by the offline script and the results produced by the streaming code.
A machine-readable non-transitory storage medium having instruction data executable by a machine to cause the machine to perform operations comprising:
- accessing an offline script configured to perform a data processing task, the offline script being in a scripting language;
- parsing the offline script and converting the offline script into a relational algebra plan;
- optimizing the relational algebra plan into a physical plan;
- based on the physical plan, generating streaming code to perform the data processing task in a nearline environment, the streaming code corresponding to the offline script;
- and
- storing the streaming code;
- in response to a query with respect to the data processing task;
- accessing results produced by the offline script;
- accessing results produced by the streaming code;
- and
- causing a presentation on a display device of the results produced by the offline script and the results produced by the streaming code.
Dependencies are inferred from the claim text.