Lineage information for streaming event data and event lineage graph structures for visualization

A computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for providing transparency in streaming event data, the operations comprising:

• receiving, at one or more computing devices, a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity;
• processing, using the one or more computing devices, the plurality of events using one or more complex event processing (CEP) rules to generate a complex event;
• in response to generating the complex event, generating at least one lineage event that comprises lineage information, the lineage information comprising respective event information representative of each of one or more source events and respective rule information representative of each of the one or more CEP rules, each of the one or more source events contributing to the complex event, the event information enabling lineage tracking and retrieval of the one or more source events from an event archive, the rule information enabling retrieval of the one or more CEP rules from the event archive and defining a condition of applying an action in response to the complex event;
• storing the lineage event and the one or more source events in the event archive, the event archive being provided as a non-transitory computer-readable storage medium;
• receiving the lineage event, the complex event and each of the one or more source events at a graph builder, the graph builder provided as a program that is executed using one or more processors;
• processing the lineage event, the complex event and each of the one or more source events using the graph builder to generate a graph;
• and
• generating a multi-dimensional visualization based on the graph, the multi-dimensional visualization being navigable based on user input.

A system, comprising:

• a computing device;
• and
• a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for providing transparency in streaming event data, the operations comprising:
• receiving a plurality of events, each event comprising event data and being generated by an event source in response to a real-world activity;
• processing the plurality of events using one or more complex event processing (CEP) rules to generate a complex event;
• in response to generating the complex event, generating at least one lineage event that comprises lineage information, the lineage information comprising respective event information representative of each of one or more source events and respective rule information representative of each of the one or more CEP rules, each of the one or more source events contributing to the complex event, the event information enabling retrieval of the one or more source events from an event archive, the rule information enabling lineage tracking and retrieval of the one or more source events from an event archive, the rule information enabling retrieval of the one or more CEP rules from the event archive and defining a condition of applying an action in response to the complex event;
• storing the lineage event and the one or more source events in the event archive the event archive being provided as a non-transitory computer-readable storage medium;
• receiving the lineage event, the complex event and each of the one or more source events at a graph builder, the graph builder provided as a program that is executed using one or more processors;
• processing the lineage event, the complex event and each of the one or more source events using the graph builder to generate a graph;
• and
• generating a multi-dimensional visualization based on the graph, the multi-dimensional visualization being navigable based on user input.