Methods and Apparatus for Functional Model-Based Data Provenance in Stream Processing Environments

Apparatus for deriving a provenance of one or more of a plurality of output data elements generated from a given output port of a processing component, comprising:

• a memory;
• and
• at least one processor coupled to the memory and operative to:
• (i) create at least one dependency function that relates the one or more output data elements to a set of one or more input ports of the processing component and a corresponding plurality of input data elements, wherein the dependency function comprises an encoding of at least one of one or more temporal filters and one or more sequence filters relating to the plurality of input data elements;
• (ii) store the at least one dependency function;
• (iii) store a history of stream-level bindings of one or more input streams to one or more input ports of the processing component and one or more output streams from one or more output ports of the processing component, and store the plurality of input data elements belonging to the one or more input streams and the plurality of output data elements belonging to the one or more output streams;
• and (iv) determine a set of one or more input data elements from the plurality of input data elements that relate to the one or more output data elements in accordance with the at least one dependency function and the history of stream-level bindings.

A method for making a computer implemented process to enable derivation of a provenance of one or more of a plurality of output data elements generated from a given output port of a processing component comprising the steps of

• instantiating first computer instructions onto a computer readable medium, the first computer instructions configured to create at least one dependency function that relates the one or more output data elements to a set of one or more input ports of the processing component and a corresponding plurality of input data elements, wherein the dependency function comprises an encoding of at least one of one or more temporal filters and one or more sequence filters relating to the plurality of input data elements;
• instantiating second computer instructions onto a computer readable medium, the second computer instructions configured to store the at least one dependency function;
• instantiating third computer instructions onto a computer readable medium, the third computer instructions configured to store a history of stream-level bindings of one or more input streams to one or more input ports of the processing component and one or more output streams from one or more output ports of the processing component, and store the plurality of input data elements belonging to the one or more input streams and the plurality of output data elements belonging to the one or more output streams;
• and
• instantiating fourth computer instructions onto a computer readable medium, the fourth computer instructions configured to determine a set of one or more input data elements from the plurality of input data elements that relate to the one or more output data elements in accordance with the at least one dependency function and the history of stream-level bindings.