Modular memoization, tracking and train-data management of feature extraction

A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code being executable by at least one hardware processor to:

• receive at least one electronic document representing a dependency graph comprising feature extractors at each graph node and directed edges corresponding to computational dependencies of the feature extractors;
• for at least some feature extractors:
• i) determine extractor defining data, comprising extractor data and computational dependencies of said graph node in said dependency graph,
• ii) compute a node lookup key based on said extractor defining data,
• iii) when said node lookup key is associated with stored data, said stored data may be retrieved, wherein a non-transitory computer-readable storage medium has stored therein previously-computed node lookup keys and associated previously-computed data, and
• iv) when node lookup key is not associated with stored data, compute new data, store said new data on the non-transitory computer-readable storage medium, and associate said node lookup key with said new data;
• and
• send some of said stored data as an output set of said dependency graph.

A computerized system, comprising:

• (a) a non-transitory computer-readable storage medium having stored thereon program code for:
• receiving at least one electronic document representing a dependency graph comprising feature extractors at each graph node and directed edges corresponding to computational dependencies of the feature extractors;
• for at least some feature extractors:
• (1) determining extractor defining data, comprising extractor data and computational dependencies of said graph node in said dependency graph,
• (2) computing a node lookup key based on said extractor defining data,
• (3) when said node lookup key is associated with stored data, said stored data may be retrieved, wherein a non-transitory computer-readable storage medium has stored therein previously-computed node lookup keys and associated previously-computed data, and
• (4) when node lookup key is not associated with stored data, computing new data, storing said new data on the non-transitory computer-readable storage medium, and associating said node lookup key with said new data;
• sending stored data as an output set of said dependency graph, and
• (b) at least one hardware processor configured to execute said program code.