Scripting distributed, parallel programs

A system, comprising:

• one or more processors;
• and
• machine readable media storing instructions that when executed by at least some of the processors effectuate operations comprising:
• obtaining a specification of a data analysis to be performed in parallel on a computing cluster comprising a plurality of computing nodes;
• parsing the specification of the data analysis to identify rules applicable to the data analysis;
• based on the parsed specification of the data analysis, determining which data is implicated in each portion of the data analysis to be assigned to the plurality of computing nodes of the computing cluster;
• determining that a portion of the implicated data is not already present in memory of at least some of the plurality of computing nodes of the computing cluster;
• in response to the determination, distributing the portion of the implicated data according to an index that positions related values of the data on the same computing nodes of the computing cluster;
• determining which computing nodes of the computing cluster have data relevant to which rules in the data analysis and sending relevant rules to the corresponding computing nodes;
• executing the rules on the computing nodes;
• and
• aggregating results of executing the rules.