Open Source Software Testing

A system comprising:

• a presubmit check server system that is configured to perform presubmit checks on system software code updates;
• and
• one or more storage devices storing instructions that are operable, when executed by the presubmit check server system, to cause the presubmit check server system to perform operations comprising:
• receiving, from a user device, a system software code update and a request to perform a presubmit check on the system software code update;
• in response to receiving the request to perform the presubmit check on the system software code update, obtaining a presubmit check code module;
• generating a workflow model that includes multiple vertices that are connected by edges, wherein each vertex represents a presubmit check and each edge represents a dependency between connected presubmit checks;
• executing a particular presubmit check of the workflow model against the system software code update;
• determining that a result of the particular presubmit check is probabilistic in that the particular presubmit check relies on a signal or data that is unpredictable during the particular presubmit check;
• and
• based on determining that the result of the particular presubmit check is probabilistic, executing a subsequent presubmit check that is connected to the particular presubmit check in the workflow model.

A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers that, upon such execution by the one or more computers, cause the one or more computers to perform operations comprising:

• receiving, from a user device, a system software code update and a request to perform a presubmit check on the system software code update;
• in response to receiving the request to perform the presubmit check on the system software code update, obtaining a presubmit check code module;
• generating a workflow model that includes multiple vertices that are connected by edges, wherein each vertex represents a presubmit check and each edge represents a dependency between connected presubmit checks;
• executing a particular presubmit check of the workflow model against the system software code update;
• determining that a result of the particular presubmit check is probabilistic in that the particular presubmit check relies on a signal or data that is unpredictable during the particular presubmit check;
• and
• based on determining that the result of the particular presubmit check is probabilistic, executing a subsequent presubmit check that is connected to the particular presubmit check in the workflow model.