Architecture for end-to-end testing of long-running, multi-stage asynchronous data processing services

A non-transitory computer-readable storage medium storing program instructions that when executed cause a server to implement a test data repository service, wherein the test data repository service comprises:

• an output receiving function for receiving from a particular test of a plurality of independent tests output of a particular stage of an asynchronous multi-stage data processing system;
• a test stage chaining function for designating the output of the particular stage of the asynchronous multi-stage data processing system as test input data for a subsequent stage of the asynchronous multi-stage data processing system;
• a test input data subscription delivery function configured to:
• receive a subscription request indicating a set of criteria describing test input data for which notifications are to be sent;
• and
• in response to identifying test input data on the test data repository matching the set of criteria, send a notification of availability of the test input data on the test data repository matching the set of criteria;
• a request handling function for receiving a request for the test input data from a subsequent test of the plurality of independent tests to use in testing the subsequent stage;
• and
• a test input delivery function for sending to the subsequent test of the plurality of independent tests the test input data for the subsequent stage.

A method for performing end-to-end testing of a multi-stage asynchronous data processing system, the method comprising:

• a particular test, executing on one or more computers, of a plurality of independent tests used for performing end-to-end testing, to test a particular stage of the asynchronous multi-stage data processing system, requesting test input data from a test data repository service for the particular stage, wherein the test input data for the particular stage is data for input to the particular stage;
• subscribing to notifications of availability of test input data matching a set of criteria describing the end-to-end test;
• obtaining a converted version of the test input data for the particular stage, wherein the converted version is expected to match an output of the particular stage corresponding to the test input data for the particular stage;
• obtaining a comparison of the converted version to the output of the particular stage to verify operation of the particular stage;
• and
• sending to the test data repository service the output of the particular stage for storage in a test data storage as test input data for a next stage of the asynchronous multi-stage data processing system.

A system, comprising:

• one or more computers configured to run an end-to-end test comprising a plurality of independent tests of multiple stages of an asynchronous multi-stage data processing system, wherein
• one of the plurality of independent tests is configured to:
• send a request for test input data from a test data repository service for a particular stage,
• obtain a converted version of the test input data for the particular stage,
• provide an output of the particular stage to a comparison service external to the one of the plurality of independent tests,
• receive, from the comparison service, a comparison result of a comparison of the converted version to the output of the particular stage to verify operation of the particular stage, and
• transmit the output of the particular stage to the test data repository service;
• and
• one or more computers configured to provide the test data repository service, wherein
• the test data repository service is configured to store in the test data storage the output of the particular stage as test input data for a next stage of the asynchronous multi-stage data processing system.