Distributed computing pipeline processing

A system comprising one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• receiving a request to execute a processing pipeline, the processing pipeline comprising (i) first operations that transform raw inputs into pre-processed inputs and (ii) second operations that operate on the pre-processed inputs;
• and
• in response to receiving the request:
• assigning the first operations to two or more of a plurality of computing devices, wherein each of the plurality of computing devices is configured to execute assigned operations and maintain a queue of pre-processed inputs generated from repeatedly executing the first operations,
• assigning the second operations to one or more hardware accelerators of a plurality of hardware accelerators, wherein each of the one or more hardware accelerators is assigned to interconnect with a respective number of computing devices of the plurality of computing devices over a network, and configured to (i) receive pre-processed inputs from respective queues of the two or more computing devices assigned the first operations and (ii) perform the second operations on the received pre-processed inputs, wherein assigning each of the one or more hardware accelerators to interconnect with the respective number of computing devices comprises:
• identifying, for each of the one or more hardware accelerators, a ratio that represents assigning the respective number of computing devices to the hardware accelerator such that the respective number of computing devices are configured to provide pre-processed inputs to the hardware accelerator so as to limit or prevent starvation of the hardware accelerator;
• and
• automatically assigning, for each of the one or more hardware accelerators, the respective number of computing devices to the hardware accelerator using the identified ratio, and
• executing the first operations of the processing pipeline on the two or more computing devices and the second operations of the processing pipeline on the one or more hardware accelerators.

One or more non-transitory computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

• receiving a request to execute a processing pipeline, the processing pipeline comprising (i) first operations that transform raw inputs into pre-processed inputs and (ii) second operations that operate on the pre-processed inputs;
• and
• in response to receiving the request:
• assigning the first operations to two or more of a plurality of computing devices, wherein each of the plurality of computing devices is configured to execute assigned operations and maintain a queue of pre-processed inputs generated from repeatedly executing the first operations,
• assigning the second operations to one or more hardware accelerators of a plurality of hardware accelerators, wherein each of the one or more hardware accelerators is assigned to interconnect with a respective number of computing devices of the plurality of computing devices over a network, and configured to (i) receive pre-processed inputs from respective queues of the two or more computing devices assigned the first operations and (ii) perform the second operations on the received pre-processed inputs, wherein assigning each of the one or more hardware accelerators to interconnect with the respective number of computing devices comprises:
• identifying, for each of the one or more hardware accelerators, a ratio that represents assigning the respective number of computing devices to the hardware accelerator such that the respective number of computing devices are configured to provide pre-processed inputs to the hardware accelerator so as to limit or prevent starvation of the hardware accelerator;
• and
• automatically assigning, for each of the one or more hardware accelerators, the respective number of computing devices to the hardware accelerator using the identified ratio, and
• executing the first operations of the processing pipeline on the two or more computing devices and the second operations of the processing pipeline on the one or more hardware accelerators.