Automatic selection of computer hardware configuration for data processing pipelines

A system for recommending a computer hardware configuration, comprising:

• a processor;
• and
• memory storing instructions that, when executed by the processor, cause the processor to:
• receive a machine-readable specification of a computing task;
• extract a plurality of features from the machine-readable specification of the computing task;
• supply the plurality of features to a reinforcement learning model to generate a proposed computer hardware configuration to execute the computing task;
• and
• provide the proposed computer hardware configuration to a user.

Computer readable media having instructions stored thereon that, when executed by a processor, cause the processor to:

• receive a machine-readable specification of a computing task;
• extract a plurality of features from the machine-readable specification of the computing task;
• supply the plurality of features to a reinforcement learning model comprising a contextual bandit model to generate a proposed computer hardware configuration to execute the computing task;
• and
• provide the proposed computer hardware configuration to a user.