Dynamically monitoring and modifying distributed execution of programs

A computer-implemented method for dynamically monitoring distributed execution of programs, the method comprising:

• determining, by one or more computing systems configured to provide a distributed program execution service, an actual aggregate quantity of computing resources of a first computing resource type that are being used at a first time for executing an indicated program in a distributed manner on multiple computing nodes provided by the distributed program execution service, the actual aggregate quantity of computing resources exceeding an expected aggregate quantity of computing resources of the first computing resource type;
• determining, by the one or more configured computing systems, that one or more bottlenecks exist regarding use of a plurality of computing resources provided by the distributed program execution service for executing programs for users of the distributed program execution service, at least one of the one or more bottlenecks being based at least in part on the actual aggregate quantity of computing resources of the first computing resource type exceeding the expected aggregate quantity of the first computing resource type;
• and
• initiating, by the one or more configured computing systems and responsive to the determining that the one or more bottlenecks exist, a modification of the executing of the indicated program on the multiple computing nodes.

A non-transitory computer-readable medium whose contents configure a computing system to dynamically modify distributed execution of programs, by performing a method comprising:

• initiating execution of an indicated program in a distributed manner on a cluster of multiple computing nodes at a first time;
• at a second time subsequent to the first time, determining that the execution of the indicated program on the multiple computing nodes is using an actual aggregate quantity of one or more types of computing resources that differs from an expected aggregate quantity of the one or more types of computing resources to be used for the execution of the indicated program;
• and
• initiating a modification of the execution of the indicated program on the multiple computing nodes to reduce the actual aggregate quantity of at least one of the one or more types of computing resources that are used for the executing of the indicated program after the second time.

A computing system configured to dynamically monitor distributed execution of programs, comprising:

• one or more processors;
• and
• one or more components of a distributed execution service that are configured to, when executed by at least one of the one or more processors, dynamically modify distributed execution of programs for users by, for each of multiple of the users:
• initiating execution of an indicated program for the user in a distributed manner on a cluster of multiple computing nodes at a first time based at least in part on information received from the user;
• at a second time subsequent to the first time, automatically determining that the execution of the indicated program on the multiple computing nodes is using an actual quantity of computing resources that is at least part of a bottleneck regarding performing the distributed execution of the programs for the users of the distributed execution service;
• and
• automatically initiating a modification of the execution of the indicated program on the multiple computing nodes so as to reduce the bottleneck.