Dynamic scaling of a cluster of computing nodes used for distributed execution of a program

A non-transitory computer-readable medium whose contents configure a computing system to perform a method, the method comprising:

• initiating execution of an indicated program in a distributed manner on a cluster of multiple computing nodes at a first time;
• at a second time subsequent to the first time and while the execution of the indicated program is ongoing, determining, by the configured computing system, to modify a quantity of the multiple computing nodes in the cluster during the ongoing execution of the indicated program;
• identifying, by the configured computing system, a first subset of the multiple computing nodes that each are part of distributed storage at the second time for the ongoing execution of the indicated program, and a second subset of one or more of the multiple computing nodes that are distinct from the multiple computing nodes of the first subset and that are not part of the distributed storage at the second time;
• and
• initiating, by the configured computing system, a change in the quantity of the multiple computing nodes of the cluster while the execution of the indicated program is ongoing in response to the determining to modify the quantity, the initiated change in the quantity being performed for one of the first subset and of the second subset that is selected for the initiated change based at least in part on a type of the determined modifying of the quantity, the initiating of the change being performed by the configured computing system.

A computing system, comprising:

• one or more processors;
• and
• one or more modules of a distributed execution service that are configured to, when executed by at least one of the one or more processors, manage distributed execution of programs for clients by, for each of one or more of the clients:
• receiving information from the client regarding execution of an indicated program in a distributed manner on a cluster of multiple computing nodes;
• initiating the execution of the indicated program in the distributed manner on the cluster of multiple computing nodes at a first time;
• at a second time subsequent to the first time and while the execution of the indicated program is ongoing, determining to modify a quantity of first computing nodes in the cluster that are not part of distributed storage for information used for the ongoing execution of the indicated program, the first computing nodes being distinct from at least some computing nodes in the cluster at the second time that each are part of the distributed storage;
• and
• initiating a change in the quantity of the first computing nodes of the cluster while the execution of the indicated program is ongoing and in response to the determining to modify the quantity.