Data processing in a hybrid cluster environment

A computer system, comprising:

• one or more processors, one or more computer-readable memories and one or more computer-readable, tangible storage devices;
• and
• program instructions, stored on at least one of the one or more computer-readable, tangible storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to perform operations comprising:
• providing a hybrid cluster environment with a public cloud cluster having nodes storing data and a plurality of private clusters, wherein each of the plurality of private clusters has nodes storing data;
• receiving registration data that indicates a customer identifier, a new private cluster, and a file transfer server;
• adding the new private cluster to the plurality of private clusters in the hybrid cluster environment;
• receiving input to design a job to process data in the hybrid cluster environment;
• determining that the job is to be deployed to the new private cluster;
• deploying the job to the new private cluster using the file transfer server, wherein the job is executed at the new private cluster;
• and
• receiving job status information and one or more job logs with the file transfer server.