Semiconductor inspection and metrology systems for distributing job among the CPUs or GPUs based on logical image processing boundaries

A system for scalable and flexible job distribution having a plurality of worker nodes coupled to a master node comprising:

• the master node having at least one processor to run a master job manager, wherein the master job manager is configured to:
• receive input image data from a semiconductor inspection tool or a semiconductor metrology tool, wherein the input image data is of a semiconductor wafer or reticle;
• divide the input image data into at least a first job;
• and
• distribute the first job to a first CPU worker node of the plurality of worker nodes;
• the plurality of worker nodes including:
• the first CPU worker node coupled with the master node, wherein the first CPU worker node includes two or more CPUs and two or more GPUs, wherein one of the CPUs in the first CPU worker node runs a worker job manager for the first CPU worker node, wherein the worker job manager includes a module with a deep learning model;
• a second CPU worker node coupled with the master node that includes one or more CPUs without any GPU;
• and
• at least one GPU worker node coupled with the master node that includes one or more GPUs without any CPU other than to run the worker job manager for the GPU worker node;
• wherein the worker job manager of the first CPU worker node is configured to divide the first job into a plurality of tasks to be processed by the two or more CPUs or the two or more GPUs based on logical image processing boundaries, wherein the plurality of tasks include defect detection and defect classification;
• wherein the deep learning model of the first CPU worker node is configured to:
• determine whether each of the tasks is a CPU-bound task or a GPU-bound task based on the logical image processing boundaries, wherein the CPU-bound task is assigned to one of the two or more CPUs in the first CPU worker node instead of to one of the two or more GPUs in the first CPU worker node, and wherein the GPU-bound task is assigned to one of the two or more GPUs in the first CPU worker node instead of to one of the two or more CPUs in the first CPU worker node;
• wherein the worker job manager of the first CPU worker node is further configured to:
• dispatch the CPU-bound tasks for CPU-bound algorithm processes and the GPU-bound tasks for GPU-bound algorithm processes or for a GPU job manager;
• and
• wherein the GPU job manager of the first CPU worker node is configured to:
• queue in an input queue the input image data of at least some of the GPU-bound tasks for the GPU job manager to prioritize ahead of a later GPU bound-task;
• and
• distribute the input image data of the at least some of the GPU-bound tasks in equal batches to the two or more GPUs of the first CPU worker node for processing such that completion time of the plurality of tasks is minimized.