Mask optimization preferentially accounting for overlap regions

A non-transitory machine-readable medium storing a program which when executed by at least one processing unit optimizes a mask layout generated from a design layout of an integrated circuit (IC), the program comprising sets of instructions for:

• based on an initial mask layout for a first layer of the design layout, generating a simulated wafer image comprising shapes representing IC components of the first layer, the first layer comprising a first IC component that has a relationship with a second IC component on a second layer of the design layout;
• identifying, in the simulated wafer image, (i) a first set of evaluation points along a boundary of a first shape of the first IC component that overlap with a second shape of the second IC component and (ii) a second set of evaluation points along the boundary of the first IC component shape that do not overlap with the second IC component shape;
• identifying an objective function for use in modifying the initial mask layout, the objective function accounting for differences at each evaluation point between the first shape in the simulated wafer image and a corresponding shape in a target wafer image for the first layer, wherein differences between the first shape and the corresponding shape at the first set of evaluation points are weighted more heavily in the objective function than differences between the first shape and the corresponding shape at the second set of evaluation points;
• and
• to improve overlap between the first shape and the second shape, modifying the initial mask layout based on an evaluation of the objective function to reach an extrema of the objective function, in order to produce a modified mask layout to produce a mask to use to manufacture the IC.