# US 9,752,994 B2 · Image processing device, method for controlling same, program, and inspection system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,752,994 B2 - Document ID: US9752994B2 - Status: Granted - Kind: B2 - Application number: 14/475,603 - Filed: 2014-09-03 - Published: 2014-12-18 - Granted: 2017-09-05 - Earliest claimed: 2012-03-08 - Assignees: OMRON Corporation - Applicants: OMRON Corporation - Inventors: Takeshi Kojima; Yasumoto Mori; Yasuhiro Ohnishi - Web page: https://patentgrep.com/patent/US9752994B2 ## CPC - G01N 21/956: Inspecting patterns on the surface of objects {(contactless testing of electronic circuits G01R31/308; testing currency G07D; manufacturing processes per se of semiconductor devices implementing a measuring step H10P74/20)} - G01B 11/00: Measuring arrangements characterised by the use of optical techniques - G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50}) - G01B 11/254: {Projection of a pattern, viewing through a pattern, e.g. moiré} - G01N 21/4738: {Diffuse reflection (precedence is given to G01N21/55 - G01N21/57 if specular component is taken into consideration), e.g. also for testing fluids, fibrous materials} - G01N 21/55: Specular reflectivity - G01N 21/8803: {Visual inspection (measuring projectors G01B9/08)} - G01N 21/8851: {Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges (G01N21/8806 and G01N21/93 - G01N21/95692 take precedence; optical measurement of dimensions G01B11/00; optical scanning G02B26/10; image transformation G06T3/00; computerised image enhancement G06T5/00; image processing per se for flaw detection G06T7/0002)} - G01N 21/95684: {Patterns showing highly reflecting parts, e.g. metallic elements} - G01N 2021/557: {Detecting specular reflective parts on sample} - G01N 2021/95638: {for PCB's} - G01N 2021/95646: {Soldering} - G01N 2021/95661: {for leads, e.g. position, curvature} - G01N 2021/95669: {for solder coating, coverage} - G01N 2201/062: LED's - G01N 2201/12: Circuits of general importance; Signal processing - G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q) - G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS ## Abstract A measurement target is captured using two types of illumination patterns set such that only color features of specular objects change, the two images that are obtained are compared, and objects are identified as being specular objects or diffuse objects depending on whether the color features change markedly. At this time, the emission intensity distribution of each illumination pattern is devised such that specular lobe components that are included in reflected light are canceled out. ## Claims ### Claim 1 An image processing device comprising: illumination means having a surface light source for irradiating a measurement target including a specular object and a diffuse object with light of a predetermined illumination pattern; illumination control means configured to control the illumination pattern of the illumination means; imaging means configured to image the measurement target; and region recognition means configured to recognize a specular object region on the measurement target by analyzing an image obtained by the imaging means, wherein the region recognition means compares a first image obtained by imaging the measurement target in a state where light of a first illumination pattern is irradiated from the illumination means and a second image obtained by imaging the measurement target in a state where light of a second illumination pattern is irradiated from the illumination means, and recognizes the specular object region on the measurement target based on a result of the comparison, the first image and the second image are images that include a plurality of the same channels, the first illumination pattern includes a plurality of first illumination sub-patterns that each corresponds to a different one of the plurality of channels, and that have mutually different emission intensity distributions, one first illumination sub-pattern among the plurality of first illumination sub-patterns has an emission intensity distribution in which an emission intensity is uniform in-plane, and the other first illumination sub-patterns have an emission intensity distribution in which an emission intensity changes linearly in accordance with an angle of incidence of light with respect to the measurement target when considered at a certain cross-section that includes an optical axis of the imaging means, the second illumination pattern includes a plurality of second illumination sub-patterns that each corresponds to a different one of the plurality of channels, and that have mutually different emission intensity distributions, the plurality of second illumination sub-patterns all have an emission intensity distribution in which an emission intensity changes linearly in accordance with an angle of incidence of light with respect to the measurement target when considered at a certain cross-section that includes the optical axis of the imaging means, and each first illumination sub-pattern and second illumination sub-pattern corresponding to the same channel have mutually different emission intensity distributions. ### Claim 2 (depends on 1) The image processing device according to claim 1, wherein the region recognition means derives a feature amount representing a difference between the first image and the second image, using a value obtained by dividing a value of all the channels of the first image and the second image by a value of the channel corresponding to the first illumination sub-pattern, in the first image, having the emission intensity distribution in which the emission intensity is uniform in-plane. ### Claim 3 (depends on 1) The image processing device according to claim 1, wherein a ratio of a total amount of light in-plane between the plurality of first illumination sub-patterns constituting the first illumination pattern is equal to a ratio of the total amount of light in-plane between the plurality of second illumination sub-patterns constituting the second illumination pattern. ### Claim 4 (depends on 1) The image processing device according to claim 1, wherein each first illumination sub-pattern and second illumination sub-pattern corresponding to the same channel have total amounts of light in-plane that are equal to each other. ### Claim 5 (depends on 1) The image processing device according to claim 1, wherein the emission intensity distribution of each first illumination sub-pattern and the emission intensity distribution of each second illumination sub-pattern are set so as to satisfy {L11(q), . . . , L1n(q)}≠{L21(q), . . . , L2n(q)} for all points q, where q is a point on the surface light source of the illumination means, i is a channel (i=1, . . . , n; n is an integer of 2 or more), L1i(q) is the emission intensity at the point q of the first illumination sub-pattern corresponding to the channel i, and L2i(q) is the emission intensity at the point q of the second illumination sub-pattern corresponding to the channel i. ### Claim 6 (depends on 1) The image processing device according to claim 1, wherein the emission intensity distribution of each first illumination sub-pattern and the emission intensity distribution of each second illumination sub-pattern are set such that f1(L11(q)−L21(q))+ . . . +fn(L1n(q)−L2n(q)) takes the same value for all points q, where q is a point on the surface light source of the illumination means, i is a channel (i=1, . . . , n; n is an integer of 2 or more), L1i(q) is the emission intensity at the point q of the first illumination sub-pattern corresponding to the channel i, L2i(q) is the emission intensity at the point q of the second illumination sub-pattern corresponding to the channel i, and fi is a function determined in advance for each channel i. ### Claim 7 (depends on 6) The image processing device according to claim 6, wherein the first illumination sub-pattern corresponding to a channel k (1≦k≦n) has an emission intensity that is uniform in-plane, and the region recognition means derives a feature amount representing a difference of the first image and the second image relating to the point p using a value that is obtained by dividing f1(V11(p)−V21(p))+ . . . +fn(V1n(p)−V2n(p)) by a value of V1k(p) where p is a point on the measurement target, V1i(p) is the value of the channel i of the pixel corresponding to the point p in the first image, and V2i(p) is the value of the channel i of the pixel corresponding to the point p in the second image, and the region recognition means determines that a portion at the point p is a specular object if the feature amount is greater than a threshold. ### Claim 8 (depends on 1) The image processing device according to claim 1, further comprising: three-dimensional measurement processing means configured to compute a direction of a normal of a surface of the specular object on the measurement target, by analyzing an image obtained by the imaging means, and configured to compute a three-dimensional shape of the surface of the specular object from a result of the direction computation, wherein the first image is also utilized in computing the three-dimensional shape by the three-dimensional measurement processing means. ### Claim 9 A method for controlling an image processing device having illumination means having a surface light source for irradiating a measurement target including a specular object and a diffuse object with light of a predetermined illumination pattern, illumination control means configured to control the illumination pattern of the illumination means, and region recognition means configured to recognize a specular object region on the measurement target by analyzing an image obtained by the imaging means, the method comprising the steps of: acquiring a first image by imaging the measurement target using the imaging means, in a state where light of a first illumination pattern is irradiated from the illumination means; acquiring a second image by imaging the measurement target using the imaging means, in a state where light of a second illumination pattern is irradiated from the illumination means; and recognizing a specular object region on the measurement target based on a result of a comparison between the first image and the second image by the region recognition means, wherein the first image and the second image are images that include a plurality of the same channels, the first illumination pattern includes a plurality of first illumination sub-patterns that each corresponds to a different one of the plurality of channels, and that have mutually different emission intensity distributions, one first illumination sub-pattern among the plurality of first illumination sub-patterns has an emission intensity distribution in which an emission intensity is uniform in-plane, and the other first illumination sub-patterns have an emission intensity distribution in which an emission intensity changes linearly in accordance with an angle of incidence of light with respect to the measurement target when considered at a certain cross-section that includes an optical axis of the imaging means, the second illumination pattern includes a plurality of second illumination sub-patterns that each corresponds to a different one of the plurality of channels, and that have mutually different emission intensity distributions, the plurality of second illumination sub-patterns all have an emission intensity distribution in which an emission intensity changes linearly in accordance with an angle of incidence of light with respect to the measurement target when considered at a certain cross-section that includes the optical axis of the imaging means, and each first illumination sub-pattern and second illumination sub-pattern corresponding to the same channel have mutually different emission intensity distributions. ### Claim 10 (depends on 9) The method for controlling an image processing device according to claim 9, wherein the region recognition means derives a feature amount representing a difference between the first image and the second image, using a value obtained by dividing a value of all the channels of the first image and the second image by a value of the channel corresponding to the first illumination sub-pattern, in the first image, having the emission intensity distribution in which the emission intensity is uniform in-plane. ### Claim 11 (depends on 10) A non-transitory computer-readable medium storing computer executable instructions that when executed by a computer cause an image processing device to execute the steps of the method for controlling an image processing device according to claim 10. ### Claim 12 An inspection system comprising: illumination means having a surface light source for irradiating a measurement target including a specular object and a diffuse object with light of a predetermined illumination pattern; illumination control means configured to control the illumination pattern of the illumination means; imaging means configured to image the measurement target; region recognition means configured to recognize a specular object region on the measurement target by analyzing an image obtained by the imaging means; and inspection means configured to extract an image of the specular object region from the image obtained by the imaging means and inspecting the extracted image, wherein the region recognition means compares a first image obtained by imaging the measurement target in a state where light of a first illumination pattern is irradiated from the illumination means and a second image obtained by imaging the measurement target in a state where light of a second illumination pattern is irradiated from the illumination means, and recognizes the specular object region on the measurement target based on a result of the comparison, the first image and the second image are images that include a plurality of the same channels, the first illumination pattern includes a plurality of first illumination sub-patterns that each corresponds to a different one of the plurality of channels, and that have mutually different emission intensity distributions, one first illumination sub-pattern among the plurality of first illumination sub-patterns has an emission intensity distribution in which an emission intensity is uniform in-plane, and the other first illumination sub-patterns have an emission intensity distribution in which an emission intensity changes linearly in accordance with an angle of incidence of light with respect to the measurement target when considered at a certain cross-section that includes an optical axis of the imaging means, the second illumination pattern includes a plurality of second illumination sub-patterns that each corresponds to a different one of the plurality of channels, and that have mutually different emission intensity distributions, the plurality of second illumination sub-patterns all have an emission intensity distribution in which an emission intensity changes linearly in accordance with an angle of incidence of light with respect to the measurement target when considered at a certain cross-section that includes the optical axis of the imaging means, and each first illumination sub-pattern and second illumination sub-pattern corresponding to the same channel have mutually different emission intensity distributions. ### Claim 13 (depends on 12) The inspection system according to claim 12, wherein the region recognition means derives a feature amount representing a difference between the first image and the second image, using a value obtained by dividing a value of all the channels of the first image and the second image by a value of the channel corresponding to the first illumination sub-pattern, in the first image, having the emission intensity distribution in which the emission intensity is uniform in-plane.