# US 10,482,341 B2 · Object recognition device and object recognition method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,341 B2 - Document ID: US10482341B2 - Status: Granted - Kind: B2 - Application number: 15/704,083 - Filed: 2017-09-14 - Published: 2018-03-29 - Granted: 2019-11-19 - Earliest claimed: 2016-09-29 - Assignees: Fanuc Corporation - Applicants: Fanuc Corporation - Inventors: Yuuki Takahashi; Atsushi Watanabe; Minoru Nakamura - Web page: https://patentgrep.com/patent/US10482341B2 ## CPC - G06T 7/74: {involving reference images or patches} - G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)} - G06V 10/225: {based on a marking or identifier characterising the area} - G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes} - G06V 10/32: Normalisation of the pattern dimensions - G06T 2200/04: involving 3D image data - G06T 2207/10024: Color image - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30204: Marker - G06V 2201/12: Acquisition of 3D measurements of objects - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The object recognition device includes an imaging unit that captures images of a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image, an object extraction unit that extracts an area having pixels whose pixel values are within a predetermined range from the three-dimensional image acquired by the imaging unit, an image searching unit that searches the two-dimensional image, acquired by the imaging unit, for a reference image registered in advance according to the type of an object, and a determination unit that determines the type of the object depending on whether or not the reference image searched for by the image searching unit exists in the area extracted by the object extraction unit. ## Claims ### Claim 1 An object recognition device comprising: an imaging unit that captures images of a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image; and a controller which has positional-relation-correspondence information between pixels of the three dimensional image and pixels of the two dimensional image, wherein the controller is configured to conduct: an object extraction process that extracts, in the three dimensional image whose pixels respectively have their height information, an area having pixels whose pixel values are within a predetermined height range, and that determines a search area in the two-dimensional image so as to correspond to the extracted area based on the positional-relation-correspondence information; an image searching process that searches the two-dimensional image, acquired by the imaging unit, for a reference image registered in advance according to a type of an object; and a determination process that determines the type of the object depending on whether or not the reference image searched for in the image searching process exists in the search area determined in the two dimensional image; and a control unit that controls a robot based on the determination process. ### Claim 2 (depends on 1) The object recognition device according to claim 1, wherein the controller is configured to further conduct a search area setting process that sets a search area to be searched in the image searching process so as to contain the area extracted in the object extraction process. ### Claim 3 (depends on 1) The object recognition device according to claim 1, wherein in the image searching process, the controller is configured to perform the search while changing scale of the reference image based on a distance, from the imaging unit, of each part in the three-dimensional image acquired by the imaging unit. ### Claim 4 (depends on 1) The object recognition device according to claim 1, wherein in the image searching process, the controller is configured to perform the search while deforming the reference image based on an inclination angle, relative to an optical axis of the imaging unit, of each part in the three-dimensional image acquired by the imaging unit. ### Claim 5 An object recognition method comprising: an imaging step of capturing images of a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image; an object extraction step of extracting, in the three dimensional image whose pixels respectively have their height information, an area having pixels whose pixel values are within a predetermined height range and that determines a search area in the two-dimensional image so as to correspond to the extracted area based on positional-relation-correspondence information; an image searching step of searching the two-dimensional image, acquired by the imaging step, for a reference image registered in advance according to a type of an object; and a determination step of determining the type of the object depending on whether or not the reference image searched for by the image searching step exists in the search area determined in the two dimensional image; and a control step of controlling a robot based on the determination process. ### Claim 6 An object recognition device comprising: an imaging unit that captures images of a predetermined monitoring area to acquire a three-dimensional image and a two-dimensional image; and a controller which has positional-relation-correspondence information between pixels of the three dimensional image and pixels of the two dimensional image, wherein the controller is configured to conduct: an object extraction process that binarizes and extracts, in the three dimensional image whose pixels respectively have their height information, an area having pixels whose pixel values are within a predetermined height range, and that determines a search area in the two-dimensional image so as to correspond to the extracted area based on the positional-relation-correspondence information; an image searching process that searches the two-dimensional image, acquired by the imaging unit, for a reference image registered in advance according to a type of an object; and a determination process that determines the type of the object depending on whether or not the reference image searched for in the image searching process exists in the search area determined in the two dimensional image; and a control unit that controls a robot based on the determination process.