# US 10,872,262 B2 · Information processing apparatus and information processing method for detecting position of object > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,262 B2 - Document ID: US10872262B2 - Status: Granted - Kind: B2 - Application number: 16/092,875 - Filed: 2017-04-07 - Published: 2019-07-04 - Granted: 2020-12-22 - Earliest claimed: 2016-04-14 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Kotaro Yano; Ichiro Umeda; Muling Guo; Hiroyuki Uchiyama - Web page: https://patentgrep.com/patent/US10872262B2 ## CPC - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G06F 9/3004: {to perform operations on memory} - G06N 3/02: Neural networks - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 7/254: involving subtraction of images - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06T 2207/20084: Artificial neural networks [ANN] - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An information processing apparatus includes an acquiring unit to acquire an input image including an object, an extraction unit to extract a region that has been changed from a background image in the input image, as a changing region, a generating unit to generate a change region image in which a pixel value of each pixel represents whether or not a corresponding pixel in the input image belongs to the changing region, an extraction unit to extract a plurality of feature images from the input image as an output of a neural network by inputting a combination of the input image and the change region image to the neural network, and a detection unit to detect a position of the object from the plurality of feature images. ## Claims ### Claim 1 An information processing apparatus comprising: (A) one or more processors; and (B) at least one memory coupled to the one or more processors, the at least one memory having stored thereon instructions that, when executed by the one or more processors, cause the apparatus: (a) to acquire an input image including an object; (b) to extract a region that has been changed from a background image in the input image, as a changing region, a binary image in which each pixel value is one or zero and the pixel value of the pixel in the change region is different from the pixel value of the pixel in the region other than the change region; (c) to generate a change region image in which a pixel value of each pixel represents whether or not a corresponding pixel in the input image belongs to the changing region; (d) to extract a plurality of feature images from the input image as an output of a neural network by inputting a combination of the input image and the change region image to the neural network; and (e) to detect a position of the object from the plurality of feature images. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the changed region image is extracted based on the input image and a background image. ### Claim 3 (depends on 1) The information processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the apparatus: (f) to acquire a plurality of images that are continuous on a time series and each of which does not include the object; and g) to generate the background image based on the plurality of images that are continuous on the time series and each of which does not include the object. ### Claim 4 (depends on 3) The information processing apparatus according to claim 3, wherein the plurality of images that are continuous on the time series are photographed in a fixed visual field. ### Claim 5 (depends on 1) The information processing apparatus according to claim 1, wherein the plurality of feature images are extracted by combining the input image with the change region image and using a convolutional neural network. ### Claim 6 (depends on 1) The information processing apparatus according to claim 1, wherein a candidate region is a candidate for a region where the object exists in the input image, features of the plurality of feature images are based on the candidate region, and the position of the object is based on the integrated features. ### Claim 7 (depends on 1) The information processing apparatus according to claim 1, wherein the object is detected from the plurality of feature images by using a fully connected neural network. ### Claim 8 (depends on 1) The information processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the apparatus: (f) to detect respective positions of a plurality of predetermined portions of the object; and (g) estimate a position of the object by integrating detection results of the respective positions of the plurality of predetermined parts of the object. ### Claim 9 (depends on 1) The information processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the apparatus (f) to output a detection result. ### Claim 10 (depends on 9) The information processing apparatus according to claim 9, wherein the instructions, when executed by the one or more processors, further cause the apparatus: (g) to generate an appearance probability image indicating appearance probability of the object from the plurality of feature images; and (h) to output the generated appearance probability image. ### Claim 11 An information processing method comprising: (a) acquiring an input image including an object; (b) extracting a region that has been changed from a background image in the acquired input image, as a changing region, a binary image in which each pixel value is one or zero and the pixel value of the pixel in the change region is different from the pixel value of the pixel in the region other than the change region; (c) generating a change region image in which a pixel value of each pixel represents whether or not a corresponding pixel in the input image belongs to the changing region; (d) extracting a plurality of feature images from the acquired input image as an output of a neural network by inputting a combination of the input image and the extracted change region image to the neural network; and (e) detecting a position of the object from the plurality of extracted feature images. ### Claim 12 (depends on 11) The information processing method according to claim 11, wherein the change region image is extracted based on the input image and a background image. ### Claim 13 (depends on 11) The information processing method according to claim 11, further comprising: (f) acquiring a plurality of images that are continuous on a time series and each of which does not include the object; and (g) generating the background image based on the plurality of images that are continuous on the time series and each of which does not include the object. ### Claim 14 (depends on 11) The information processing method according to claim 11, wherein the plurality of images that are continuous on the time series are photographed in a fixed visual field. ### Claim 15 (depends on 11) The information processing method according to claim 11, wherein the plurality of feature images are extracted by combining the input image with the change region image and using a convolutional neural network. ### Claim 16 (depends on 11) The information processing method according to claim 11, wherein a candidate region is a candidate for a region where the object exists in the input image, features of the plurality of feature images are based on the candidate region, and the position of the object is based on the integrated features. ### Claim 17 (depends on 11) The information processing method according to claim 11, wherein the object is detected from the plurality of feature images by using a fully connected neural network. ### Claim 18 (depends on 11) The information processing method according to claim 11, further comprising: (f) detecting respective positions of a plurality of predetermined portions of the object; and (g) estimating a position of the object by integrating detection results of the respective positions of the plurality of predetermined parts of the object. ### Claim 19 (depends on 11) The information processing method according to claim 11, further comprising (f) outputting a detection result. ### Claim 20 (depends on 19) The information processing method according claim 19, further comprising: (g) generating an appearance probability image indicating appearance probability of the object from the plurality of feature images; and (h) outputting the generated appearance probability image generated by the detecting step. ### Claim 21 A non-transitory computer-readable storage medium that stores a program for causing a computer to perform: acquiring an input image including an object; extracting a region that has been changed from a background image in the acquired input image, as a changing region, a binary image in which each pixel value is one or zero and the pixel value of the pixel in the change region is different from the pixel value of the pixel in the region other than the change region; generating a change region image in which a pixel value of each pixel represents whether or not a corresponding pixel in the input image belongs to the changing region; extracting a plurality of feature images from the acquired input image as an output of a neural network by inputting a combination of the input image and the extracted change region image to the neural network; and detecting a position of the object from the plurality of extracted feature images. ### Claim 22 (depends on 21) The storage medium according to claim 21, wherein the change region image is extracted based on the input image and a background image. ### Claim 23 (depends on 21) The storage medium according to claim 21, in which the medium stores a program for causing a computer further to perform: acquiring a plurality of images that are continuous on a time series and each of which does not include the object; and generating the background image based on the plurality of images that are continuous on the time series and each of which does not include the object. ### Claim 24 (depends on 21) The storage medium according to claim 21, wherein the plurality of images that are continuous on the time series are photographed in a fixed visual field. ### Claim 25 (depends on 21) The storage medium according to claim 21, wherein the plurality of feature images are extracted by combining the input image with the change region image and using a convolutional neural network. ### Claim 26 (depends on 21) The storage medium according to claim 21, wherein a candidate region is a candidate for a region where the object exists in the input image, features of the plurality of feature images are based on the candidate region, and the position of the object is based on the integrated features. ### Claim 27 (depends on 21) The storage medium according to claim 21, wherein the object is detected from the plurality of feature images by using a fully connected neural network. ### Claim 28 (depends on 21) The storage medium according to claim 21, in which the medium stores a program for causing a computer further to perform: detecting respective positions of a plurality of predetermined portions of the object; and estimating a position of the object by integrating detection results of the respective positions of the plurality of predetermined parts of the object. ### Claim 29 (depends on 28) The storage medium according to claim 28, in which the medium stores a program for causing a computer further to perform outputting a detection result. ### Claim 30 (depends on 29) The storage medium according to claim 29, wherein the detecting step is configured to further generate an appearance probability image indicating appearance probability of the object from the plurality of feature images, and the outputting step is configured to further output the appearance probability image generated by the detecting step. ### Claim 31 (depends on 8) The information processing apparatus according to claim 8, wherein a region that includes the respective positions of the plurality of predetermined parts of the object is estimated as the position of the object. ### Claim 32 (depends on 18) The information processing method according to claim 18, wherein a region that includes the respective positions of the plurality of predetermined parts of the object is estimated as the position of the object. ### Claim 33 (depends on 28) The storage medium according to claim 28, wherein a region that includes the respective positions of the plurality of predetermined parts of the object is estimated as the position of the object.