# US 2019/0318152 A1 · Image analysis apparatus, method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318152 A1 - Document ID: US20190318152A1 - Status: Published application - Kind: A1 - Application number: 16/299,210 - Filed: 2019-03-12 - Published: 2019-10-17 - Earliest claimed: 2018-04-12 - Assignees: OMRON Corporation - Applicants: OMRON Corporation - Inventors: Hatsumi AOI; Tomoyoshi AIZAWA - Web page: https://patentgrep.com/patent/US20190318152A1 ## CPC - G06V 40/166: {using acquisition arrangements} - G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness} - G06V 30/144: using a slot moved over the image; using discrete sensing elements at predetermined points; using automatic curve following means - G06V 40/165: {using facial parts and geometric relationships} - G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract In a reference position determination unit, for example, a plurality of feature points of eyes and a nose of a face are detected by rough search from an image area including a driver's face extracted by a face area extractor with a rectangular frame. Based on the feature points of the respective organs, a position between eyebrows of the driver's face is detected, and this is determined as a reference position of the face. Then, a face area re-extractor corrects the position of the rectangular frame with respect to image data so that the determined reference position of the face is the center of the rectangular frame, and an image area including the face is re-extracted from the image data by using the rectangular frame in the corrected position. ## Claims ### Claim 1 An image analysis apparatus comprising: a processor configured with a program to perform operations comprising: operation as an image acquisition unit configured to acquire an image obtained by capturing an image of a scene comprising an object to be detected; operation as a partial image extractor configured to extract a partial image of an area in which the object to be detected exists from the acquired image by using an extraction frame of a predetermined size surrounding the partial image; operation as a reference position determination unit configured to detect a position of a feature point of the object to be detected from the extracted partial image and determine a reference position of the object to be detected based on the position of the feature point; operation as a re-extractor configured to correct an extraction position in which the partial image is extracted with the extraction frame based on the determined reference position and re-extract the partial image with the extraction frame at the corrected extraction position; and operation as a state detector configured to detect a state of the object to be detected from the re-extracted partial image. ### Claim 2 (depends on 1) The image analysis apparatus according to claim 1, wherein the processor is configured with the program to perform operations such that operation as the image acquisition unit comprises operation as the image acquisition unit that acquires an image obtained by capturing an image of a scene comprising a human face, operation as the partial image extraction comprises operation as the partial image extractor that extracts a partial image of an area in which the human face exists from the acquired image by using an extraction frame of a predetermined size surrounding the partial image, operation as the reference position determination unit comprises operation as the reference position determination unit that detects positions of feature points corresponding to a plurality of organs of the human face from the extracted partial image and determines, as the reference position, an arbitrary position on a center line of the human face based on the detected positions of the respective feature points, operation as the re-extractor comprises operation as the re-extractor that corrects the extraction position in which the partial image is extracted with the extraction frame so that the reference position of the partial image is a center of the extraction frame based on the determined reference position, and re-extracts the partial image included in the extraction frame at the corrected extraction position, and operation as the state detector comprises operation as the state detector that detects a state of the human face from the re-extracted partial image. ### Claim 3 (depends on 2) The image analysis apparatus according to claim 2, wherein the processor is configured with the program to perform operations such that operation as the reference position determination unit comprises operation as the reference position determination unit that determines, on the human face, as the reference position, any of: a position between eyebrows, a vertex of a nose, a mouth center point, a midpoint of the position between the eyebrows and the nose vertex, a midpoint of the position between the eyebrows and the mouth center point, and an average position of: the position between the eyebrows; the nose vertex; and the mouth center point. ### Claim 4 (depends on 1) The image analysis apparatus according to claim 1, wherein the processor is configured with the program to perform operations such that: operation as the reference position determination unit comprises operation as the reference position determination unit that searches a position of a feature point of the object to be detected from the extracted partial image with first search accuracy, and determines a reference position of the object to be detected based on the searched feature point, and operation as the state detector comprises operation as the state detector that searches a feature point of the object to be detected with second search accuracy higher than the first search accuracy from the re-extracted partial image, and detects a state of the object to be detected based on the searched feature point. ### Claim 5 (depends on 1) The image analysis apparatus according to claim 1, wherein the processor is configured with the program to perform operations further comprising operation as an output unit configured to output information representing a state of the object to be detected, detected by the state detector. ### Claim 6 (depends on 2) The image analysis apparatus according to claim 2, wherein the processor is configured with the program such that: operation as the reference position determination unit comprises operation as the reference position determination unit that searches a position of a feature point of the object to be detected from the extracted partial image with first search accuracy, and determines a reference position of the object to be detected based on the searched feature point, and operation as the state detector comprises operation as the state detector that searches a feature point of the object to be detected with second search accuracy higher than the first search accuracy from the re-extracted partial image, and detects a state of the object to be detected based on the searched feature point. ### Claim 7 (depends on 2) The image analysis apparatus according to claim 2, wherein the processor is configured with the program to perform operations further comprising operation as an output unit configured to output information representing a state of the object to be detected, detected by the state detector. ### Claim 8 (depends on 3) The image analysis apparatus according to claim 3, wherein the processor is configured with the program such that: operation as the reference position determination unit comprises operation as the reference position determination unit that searches a position of a feature point of the object to be detected from the extracted partial image with first search accuracy, and determines a reference position of the object to be detected based on the searched feature point, and operation as the state detector comprises operation as the state detector that searches a feature point of the object to be detected with second search accuracy higher than the first search accuracy from the re-extracted partial image, and detects a state of the object to be detected based on the searched feature point. ### Claim 9 (depends on 3) The image analysis apparatus according to claim 3, wherein the processor is configured with the program to perform operations further comprising operation as an output unit configured to output information representing a state of the object to be detected, detected by the state detector. ### Claim 10 (depends on 4) The image analysis apparatus according to claim 4, wherein the processor is configured with the program to perform operations further comprising operation as an output unit configured to output information representing a state of the object to be detected, detected by the state detector. ### Claim 11 An image analysis method executed by an image analysis apparatus comprising a hardware processor and a memory, the image analysis method comprising: acquiring, by the image analysis apparatus, an image obtained by capturing an image of a scene including an object to be detected; extracting, by the image analysis apparatus, a partial image of an area in which the object to be detected exists from the acquired image by using an extraction frame of a predetermined size surrounding the partial image; detecting, by the image analysis apparatus, a position of a feature point of the object to be detected from the extracted partial image and determining, by the image analysis apparatus, a reference position of the object to be detected based on the position of the feature point; correcting, by the image analysis apparatus, an extraction position in which the partial image is extracted with the extraction frame based on the determined reference position and re-extracting, by the image analysis apparatus, the partial image with the extraction frame at the corrected extraction position; and detecting, by the image analysis apparatus, information representing a feature of the object to be detected from the re-extracted partial image. ### Claim 12 (depends on 1) A non-transitory computer-readable storage medium storing a program, which when read and executed, causes a processor to perform operations comprising the operations of the image analysis apparatus according to claim 1. ### Claim 13 (depends on 2) A non-transitory computer-readable storage medium storing a program, which when read and executed, causes a processor to perform operations comprising the operations of the image analysis apparatus according to claim 2. ### Claim 14 (depends on 3) A non-transitory computer-readable storage medium storing a program, which when read and executed, causes a processor to perform operations comprising the operations of the image analysis apparatus according to claim 3. ### Claim 15 (depends on 4) A non-transitory computer-readable storage medium storing a program, which when read and executed, causes a processor to perform operations comprising the operations of the image analysis apparatus according to claim 4. ### Claim 16 (depends on 5) A non-transitory computer-readable storage medium storing a program, which when read and executed, causes a processor to perform operations comprising the operations of the image analysis apparatus according to claim 5. ### Claim 17 (depends on 6) A non-transitory computer-readable storage medium storing a program, which when read and executed, causes a processor to perform operations comprising the operations of the image analysis apparatus according to claim 6. ### Claim 18 (depends on 7) A non-transitory computer-readable storage medium storing a program, which when read and executed, causes a processor to perform operations comprising the operations of the image analysis apparatus according to claim 7.