# US 2019/0377355 A1 · Robot for detecting human head and control method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0377355 A1 - Document ID: US20190377355A1 - Status: Published application - Kind: A1 - Application number: 16/550,804 - Filed: 2019-08-26 - Published: 2019-12-12 - Earliest claimed: 2019-07-26 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Seungchul KWAK; Jeongwoo JU - Web page: https://patentgrep.com/patent/US20190377355A1 ## CPC - A47L 9/009: {Carrying-vehicles; Arrangements of trollies or wheels; Means for avoiding mechanical obstacles} - G05D 1/243: Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals (using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249) - A47L 9/2805: {Parameters or conditions being sensed} - A47L 9/2826: {the condition of the floor} - A47L 9/2852: {Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles} - A47L 9/2889: {Safety or protection devices or systems, e.g. for prevention of motor over-heating or for protection of the user} - A47L 11/4011: {Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor} - A47L 11/4061: {Steering means; Means for avoiding obstacles; Details related to the place where the driver is accommodated} - G05D 1/0238: {using obstacle or wall sensors (G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93)} - G05D 1/0251: {extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593)} - G06T 7/73: using feature-based methods - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - A47L 2201/04: Automatic control of the travelling movement; Automatic obstacle detection - G06T 2200/04: involving 3D image data - G06T 2207/10016: Video; Image sequence - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30196: Human being; Person - G06T 2207/30201: Face - G06T 2207/30261: Obstacle - A47L: DOMESTIC WASHING OR CLEANING (brushes A46B; cleaning quantities of bottles or of other hollow articles of one kind B08B9/00; laundry D06F); SUCTION CLEANERS IN GENERAL (cleaning in general B08) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Disclosed is a robot and a control method thereof. The robot includes: a camera configured to acquire a depth image of a specific area; a processor configured to create an elevation map for the specific area based on the depth image, and determine whether a head of a person is in the specific area based on the elevation map. The processor, when determining whether the head of the person is in the specific area, is further configured to partition pixels of the elevation map into floor pixels corresponding to a floor area and non-floor pixels corresponding to a non-floor area using elevation values of the elevation map, set paths through the elevation map with a subset of the non-floor pixels as starting points for the paths, and determine whether the head of the person is in the specific area based on whether the paths converge. ## Claims ### Claim 1 A robot comprising: a camera configured to acquire a depth image of a specific area; and a processor configured to: create an elevation map for the specific area based on the depth image, and determine whether a head of a person is in the specific area based on the elevation map, wherein the processor, when determining whether the head of the person is in the specific area, is further configured to: partition pixels of the elevation map into floor pixels corresponding to a floor area and non-floor pixels corresponding to a non-floor area using elevation values of the elevation map, set paths through the elevation map with a subset of the non-floor pixels as starting points for the paths, and determine whether the head of the person is in the specific area based on whether the paths converge. ### Claim 2 (depends on 1) The robot according to claim 1, further comprising: a driving wheel to move the robot; a cleaning head to perform a cleaning function; and a processor configured to control the driving wheel and the cleaning head, wherein the processor is configured to control the driving wheel and the cleaning head such that the robot moves to avoid the head of the person, and a degree of the cleaning function decreases near the head of the person based on determining that the head of the person is in the specific area. ### Claim 3 (depends on 1) The robot according to claim 1, wherein the processor is configured to calculate an elevation value gradient with respect to each of the subset of the non-floor pixels and to set the paths in directions in which each of the elevation value gradients has a maximum value. ### Claim 4 (depends on 1) The robot according to claim 1, wherein the processor is configured to extract outermost non-floor pixels which are located on the outermost side on the elevation map among the non-floor pixels and to randomly select the subset of the non-floor pixels from the outermost non-floor pixels. ### Claim 5 (depends on 4) The robot according to claim 4, wherein the processor is configured to determine that the head of the person is in the specific area when the paths converge on ones of the pixels having a largest one of the elevation values in the elevation map. ### Claim 6 (depends on 5) The robot according to claim 5, wherein the processor is configured to determine whether the head of the person is in the specific area further based on whether a circular or elliptical curve is detected in the pixels of the depth image corresponding to the ones of the pixels where the paths converge and having the largest one of the elevation values. ### Claim 7 (depends on 5) The robot according to claim 5, wherein the processor is configured to determine whether the head of the person is in the specific area further based on brightness values of the pixels included in the paths in the depth image. ### Claim 8 (depends on 7) The robot according to claim 7, wherein the processor is configured to determine that the head of the person is in the specific area when an average value of the brightness values for ones of the pixels included in the converging paths is equal to or greater than a threshold value. ### Claim 9 (depends on 8) The robot according to claim 8, wherein the threshold value corresponds to an average value of the brightness values of the pixels in the depth image. ### Claim 10 (depends on 5) The robot according to claim 5, wherein the processor is configured to determine a style of the hair of the person when the processor determines that the head of the person is in the specific area, wherein the processor is configured to determine that the style of the hair of the person is short hair when differences between the elevation value of the floor pixels and the elevation values of the outermost non-floor pixels are equal to or greater than a threshold value, and wherein the processor is configured to determine that the style of the hair of the person is long hair when the differences between the elevation value of the floor pixels and the elevation values of the outermost non-floor pixels are less than the threshold value. ### Claim 11 (depends on 10) The robot according to claim 10, wherein second threshold value corresponds to a median value of an average value of the elevation values of the outermost non-bottom pixels associated with long hair and an average value of the elevation values of the outermost non-bottom pixels associated with short hair. ### Claim 12 A robot comprising: a camera configured to acquire a depth image of a specific area; and a processor configured to: create an elevation map for the specific area based on the depth image; and determine whether a head of a person is in the specific area based on the elevation map, wherein the processor, when determining whether the head of the person is in the specific area is further configured to: partition pixels of the elevation map into floor pixels corresponding to a floor area and non-floor pixels corresponding to a non-floor area using elevation values of the elevation map, set paths through the elevation map with a subset of the non-floor pixels as starting points, and determine whether the head of the person is in the specific area based on whether the paths converge, based on whether a circular or elliptical curve is detected in the pixels of the depth image corresponding to the pixels on which the paths converge, and based on brightness values of the pixels included in the paths in the depth image. ### Claim 13 A method of controlling a robot, comprising: acquiring a depth image of a specific area; creating an elevation map for the specific area based on the depth image; determining whether a head of a person is in the specific area based on the elevation map; and controlling an operation of the robot based on whether the head of the person is in the specific area, wherein determining whether the head of the person is in the specific area includes: partitioning pixels of the elevation map into floor pixels corresponding to a floor area and non-floor pixels corresponding to a non-floor area using elevation values of the elevation map; setting paths through the elevation map with a subset of the non-floor pixels as starting points; and determining whether the head of the person is in the specific area based on whether the paths converge. ### Claim 14 (depends on 13) The method according to claim 13, wherein setting the paths includes: calculating an elevation value gradient with respect to each of the subset of the non-floor pixels, and setting the paths in directions in which each of the elevation value gradients has a maximum value. ### Claim 15 (depends on 13) The method according to claim 13, wherein determining whether the head of the person is in the specific area based on whether the paths converge includes: extracting outermost non-floor pixels which are located on the outermost side on the elevation map among the non-floor pixels, and randomly selecting the subset of the non-floor pixels from the outermost non-floor pixels. ### Claim 16 (depends on 13) The method according to claim 13, wherein determining whether the head of the person is in the specific area based on whether the paths converge includes: determining that the head of the person is in the specific area when the paths converge on ones of the pixels having the largest elevation in the elevation map, and a circular or elliptical curve is detected in the pixels of the depth image corresponding to the ones of the pixels where the paths converge and having the largest one of the elevation values. ### Claim 17 (depends on 16) The method according to claim 16, wherein determining whether the head of the person is in the specific area based on whether the paths converge includes: determining whether the head of the person is in specific area based on brightness values of ones of the pixels included in the paths in the depth image, and determining that the head of the person is in the specific area when an average value of the brightness values of a group of the pixels included in the converging paths is equal to or greater than a threshold value. ### Claim 18 (depends on 17) The method according to claim 17, wherein the threshold value corresponds to an average value of the brightness values of the pixels in the depth image. ### Claim 19 (depends on 15) The method according to claim 15, further comprising: determining a style of hair of the person when the head of the person is in the specific area, wherein determining the style of hair of the person includes: determining that the hair of the person is short hair when differences between the elevation value of the floor pixels and the elevation values of the outermost non-floor pixels are equal to or greater than a threshold value, and determining that the hair of the person is long hair when the differences between the elevation value of the floor pixels and the elevation values of the outermost non-floor pixels are less than the threshold value. ### Claim 20 (depends on 13) The method according to claim 13, wherein the robot is a robotic vacuum cleaner that performs a cleaning function while moving, and wherein controlling an operation of the robot includes controlling the robot such that the robot moves to avoid the head of the person and a degree of the cleaning function decreases near the head of the person.