# US 10,499,782 B2 · Robot cleaner and a system including the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,499,782 B2 - Document ID: US10499782B2 - Status: Granted - Kind: B2 - Application number: 15/688,315 - Filed: 2017-08-28 - Published: 2018-03-01 - Granted: 2019-12-10 - Earliest claimed: 2016-08-30 - Assignees: LG Electronics Inc. - Applicants: LG Electronics Inc. - Inventors: Jaeduck Jung - Web page: https://patentgrep.com/patent/US10499782B2 ## CPC - A47L 11/4011: {Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor} - A47L 9/2805: {Parameters or conditions being sensed} - A47L 9/2852: {Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles} - A47L 9/2857: {User input or output elements for control, e.g. buttons, switches or displays} - G05D 1/0212: {with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287)} - G05D 1/2249: {using augmented reality} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - A47L 2201/04: Automatic control of the travelling movement; Automatic obstacle detection - G05D 1/0246: {using a video camera in combination with image processing means} - G05D 1/0255: {using acoustic signals, e.g. ultra-sonic singals (sonar systems designed for anti-collision purposes G01S15/93)} - G05D 2105/10: for cleaning, vacuuming or polishing - G05D 2107/40: Indoor domestic environment - G05D 2109/10: Land vehicles - A47L: DOMESTIC WASHING OR CLEANING (brushes A46B; cleaning quantities of bottles or of other hollow articles of one kind B08B9/00; laundry D06F); SUCTION CLEANERS IN GENERAL (cleaning in general B08) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A robot cleaner includes a camera, a display unit configured to display a cleaning region projected by the camera, a communication unit configured to perform communication with a mobile terminal to receive an image including the cleaning region captured by the mobile terminal, and a control unit configured to search a cleaning region based on a traveling path of the robot cleaner, to predict an estimated traveling path of the robot cleaner based on the cleaning region and the image including the cleaning region, to generate an augmented reality (AR) image of the estimated traveling path, and to display the AR image to be superimposed on the image including the cleaning region. ## Claims ### Claim 1 A robot cleaner comprising: a communication unit configured to receive an image captured by a mobile terminal; a control unit configured to search a cleaning region based on a traveling path of the robot cleaner, to acquire an estimated traveling path of the robot cleaner based on the searched cleaning region and the image received from the mobile terminal, and to generate an augmented reality (AR) image of the estimated traveling path; and a display unit configured to display the AR image to be superimposed on the image received from the mobile terminal. ### Claim 2 (depends on 1) The robot cleaner according to claim 1, wherein the AR image of the estimated traveling path includes a cleaned region and a region to be cleaned. ### Claim 3 (depends on 2) The robot cleaner according to claim 2, wherein the control unit controls the display unit to distinguishably display the cleaned region and the region to be cleaned. ### Claim 4 (depends on 3) The robot cleaner according to claim 3, wherein the control unit controls the display unit to display the cleaned region by a solid line and displays the region to be cleaned by a dotted line. ### Claim 5 (depends on 1) The robot cleaner according to claim 1, wherein the control unit controls the display unit to display the AR image of the estimated traveling path in the form of at least one of a stereoscopic image, a holographic image and a moving image. ### Claim 6 (depends on 1) The robot cleaner according to claim 1, wherein the cleaning region includes a region to be cleaned by the robot cleaner. ### Claim 7 (depends on 1) The robot cleaner according to claim 1, wherein the control unit activates an AR mode for generating and displaying the AR image if the robot cleaner is included in a screen of the mobile terminal. ### Claim 8 An augmented reality (AR) system comprising: a mobile terminal configured to capture an image and to transmit the captured image to a robot cleaner; and the robot cleaner including: a communication unit configured to receive the image from the mobile terminal; a control unit configured to search a cleaning region based on a traveling path of the robot cleaner, to acquire an estimated traveling path of the robot cleaner based on the searched cleaning region and the image received from the mobile terminal, and to generate an augmented reality (AR) image of the estimated traveling path; and a display unit configured to display the AR image to be superimposed on the image received from the mobile terminal. ### Claim 9 (depends on 8) The AR system according to claim 8, wherein the AR image of the estimated traveling path includes a cleaned region and a region to be cleaned. ### Claim 10 (depends on 9) The AR system according to claim 9, wherein the control unit controls the display unit to distinguishably display the cleaned region and the region to be cleaned. ### Claim 11 (depends on 10) The AR system according to claim 10, wherein the control unit controls the display unit to display the cleaned region by a solid line and displays the region to be cleaned by a dotted line. ### Claim 12 (depends on 8) The AR system according to claim 8, wherein the control unit controls the display unit to display the AR image of the estimated traveling path in the form of at least one of a stereoscopic image, a holographic image and a moving image. ### Claim 13 (depends on 8) The AR system according to claim 8, wherein the cleaning region includes a region to be cleaned by the robot cleaner. ### Claim 14 (depends on 8) The AR system according to claim 8, wherein the control unit activates an AR mode for generating and displaying the AR image if the robot cleaner is included in a screen of the mobile terminal. ### Claim 15 A non-transitory recording medium having recorded thereon a computer-readable program for performing a method of operating a robot cleaner, the method comprising: receiving an image captured by a mobile terminal from the mobile terminal; searching a cleaning region based on a traveling path of the robot cleaner; acquiring an estimated traveling path of the robot cleaner based on the searched cleaning region and the image received from the mobile terminal; generating an augmented reality (AR) image of the estimated traveling path; and displaying the AR image to be superimposed on the image received from the mobile terminal. ### Claim 16 (depends on 15) The non-transitory recording medium according to claim 15, wherein the displaying of the AR image to be superimposed on the image received from the mobile terminal includes distinguishably displaying a cleaned region and a region to be cleaned. ### Claim 17 (depends on 16) The non-transitory recording medium according to claim 16, wherein the distinguishably displaying of the cleaned region and the region to be cleaned includes displaying the cleaned region by a solid line and displaying the region to be cleaned by a dotted line. ### Claim 18 (depends on 15) The non-transitory recording medium according to claim 15, wherein the displaying of the AR image to be superimposed on the image received from the mobile terminal includes displaying the AR image in the form of at least one of a stereoscopic image, a holographic image and a moving image. ### Claim 19 (depends on 15) The non-transitory recording medium according to claim 15, wherein the cleaning region includes a region to be cleaned by the robot cleaner. ### Claim 20 (depends on 15) The non-transitory recording medium according to claim 15, wherein the method further includes activating an AR mode for generating and displaying the AR image if the robot cleaner is included in a screen of the mobile terminal.