# US 2024/0176355 A1 · Pool cleaning system and method to automatically clean surfaces of a pool using images from a camera > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0176355 A1 - Document ID: US20240176355A1 - Status: Published application - Kind: A1 - Application number: 18/529,301 - Filed: 2023-12-05 - Published: 2024-05-30 - Earliest claimed: 2012-07-10 - Assignees: Zodiac Pool Systems LLC - Applicants: Zodiac Pool Systems LLC - Inventors: Kameshwar Durvasula; William Londono Correa - Web page: https://patentgrep.com/patent/US20240176355A1 ## CPC - G05D 1/0246: {using a video camera in combination with image processing means} - E04H 4/1654: {Self-propelled cleaners} - G05D 1/0016: {characterised by the operator's input device (input arrangements for computing systems in general G06F3/00)} - G05D 1/0033: {by having the operator tracking the vehicle either by direct line of sight or via one or more cameras located remotely from the vehicle} - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - G05D 1/22: Command input arrangements - G05D 1/221: Remote-control arrangements - G05D 1/223: Command input arrangements on the remote controller, e.g. joysticks or touch screens - G05D 1/249: from positioning sensors located off-board the vehicle, e.g. from cameras - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 23/66: Remote control of cameras or camera parts, e.g. by remote control devices - E04H 4/06: Safety devices; Coverings for baths - E04H 4/1281: {Devices for distributing chemical products in the water of swimming pools} - G06T 7/001: {using an image reference approach} - G06T 7/90: Determination of colour characteristics - G06T 2207/10024: Color image - H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes} - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - E04H: BUILDINGS OR LIKE STRUCTURES FOR PARTICULAR PURPOSES; SWIMMING OR SPLASH BATHS OR POOLS; MASTS; FENCING; TENTS OR CANOPIES, IN GENERAL (foundations E02D) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A pool cleaning system for cleaning debris from a submerged surface of a swimming pool includes a self-propelled pool cleaner having rotatably-mounted supports for supporting and guiding the cleaner on the pool surface; an electric motor for enabling the rotation of the rotatably-mounted supports on the pool surface; at least one camera to capture imagery of the pool surface; a controller, in electronic communication with the at least one camera, to determine a cleanliness characteristic of the pool surface on which the cleaner has passed based on the camera imagery and generate a control signal to direct movement of the cleaner based on the cleanliness characteristic of the pool surface, and a portable electronic device configured to present a graphic on a display, the graphic depicting the submerged surface of the pool and those portions of the surface that remain uncleaned as the cleaner traverses the pool surface. ## Claims ### Claim at position 1 -19. (canceled) ### Claim 20 A pool system comprising: at least one piece of equipment for a swimming pool or spa; one or more cameras configured to capture video or images of the at least one piece of equipment or at least a portion of the swimming pool or spa; and a control system configured to receive visual data from the one or more cameras and determine a control action for the at least one piece of equipment based on the received visual data. ### Claim 21 (depends on 20) The pool system of claim 20, wherein the at least one piece of equipment comprises a cleaning device with one or more propulsion mechanisms, and wherein the cleaning device configured to travel adjacent to submerged or water-containing surfaces of the swimming pool or spa and to perform a cleaning operation. ### Claim 22 (depends on 20) The pool system of claim 20, wherein the one or more cameras are mounted external to the swimming pool or spa. ### Claim 23 (depends on 20) The pool system of claim 20, wherein the one or more cameras are mounted on the at least one piece of equipment. ### Claim 24 (depends on 20) The pool system of claim 20, wherein the one or more cameras are submergible or submerged within the swimming pool or spa. ### Claim 25 (depends on 20) The pool system of claim 20, wherein the control system is configured to detect a characteristic of the swimming pool or spa based on the visual data and control the piece of equipment based on the detected characteristic. ### Claim 26 (depends on 25) The pool system of claim 25, wherein the characteristic comprises a cleanliness of a surface of the swimming pool or spa, and wherein the control system is configured to control the piece of equipment based on the detected cleanliness being outside of a threshold. ### Claim 27 (depends on 20) The pool system of claim 20, wherein the control system is configured to communicate with the at least one piece of equipment and the one or more cameras via a wireless connection. ### Claim 28 (depends on 20) The pool system of claim 20, wherein the control system is further configured to present a graphical depiction on a remote device based on the received visual data as the control action. ### Claim 29 (depends on 20) The pool system of claim 20, wherein the control system is remote from the camera. ### Claim 30 A method for monitoring a pool system, the method comprising: receiving images of at least one piece of equipment for a swimming pool or spa or at least a portion of the swimming pool or spa from one or more cameras; determining a target area and detecting a characteristic of the target area based on the images; and generating a control response based on the detected characteristic of the target area. ### Claim 31 (depends on 30) The method of claim 30, wherein the target area comprises a portion of a surface of the swimming pool or spa, and wherein the characteristic comprises a cleanliness characteristic of the portion of the surface of the swimming pool or spa. ### Claim 32 (depends on 30) The method of claim 30, wherein generating the control response comprises controlling an operation of a piece of equipment of the swimming pool or spa. ### Claim 33 (depends on 30) The method of claim 30, wherein generating the control response comprises generating an alert on a remote device. ### Claim 34 (depends on 30) The method of claim 30, wherein generating the control response comprises wirelessly communicating with the at least one piece of equipment for the swimming pool or spa, another piece of equipment for the swimming pool or spa, or a remote device. ### Claim 35 (depends on 30) The method of claim 30, wherein generating the control response comprises prompting a user on a remote device, and wherein the method further comprises receiving a control input from the user on the remote device for controlling a piece of equipment of the swimming pool or spa. ### Claim 36 (depends on 35) The method of claim 35, wherein prompting the user comprises providing recommended cleaning strategies for a cleaning device based on the detected characteristic of the target area. ### Claim 37 (depends on 30) The method of claim 30, wherein generating the control response comprises controlling operation of a cleaning device configured to travel adjacent to submerged or water-containing surfaces of the swimming pool or spa and to perform a cleaning operation, wherein controlling operation of the cleaning device comprises one or more of controlling a path or route of the cleaning device, an aggressiveness of cleaning chemicals deployed, an aggressiveness of mechanical cleaning, or a speed of the cleaning device. ### Claim 38 (depends on 37) The method of claim 37, wherein the characteristic of the target area is a presence of a person at the swimming pool or spa, and wherein generating the control response is based on the detected presence of the person satisfying a predefined characteristic. ### Claim 39 A pool system comprising: one or more cameras configured to capture video or images of at least one piece of equipment for a swimming pool or spa or at least a portion of the swimming pool or spa; a control system configured to receive visual data from the one or more cameras and generate a control response to present a graphical depiction of a pool environment including the swimming pool or spa on a user device based on the received visual data. ### Claim 40 (depends on 39) The pool system of claim 39, wherein the control system comprises a plurality of controllers. ### Claim 41 (depends on 40) The pool system of claim 40, wherein a first controller of the plurality of controllers is configured to receive the visual data and wherein a second controller of the plurality of controllers is configured present the graphical depiction. ### Claim 42 (depends on 20) The pool system of claim 20, wherein the control system comprises a plurality of controllers. ### Claim 43 (depends on 42) The pool system of claim 42, wherein a first controller of the plurality of controllers is configured to receive the visual data and wherein a second controller of the plurality of controllers is configured to determine the control action for the at least one piece of equipment based on the received visual data.