# US 12,593,747 B2 · Autonomous working machine with computer vision-based monitoring and security system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,593,747 B2 - Document ID: US12593747B2 - Status: Granted - Kind: B2 - Application number: 17/437,257 - Filed: 2020-03-11 - Published: 2022-06-02 - Granted: 2026-04-07 - Earliest claimed: 2019-03-25 - Assignees: The Toro Company - Applicants: The Toro Company - Inventors: Alexander Steven Frick; Jason Thomas Kraft; Peter M. Arendt; Gregory S. Janey - Web page: https://patentgrep.com/patent/US12593747B2 ## CPC - A01D 34/008: {for automated or remotely controlled operation} - G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target} - G05D 1/0212: {with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287)} - G05D 1/0246: {using a video camera in combination with image processing means} - G05D 1/243: Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals (using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249) - G05D 1/689: Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672) - G06V 40/172: {Classification, e.g. identification} - H04W 4/40: for vehicles, e.g. vehicle-to-pedestrians [V2P] - H05B 47/19: via wireless transmission - A01D 2101/00: Lawn-mowers - A01D: HARVESTING; MOWING - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - H05B: ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL ## Abstract A supplemental monitoring and security function and system for use with an autonomous working machine such as a robotic lawn mower. The system may use cameras associated with the machine to capture image data and transmit the data to a remote computer. The system may be utilized as a theft deterrent for the mower itself, as well as provide a mobile sentry mode for surveilling and observing a property. ## Claims ### Claim 1 A method of performing a supplemental monitoring task utilizing a robotic working machine, the robotic working machine adapted to autonomously perform lawn mowing tasks within a work region of a property, wherein the method comprises: comparing a first site image to a second site image, wherein each of the site images are captured by a remote image sensor, and wherein each of the site images are captured at different times; detecting a potential unknown object in the work region based on comparing the first and second site images; communicating, from the remote image sensor to a controller located on the robotic working machine, information regarding the presence and location of the potential unknown object; dispatching, with the controller, the robotic working machine to an initial position at or near the potential unknown object, wherein the robotic working machine comprises one or more computer vision-based sensors to provide localization data of the robotic working machine as it moves to the initial position; capturing image data with the one or more computer vision-based sensors attached to the robotic working machine; comparing the image data to the site images; determining, with the controller, the presence or absence of a physical object corresponding to the potential unknown object based on the compared image data and site images; and if the potential unknown object is determined not to be a physical object, allowing for the robotic working machine to disregard the detected potential unknown object. ### Claim 2 (depends on 1) The method of claim 1, wherein dispatching the robotic working machine to the initial position at or near the potential unknown object comprises: generating user-specified location instructions using a remote computer or generating location instructions based upon the site images provided by the remote image sensor; and transmitting the location instructions to the controller. ### Claim 3 (depends on 1) The method of claim 1, further comprising triggering an action based upon determining that the potential unknown object is a physical object. ### Claim 4 (depends on 3) The method of claim 3, wherein triggering the action comprises one or more of: transmitting the image data from the robotic working machine to a remote computer; activating an alarm; and activating lighting associated with the property. ### Claim 5 (depends on 1) The method of claim 1, further comprising autonomously moving the robotic working machine to a second position different than the initial position. ### Claim 6 (depends on 1) The method of claim 1, wherein the initial position further defines a pose of the robotic working machine. ### Claim 7 (depends on 1) The method of claim 1, wherein the remote image sensor comprises a fixed-location camera. ### Claim 8 (depends on 1) The method of claim 1, wherein communicating, from the one or more remote sensors to the controller, information regarding the presence and location of the potential unknown object further comprises: wirelessly transmitting the information. ### Claim 9 (depends on 1) The method of claim 1, further comprising transmitting the image data to a remote computer. ### Claim 10 (depends on 1) The method of claim 1, wherein determining whether the potential unknown object is a physical object comprises: executing facial recognition algorithms based upon a person detected in the image data; and forgo providing notifications to a remote computer if the person is determined to be an authorized user of the robotic working machine. ### Claim 11 (depends on 1) The method of claim 1, further comprising: commanding the robotic working machine, via a remote computer, to a waypoint within the work region; and triggering an action in response to the robotic working machine sensing motion. ### Claim 12 (depends on 1) The method of claim 1, wherein, if the potential unknown object is determined to be a physical object, the method further comprises providing a notification to a user requesting removal of the physical object. ### Claim 13 (depends on 1) The method of claim 1, wherein, if the potential unknown object is determined to be a shadow, the method further comprises allowing for the robotic working machine to disregard the detected potential unknown object. ### Claim 14 A system for performing a supplemental monitoring function utilizing an autonomous working machine configured to autonomously perform a lawn mowing task, the system comprising: the autonomous working machine comprising a tool adapted to perform the lawn mowing task; a remote image sensor configured to capture at least a first site image and a second site image and configured to compare the site images, wherein each of the site images is captured at different times, and wherein the remote image sensor detects a potential unknown object based on comparing the first and second site images; a controller located on the autonomous working machine, wherein the remote image sensor communicates information regarding a presence and location of the potential unknown object to the controller, and wherein the controller dispatches the autonomous working machine to a position at or near the potential unknown object; and one or more computer vision-based sensors to provide localization data of the autonomous working machine as it moves to the position, wherein the vision-based sensors capture image data, wherein the controller is configured to compare the image data and the site images and determine whether the potential unknown object is a physical object or merely a shadow based on the compared image data and site images. ### Claim 15 (depends on 14) The system of claim 14, wherein dispatching the autonomous working machine to the position at or near the potential unknown object comprises: generating location instructions based upon the site images provided by the remote image sensor; and transmitting the location instructions to the controller. ### Claim 16 (depends on 14) The system of claim 14, wherein the controller is configured to trigger an action based upon a determination that the potential unknown object is a physical object. ### Claim 17 (depends on 16) The system of claim 16, wherein the action comprises one or more of transmitting the image data from the autonomous working machine to a remote computer, activating an alarm, or activating lighting associated with a property. ### Claim 18 (depends on 14) The system of claim 14, wherein the controller is configured to transmit the image data to a remote computer. ### Claim 19 (depends on 14) The system of claim 14, wherein, in determining whether the potential unknown object is a physical object, the controller is configured to: execute facial recognition algorithms based upon a person detected in the image data; and forgo providing notifications to a remote computer if the person is determined to be an authorized user of the autonomous working machine. ### Claim 20 (depends on 14) The system of claim 14, wherein the controller is configured to: commanding the autonomous working machine, via a remote computer, to a waypoint within a work region in which the autonomous working machine is configured to autonomously perform the lawn mowing task; and triggering an action in response to the autonomous working machine sensing motion.