# US 12,015,879 B2 · Contextual automated surveillance by a mobile robot > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,015,879 B2 - Document ID: US12015879B2 - Status: Granted - Kind: B2 - Application number: 17/370,801 - Filed: 2021-07-08 - Published: 2021-10-28 - Granted: 2024-06-18 - Earliest claimed: 2018-08-09 - Assignees: Cobalt Robotics Inc. - Applicants: Cobalt Robotics Inc. - Inventors: Travis J. Deyle; Erik Schluntz; Peregrine Badger; Sarvagya Vaish; Michael J. O'Kelly - Web page: https://patentgrep.com/patent/US12015879B2 ## CPC - H04N 7/185: {from a mobile camera, e.g. for remote control} - G01C 21/206: {specially adapted for indoor navigation} - G01C 21/3453: {Special cost functions, i.e. other than distance or default speed limit of road segments} - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - G05D 1/0219: {ensuring the processing of the whole working surface} - G05D 1/227: Handing over between remote control and on-board control; Handing over between remote control arrangements - G05D 1/648: Performing a task within a working area or space, e.g. cleaning - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES ## Abstract A mobile robot is configured for operation in a commercial or industrial setting, such as an office building or retail store. The robot can patrol one or more routes within a building, and can detect violations of security policies by objects, building infrastructure and security systems, or individuals. In response to the detected violations, the robot can perform one or more security operations. The robot can include a removable fabric panel, enabling sensors within the robot body to capture signals that propagate through the fabric. In addition, the robot can scan RFID tags of objects within an area, for instance coupled to store inventory. Likewise, the robot can generate or update one or more semantic maps for use by the robot in navigating an area and for measuring compliance with security policies. ## Claims ### Claim 1 A mobile robot, comprising: a motorized base configured to move the mobile robot within a building; a wireless transceiver configured to communicatively couple to a remote security system; a camera system configured to capture images of the mobile robot's surroundings; and a controller configured to: identify a pre-determined individual in an image captured by the camera system; configure the mobile robot to monitor the identified individual; while the individual is being monitored by the mobile robot, determine that the individual triggers a security condition comprising moving within a threshold distance of a boundary defining an area to which the individual does not have access; in response to the individual moving within the threshold distance of the boundary, cause the mobile robot to 1) project an indication of the boundary on the ground and 2) warn the individual not to cross the boundary and perform a security operation without human operator intervention; and in response to determining that the individual has crossed the boundary, perform an enhanced security operation determined to be more likely than the performed security operation to eliminate the security condition. ### Claim 2 (depends on 1) The mobile robot of claim 1, wherein the pre-determined individual is an unidentified individual. ### Claim 3 (depends on 1) The mobile robot of claim 1, wherein configuring the mobile robot to monitor the identified individual comprises configuring the mobile robot to maintain a second threshold distance or a line of sight to the individual. ### Claim 4 (depends on 1) The mobile robot of claim 1, wherein configuring the mobile robot to monitor the identified individual comprises configuring the mobile robot to capture images or video of the individual. ### Claim 5 (depends on 1) The mobile robot of claim 1, wherein determining that the individual triggers a security condition further comprises determining that a difference in location of the individual between a first image and a second image captured by the camera system exceeds a threshold distance. ### Claim 6 (depends on 1) The mobile robot of claim 1, wherein determining that the individual triggers a security condition further comprises determining that a distance between the individual and a second individual exceeds a threshold distance. ### Claim 7 (depends on 1) The mobile robot of claim 1, wherein determining that the individual triggers a security condition further comprises determining that the individual is performing a prohibited action. ### Claim 8 (depends on 1) The mobile robot of claim 1, wherein determining that the individual triggers a security condition further comprises determining that the individual enters a location the individual is not allowed to access. ### Claim 9 (depends on 1) The mobile robot of claim 1, wherein determining that the individual triggers a security condition further comprises determining that the individual is interacting with an object that the individual is not allowed to interact with. ### Claim 10 (depends on 1) The mobile robot of claim 1, wherein determining that the individual triggers a security condition further comprises determining that the mobile robot has lost sight of the individual. ### Claim 11 (depends on 1) The mobile robot of claim 1, wherein determining that the individual triggers a security condition further comprises determining that a distance between the mobile robot and the individual has exceeded a threshold distance. ### Claim 12 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises providing images or video of the individual to a central security system. ### Claim 13 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises providing images or video of the individual to a human operator or security personnel. ### Claim 14 (depends on 1) The mobile robot of claim 1, wherein the security operation is performed without providing images or video captured by the camera system to a human operator. ### Claim 15 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises continuing on a patrol route. ### Claim 16 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises monitoring a location for a period of time. ### Claim 17 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises capturing additional images of the location. ### Claim 18 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises notifying one or more security personnel of the individual and the security condition. ### Claim 19 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises enabling remote control of the mobile robot by a human operator. ### Claim 20 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises instructing one or more additional mobile robots to monitor the individual. ### Claim 21 (depends on 1) The mobile robot of claim 1, wherein performing the security operation comprises locking one or more doors within a vicinity of the individual.