# US 11,442,473 B2 · Systems and methods for surveillance with a visual marker > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,473 B2 - Document ID: US11442473B2 - Status: Granted - Kind: B2 - Application number: 16/901,974 - Filed: 2020-06-15 - Published: 2020-10-01 - Granted: 2022-09-13 - Earliest claimed: 2014-10-31 - Assignees: Sz Dji Technology Co., Ltd. - Applicants: Sz Dji Technology Co., Ltd. - Inventors: Ang Liu; Xiao Hu; Guyue Zhou - Web page: https://patentgrep.com/patent/US11442473B2 ## CPC - G05D 1/102: {specially adapted for vertical take-off of aircraft} - G01C 21/206: {specially adapted for indoor navigation} - G06K 7/1417: {2D bar codes} - G06K 19/06037: {multi-dimensional coding} - G08G 5/55: for a single aircraft - G08G 5/57: for unmanned aircraft - G08G 5/80: Anti-collision systems - B64U 2101/31: for surveillance - B64U 2101/60: for transporting passengers; for transporting goods other than weapons - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract A method of controlling an unmanned aerial vehicle (UAV) in an environment includes detecting, with aid of a sensor coupled to the UAV and while the UAV is in flight, a signal that is emitted from and uniquely identifying a locating marker; determining, with aid of a processor, a sequence of actions to control the UAV in response to a plurality of instructions encoded in the locating marker and communicated by the signal; and controlling, with aid of the processor, the UAV to effect the sequence of actions according to a specified time interval included in the plurality of instructions. The specified time interval indicates a wait time before the UAV effects the sequence of actions. ## Claims ### Claim 1 A method of controlling an unmanned aerial vehicle (UAV) in an environment, comprising: detecting, with aid of a sensor coupled to the UAV and while the UAV is in flight, a signal that is emitted from and uniquely identifying a locating marker; determining, with aid of a processor, a sequence of actions to control the UAV in response to a plurality of instructions encoded in the locating marker and communicated by the signal, the plurality of instructions encoded in the locating marker including at least one of an instruction that instructs the UAV to adjust position of one or more payloads of the UAV, an instruction that instructs the UAV to rotate the one or more payloads of the UAV, or an instruction that instructs the UAV to swap between the one or more payloads of the UAV; and controlling, with aid of the processor, the UAV to effect the sequence of actions according to a predetermined distance included in the plurality of instructions, the predetermined distance indicating a distance from the locating marker at which the UAV effects the sequence of actions. ### Claim 2 (depends on 1) The method of claim 1, wherein the plurality of instructions encoded in the locating marker further include a specified time interval, the specified time interval indicating a wait time before the UAV effects the sequence of actions. ### Claim 3 (depends on 1) The method of claim 1, wherein the plurality of instructions further include multiple sets of instructions associated with the locating marker, and controlling the UAV to effect the sequence of actions comprises: selecting a set of instructions from the multiple sets of instructions based on one or more parameters; and controlling the UAV to effect the sequence of actions according to the set of instructions. ### Claim 4 (depends on 3) The method of claim 3, wherein the one or more parameters include at least one of a user input, a time indicator provided by a clock of the UAV, a detected temperature, a detected amount of light, a detected motion, or a detected sound. ### Claim 5 (depends on 1) The method of claim 1, wherein the locating marker is adhered to an obstacle or a tracking target in the environment, and the method further comprises: analyzing, with aid of the processor, the signal to determine a distance from the obstacle or the tracking target; and controlling, with aid of the processor, the UAV to effect the sequence of actions when the distance is within a predetermined distance, wherein the plurality of instructions further include adjusting a direction, a speed, an altitude of the UAV to avoid the obstacle, or maintaining a relative distance between the UAV and the tracking target while the UAV is in flight. ### Claim 6 (depends on 1) The method of claim 1, wherein the environment is an indoor environment and the locating marker includes one or more features of the indoor environment, and wherein the one or more features include at least one of an opening, a doorway, a corner, a staircase, an incline, a decline, an obstacle, or an object. ### Claim 7 (depends on 1) The method of claim 1, wherein the locating marker is a dynamic marker that is changeable automatically, semi-automatically, or manually. ### Claim 8 (depends on 1) The method of claim 1, wherein the sensor comprises at least one of an acoustic sensor, an ultrasonic sensor, a LiDAR sensor, a vision sensor, or a WiFi sensor. ### Claim 9 (depends on 1) The method of claim 1, wherein the signal comprises at least one of an acoustic signal, a visual signal, an RFID signal, a radio signal, or a WiFi signal. ### Claim 10 A system for controlling an unmanned aerial vehicle (UAV) in an environment, the system comprising: a sensor coupled to the UAV configured to detect, while the UAV is in flight, a signal that is emitted from and uniquely identifying a locating marker; and a processor configured to: determine a sequence of actions to control the UAV in response to a plurality of instructions encoded in the locating marker and communicated by the signal, the plurality of instructions encoded in the locating marker including at least one of an instruction that instructs the UAV to adjust position of one or more payloads of the UAV, an instruction that instructs the UAV to rotate the one or more payloads of the UAV, or an instruction that instructs the UAV to swap between the one or more payloads of the UAV; and control the UAV to effect the sequence of actions according to a predetermined distance included in the plurality of instructions, the predetermined distance indicating a distance from the locating marker at which the UAV effects the sequence of actions. ### Claim 11 (depends on 10) The system of claim 10, wherein the plurality of instructions encoded in the locating marker further include a specified time interval, the specified time interval indicating a wait time before the UAV effects the sequence of actions. ### Claim 12 (depends on 10) The system of claim 10, wherein the plurality of instructions further include multiple sets of instructions associated with the locating marker, and the processor is further configured to: select a set of instructions from the multiple sets of instructions based on one or more parameters; and control the UAV to effect the sequence of actions according to the set of instructions. ### Claim 13 (depends on 12) The system of claim 12, wherein the one or more parameters include at least one of a user input, a time indicator provided by a clock of the UAV, a detected temperature, a detected amount of light, a detected motion, or a detected sound. ### Claim 14 (depends on 10) The system of claim 10, wherein the locating marker is adhered to an obstacle or a tracking target in the environment, and the processor is further configured to: analyze the signal to determine a distance from the obstacle or the tracking target; and controlling the UAV to effect the sequence of actions when the distance is within a predetermined distance, wherein the plurality of instructions further include adjusting a direction, a speed, an altitude of the UAV to avoid the obstacle, or maintaining a relative distance between the UAV and the tracking target while the UAV is in flight. ### Claim 15 (depends on 10) The system of claim 10, wherein the environment is an indoor environment and the locating marker includes one or more features of the indoor environment, and wherein the one or more features include at least one of an opening, a doorway, a corner, a staircase, an incline, a decline, an obstacle, or an object. ### Claim 16 (depends on 10) The system of claim 10, wherein the locating marker is a dynamic marker that is changeable automatically, semi-automatically, or manually. ### Claim 17 (depends on 10) The system of claim 10, wherein the sensor comprises at least one of an acoustic sensor, an ultrasonic sensor, a LiDAR sensor, a vision sensor, or a WiFi sensor, and the signal comprises at least one of an acoustic signal, a visual signal, an RFID signal, a radio signal, or a WiFi signal. ### Claim 18 An unmanned aerial vehicle (UAV), comprising: a sensor configured to detect, while the UAV is in flight, a signal that is emitted from and uniquely identifying a locating marker; and a processor configured to: determine a sequence of actions to control the UAV in response to a plurality of instructions encoded in the locating marker and communicated by the signal, the plurality of instructions encoded in the locating marker including at least one of an instruction that instructs the UAV to adjust position of one or more payloads of the UAV, an instruction that instructs the UAV to rotate the one or more payloads of the UAV, or an instruction that instructs the UAV to swap between the one or more payloads of the UAV; and control the UAV to effect the sequence of actions according to a predetermined distance included in the plurality of instructions, the predetermined distance indicating a distance from the locating marker at which the UAV effects the sequence of actions.