# US 11,442,460 B2 · External control of an autonomous vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,460 B2 - Document ID: US11442460B2 - Status: Granted - Kind: B2 - Application number: 16/715,736 - Filed: 2019-12-16 - Published: 2020-06-18 - Granted: 2022-09-13 - Earliest claimed: 2017-12-06 - Assignees: Zoox, Inc. - Applicants: Zoox, Inc. - Inventors: Marcus Hammond; Marc Wimmershoff; Timothy David Kentley-Klay - Web page: https://patentgrep.com/patent/US11442460B2 ## CPC - G05D 1/221: Remote-control arrangements - B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering - G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) - G05D 1/0011: {associated with a remote control arrangement} - G05D 1/024: {in combination with a laser (lasers per se H01S)} - G05D 1/0276: {using signals provided by a source external to the vehicle (involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058)} - B60W 2520/10: Longitudinal speed - B60W 2554/00: Input parameters relating to objects - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT ## Abstract Remote controlling of a vehicle, such as an autonomous vehicle, may sometimes be more efficient and/or reliable. Such control, however, may require processes for ensuring safety of surrounding persons and objects. Aspects of this disclosure include using onboard sensors to detect objects in an environment and alter remote commands according to such objects, e.g. by reducing a maximum permitted velocity of the vehicle as a function of distance to detected objects. In some examples described herein, such remote controlling may be performed by using objects in the environment as control objects, with movements of the control objects resulting in movement of the vehicle. ## Claims ### Claim 1 A vehicle selectively configurable for remote operation comprising: a sensor; a drive system configured to cause the vehicle to move through an environment; one or more processors; and one or more non-transitory computer readable media having instructions stored thereon that when executed by the one or more processors cause the one or more processors to perform operations comprising: configuring the vehicle in a first configuration for autonomous control; controlling the vehicle autonomously in the first configuration; receiving, with the vehicle in the first configuration, an instruction to remotely control the vehicle; authenticating a remote controller, the remote controller being remote from the vehicle and configured to generate control commands for controlling the vehicle; configuring, in response to the authenticating, the vehicle in a second configuration for remote control; receiving, from the remote controller and with the vehicle in the second configuration, a control command of the control commands comprising a command for the vehicle to proceed in a direction; receiving sensor data from the sensor comprising information about an object in the environment; determining, from the sensor data, a distance of the object from the vehicle; determining, based at least in part on the control command and the distance, a modified control command; and controlling the one or more drive systems to move the vehicle in accordance with the modified control command. ### Claim 2 (depends on 1) The vehicle of claim 1, wherein the remote controller is paired via a short field wireless protocol. ### Claim 3 (depends on 1) The vehicle of claim 1, the operations further comprising pairing the remote controller with the vehicle, wherein the pairing comprises communicating with a receiver selectively physically coupled to the vehicle and in wireless communication with the remote controller. ### Claim 4 (depends on 1) The vehicle of claim 1, wherein the control system is further configured to: determine a first safety region extending a first distance in the direction; and determine a second safety region extending a second distance in the direction, wherein the second distance is greater than the first distance, wherein the modified control command is determined based at least in part on the object being located in at least one of the first safety region or the second safety region. ### Claim 5 (depends on 4) The vehicle of claim 4, wherein: the command further comprises a commanded speed, the modified control command is based at least in part on a modified speed for the vehicle, and the modified speed is zero when the distance indicates that the object is in the first safety region. ### Claim 6 (depends on 4) The vehicle of claim 4, wherein: the command comprises a commanded speed, the modified control command is based at least in part on a modified speed for the vehicle, and the modified speed is a non-zero speed that is less than the commanded speed when the distance indicates that the object is in the second safety region. ### Claim 7 (depends on 1) The vehicle of claim 1, wherein: the command comprises a commanded speed; and the commanded speed is equal to or below about 10 km/h. ### Claim 8 (depends on 1) The vehicle of claim 1, wherein the control system is further configured to authenticate the remote controller based at least in part on one or more of: establishing a connection with the remote controller; synchronizing the remote controller with the vehicle; or recognizing the remote controller using at least one of pattern recognition or code recognition. ### Claim 9 A method of selectively remotely controlling a vehicle comprising: configuring the vehicle in a first configuration for autonomous control; controlling the vehicle in the first configuration; receiving an instruction to remotely control the vehicle; authenticating, at the vehicle, a remote controller remote from the vehicle, the remote controller being configured to generate control commands for controlling the vehicle; reconfiguring the vehicle in a second configuration for remote control; receiving, from the remote controller and with the vehicle in the second configuration, a control command of the control commands for the vehicle to proceed in an environment; receiving, from a sensor disposed on the vehicle, sensor data corresponding to the environment; determining, from the sensor data, object information about an object in the environment; based at least in part on the object information, generating a modified control command; and controlling the vehicle to move in accordance with the modified control command. ### Claim 10 (depends on 9) The method of claim 9, wherein pairing the remote controller comprises at least one of establishing a wireless connection with the remote controller or establishing communication with a receiver selectively physically coupled to the vehicle, the receiver being configured to receive the control command from the remote controller. ### Claim 11 (depends on 10) The method of claim 10, wherein the authenticating the remote controller is based at least in part on one or more of: establishing a connection with the remote controller; synchronizing the remote controller with the vehicle; or recognizing the remote controller using at least one of pattern recognition or code recognition. ### Claim 12 (depends on 9) The method of claim 9, further comprising: determining a first safety region extending a first distance in a direction of travel of the vehicle; determining a second safety region extending from the first safety region a second distance in the direction; and determining, based on the object information, an object position of the object relative to the vehicle, wherein the modified control command is determined based at least in part on the object position indicating that the object is located in at least one of the first safety region or the second safety region. ### Claim 13 (depends on 9) The method of claim 9, further comprising: determining a safety region extending a distance lateral to a direction of travel of the vehicle; and determining, based on the object information, an object position of the object relative to the vehicle, wherein the modified control command is determined based at least in part on the object position indicating that the object is located in the safety region. ### Claim 14 One or more computer readable media storing instructions that, when executed by one or more processors, cause a control system of a vehicle to perform acts comprising: operating the vehicle in a first configuration for autonomous control; receiving, with the vehicle in the autonomous configuration, an instruction to remotely control the vehicle; authenticating, at the vehicle, a remote controller remote from the vehicle, the remote controller being configured to generate control commands for controlling the vehicle; reconfiguring the vehicle to a second configuration for remote control; receiving, from the remote controller and with the vehicle in the second configuration, a control command of the control commands for the vehicle to move in a direction in an environment; receiving, from a sensor associated with the vehicle, sensor data corresponding to the environment; determining, from the sensor data, a distance of an object from the vehicle; generating, based at least in part on the distance, a modified control command; and generating instructions based at least in part on the modified control command. ### Claim 15 (depends on 14) The one or more computer readable media of claim 14, wherein: the control command includes a commanded speed; and the modified control command comprises a modified speed less than the commanded speed. ### Claim 16 (depends on 15) The one or more computer readable media of claim 15, wherein: the modified speed is zero when the distance is less than or equal to a first threshold distance; and the modified speed is a non-zero speed when the distance is between the first threshold distance and a second threshold difference greater than the first threshold distance. ### Claim 17 (depends on 15) The one or more computer readable media of claim 15, wherein the modified speed is determined as a linear function of the distance. ### Claim 18 (depends on 14) The one or more computer readable media of claim 14, the acts further comprising: determining a safety region extending a distance lateral to a direction of travel of the vehicle; and determining, based on the object information, an object position of the object relative to the vehicle, wherein the modified control command is determined based at least in part on the object position indicating that the object is located in the safety region. ### Claim 19 (depends on 14) The one or more computer readable media of claim 14, the acts further comprising: pairing the remote controller, wherein the pairing the remote controller comprises at least one of establishing a wireless connection with the remote controller or establishing communication with a receiver selectively physically coupled to the autonomous vehicle, the receiver being configured to receive the control command from the remote controller. ### Claim 20 (depends on 19) The one or more computer readable media of claim 19, the authenticating the remote controller being based at least in part on one or more of: establishing a connection with the remote controller; synchronizing the remote controller with the autonomous vehicle; or recognizing the remote controller using at least one of pattern recognition or code recognition.