# US 12,117,829 B1 · Autonomous vehicle remote control apparatus and method based on heterogeneous networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,117,829 B1
- Document ID: US12117829B1
- Status: Granted
- Kind: B1
- Application number: 18/505,068
- Filed: 2023-11-08
- Published: 2024-10-10
- Granted: 2024-10-15
- Earliest claimed: 2023-04-10
- Assignees: Zhejiang Lab
- Applicants: Zhejiang Lab
- Inventors: Yuntao Liu; Yongdong Zhu; Zhifeng Zhao; Wei Hua; Qian Huang; Shuyuan Zhao; Daoxun Li; Zimian Wu
- Web page: https://patentgrep.com/patent/US12117829B1

## CPC

- G05D 1/0022: {characterised by the communication link (data switching networks in general H04L12/00)}
- B60W 60/00: Drive control systems specially adapted for autonomous road vehicles
- G05D 1/226: Communication links with the remote-control arrangements
- G05D 1/227: Handing over between remote control and on-board control; Handing over between remote control arrangements
- G05D 1/6445: {for optimising payload operation, e.g. camera or spray coverage}
- H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
- H04W 4/44: for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
- B60W 2556/45: External transmission of data to or from the vehicle
- G05D 2109/10: Land vehicles
- Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS

## Abstract

The present disclosure discloses an autonomous vehicle remote control apparatus and a method based on heterogeneous networks. The apparatus comprises a vehicle information acquisition module, a first message sending module, a first message receiving module and a first remote control module. According to the present disclosure, the possibility of failure of remote control is avoided or greatly reduced by bypassing the area where the network quality does not support remote control when planning a vehicle path, heterogeneous network resources are reasonably utilized on the vehicle driving path, the real-time performance of obtaining vehicle-related information by a remote control terminal is improved, and the availability and reliability of remote control and the safety of vehicle driving are effectively enhanced.

## Claims

### Claim 1

An autonomous vehicle remote control apparatus based on heterogeneous networks, comprising:
at least one processor, and
a memory configured to store instructions executable by the at least one processor;
wherein the instructions cause the at least one processor to:
acquire vehicle state information, vehicle position information, vehicle surrounding environment information provided by vehicle-mounted sensing devices, and an available network list of an autonomous vehicle;
generate a corresponding message according to a state of the autonomous vehicle, and send the corresponding message to a remote control terminal through a corresponding network in the available network list;
monitor different network ports according to the state of the autonomous vehicle, receive remote control messages sent by the remote control terminal, filter and merge the remote control messages, and forward a merged message; and
receive the merged message, and respond to a remote control command contained in the merged message to realize remote control of the autonomous vehicle;
acquire the available network list in a designated area and a network quality of a corresponding network;
plan an optimal driving path for the autonomous vehicle according to a starting point and an ending point of the autonomous vehicle in a case that the network quality of the designated area is known, wherein the network quality on the optimal driving path supports remote driving;
receive messages sent by the autonomous vehicle, filter and merge the messages, and forward a merged message;
receive the merged message, and generate a corresponding remote control command according to information contained in the merged message; and
generate a corresponding message according to the remote control command, and send the corresponding message to the autonomous vehicle through a corresponding network in the available network list;
wherein the instructions further cause the at least processor to:
classify the received remote control command into in-vehicle control commands and driving commands according to different controls; and
forward different remote control commands to different control systems of the autonomous vehicle to realize a response to the remote control; wherein the in-vehicle control commands are forwarded to an intelligent cockpit system of the autonomous vehicle, and the driving commands are forwarded to an autonomous driving system of the autonomous vehicle.

### Claim 2 (depends on 1)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 1, wherein instructions further cause the at least processor to:
acquire the vehicle state information from an autonomous driving system of the autonomous vehicle;
acquire current vehicle position information from the autonomous driving system of the autonomous vehicle, and send a corresponding positioning information;
acquire vehicle surrounding environment information from the autonomous driving system of the autonomous vehicle; and
acquire the current vehicle position information, and generate and maintain the available network list according to network quality information.

### Claim 3 (depends on 1)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 1, wherein instructions further cause the at least processor to:
generate a corresponding message according to a state of the autonomous vehicle;
determine a sending network for the message to be sent according to a type of the message to be sent and the current available network list; and
send the message to be sent through the corresponding sending network, wherein a sending process comprises copying and splitting the message to be sent.

### Claim 4 (depends on 1)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 1, wherein the instructions further cause the at least processor to:
monitor corresponding network ports in the available network list, and send monitored messages aggregate the monitored messages, filter same messages from different networks to reduce a repeated execution of the remote control command; and
process the received remote control messages, extract the corresponding remote control command, and send the remote control command.

### Claim 5 (depends on 2)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 2, wherein the instructions further cause the at least processor to search for available networks according to a preset network profile when generating and maintaining the available network list, the preset network profile autonomously saves network information of a last successful access, and gives priority to a network of the last successful access during network search.

### Claim 6 (depends on 3)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 3, wherein different sending priorities are set for the vehicle state information, the vehicle position information and the vehicle surrounding environment information, and wherein the vehicle state information has a first priority, the vehicle position information has a second priority, and the vehicle surrounding environment information has a third priority.

### Claim 7 (depends on 1)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 1, wherein the instructions further cause the at least processor to:
monitor a corresponding network port, receive messages sent by the autonomous vehicle, and send monitored messages; and
aggregate the monitored messages, and filter and merge same messages from different networks to obtain an effective message.

### Claim 8 (depends on 1)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 1, further comprising program instruction for:
after the autonomous vehicle is started, searching for available networks and connecting the available networks, selecting an available network as a first network to send and receive information, and receiving a remote driving start message sent by a remote control terminal;
obtaining, by the autonomous vehicle, pre-driving area information comprising path planning information of the autonomous vehicle in a pre-driving area and heterogeneous network information of the pre-driving area sent by the remote control terminal through the first network, autonomously driving according to a planned path, and generating and maintaining an available network list according to the heterogeneous network information;
during driving of the autonomous vehicle, selecting an appropriate network or a network combination to send a vehicle operating message to the remote control terminal, and monitoring remote control messages sent by the remote control terminals on all heterogeneous networks; and
after the remote control messages are processed, responding, by the autonomous vehicle, to a remote control command sent by the remote control terminal.

### Claim 9 (depends on 8)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 8, wherein the autonomous vehicle obtains network qualities of a driving area from network quality information, sorts qualities of available heterogeneous networks according to the network quality information, and forms sorted networks into an available network list of the autonomous vehicle.

### Claim 10 (depends on 8)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 8, wherein before receiving the remote driving start message sent by the remote control terminal, the autonomous vehicle, after being started, actively reports a vehicle start message to the remote control terminal, and wherein the vehicle start message comprises vehicle identification, vehicle state information and vehicle position information.

### Claim 11 (depends on 1)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 1, further comprising program instruction for:
sending, by the remote control terminal, a remote driving start message to an autonomous vehicle through a network port by which a vehicle start message is received;
obtaining, by the remote control terminal, heterogeneous network quality information of areas corresponding to a destination and a position of the autonomous vehicle, planning an optimal driving path for the autonomous vehicle, wherein network quality on the optimal driving path supports remote driving, encapsulating path information of a pre-driving area and corresponding heterogeneous network quality information into a pre-driving area message, and sending the pre-driving area message to the autonomous vehicle through a network used by the autonomous vehicle; and
receiving, by the remote control terminal, a vehicle operating message sent by the autonomous vehicle, merging received vehicle operating information, extracting complete and effective information, generating a remote control command according to a requirement, encapsulating the remote control command, and generating a remote control message to sent to the autonomous vehicle.

### Claim 12 (depends on 11)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 11, wherein the remote control message comprises a remote control message for in-vehicle control and a remote control message for driving;
different priorities for the two types of remote control message are set by the remote control terminal, and a priority of the remote control message for driving is higher than a priority of the remote control message for in-vehicle control.

### Claim 13 (depends on 11)

The autonomous vehicle remote control apparatus based on heterogeneous networks according to claim 11, wherein when no remote control command is generated at the remote control terminal for a period of time, a heartbeat remote control command is generated to prove that the remote control terminal is in a normal working state; and the autonomous vehicle does not respond when the autonomous vehicle receives the heartbeat remote control command.