# US 2024/0201684 A1 · Remote instruction system and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2024/0201684 A1 - Document ID: US20240201684A1 - Status: Published application - Kind: A1 - Application number: 18/465,372 - Filed: 2023-09-12 - Published: 2024-06-20 - Earliest claimed: 2022-12-19 - Assignees: Toyota Jidosha Kabushiki Kaisha - Applicants: Toyota Jidosha Kabushiki Kaisha - Inventors: Yuki Suehiro; Rio Suda - Web page: https://patentgrep.com/patent/US20240201684A1 ## CPC - G05D 1/0022: {characterised by the communication link (data switching networks in general H04L12/00)} - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - G05D 1/2249: {using augmented reality} - G05D 1/2265: {involving protocol translation} - H04N 7/185: {from a mobile camera, e.g. for remote control} - B60W 60/001: {Planning or execution of driving tasks} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2556/45: External transmission of data to or from the vehicle - G05D 2107/13: Spaces reserved for vehicle traffic, e.g. roads, regulated airspace or regulated waters - G05D 2109/10: Land vehicles - G08G 1/0112: {from the vehicle, e.g. floating car data [FCD]} - G08G 1/0129: {for creating historical data or processing based on historical data} - G08G 1/0133: {for classifying traffic situation} - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract A remote instruction system displays a surrounding environment image to a remote commander based on detection data from an external sensor of a vehicle sent, via a plurality of communication networks, from the vehicle configured to execute remote support according to a remote instruction from the remote commander. The remote instruction system includes an integrated information generation unit and a display generation unit. The display control unit selects one target area from a plurality of display areas, and displays an image representing integrated information in the selected target area. ## Claims ### Claim 1 A remote instruction system configured to display a surrounding environment image to a remote commander based on detection data from an external sensor of a vehicle sent from the vehicle via a plurality of communication networks, the vehicle being configured to execute remote support according to a remote instruction from the remote commander, the remote instruction system comprising: an integrated information generation unit configured to generate, based on communication quality information of each of the communication networks, integrated information of a plurality of pieces of the communication quality information; and a display control unit configured to acquire display data of the surrounding environment image based on the detection data received from the vehicle and display the surrounding environment image in one or more of a plurality of display areas for the remote commander, wherein the display control unit is configured to select one target area from the display areas and display an image representing the integrated information in the selected target area. ### Claim 2 (depends on 1) The remote instruction system according to claim 1, wherein the display control unit is configured to, in a case where part of the detection data is sent from the vehicle, select the target area from the display areas in which the surrounding environment image corresponding to the part of the detection data is displayed. ### Claim 3 (depends on 1) The remote instruction system according to claim 1, wherein the integrated information generation unit is configured to generate a recommended speed as the integrated information, the recommended speed being a vehicle speed of the vehicle according to the communication quality information, and the display control unit is configured to display an image representing the recommended speed in the target area. ### Claim 4 A non-transitory storage medium storing a remote instruction program configured to cause a processor to operate to display a surrounding environment image to a remote commander based on detection data from an external sensor of a vehicle sent from the vehicle via a plurality of communication networks, the vehicle being configured to execute remote support according to a remote instruction from the remote commander, wherein the remote instruction program is configured to cause the processor to operate as an integrated information generation unit and a display control unit, the integrated information generation unit being configured to generate, based on communication quality information of each of the communication networks, integrated information of a plurality of pieces of the communication quality information, and the display control unit being configured to acquire display data of the surrounding environment image based on the detection data received from the vehicle and display the surrounding environment image in one or more of a plurality of display areas for the remote commander, and the remote instruction program is configured to cause the processor to operate in such a manner that the display control unit selects one target area from the display areas and displays an image representing the integrated information in the selected target area.