# US 11,906,963 B2 · Vehicle display device, vehicle control system, vehicle control method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,906,963 B2 - Document ID: US11906963B2 - Status: Granted - Kind: B2 - Application number: 17/674,946 - Filed: 2022-02-18 - Published: 2022-06-02 - Granted: 2024-02-20 - Earliest claimed: 2019-07-30 - Assignees: Toyota Jidosha Kabushiki Kaisha - Applicants: Toyota Jidosha Kabushiki Kaisha - Inventors: Shunsuke Tanimori; Toshiki Kindo; Yuji Suzuki; Mutsumi Matsuura; Tomoaki Miyazawa - Web page: https://patentgrep.com/patent/US11906963B2 ## CPC - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/22: Display screens - B60K 35/80: Arrangements for controlling instruments - B60K 35/81: for controlling displays - B60Q 1/507: {specific to autonomous vehicles} - B60Q 1/543: {for indicating other states or conditions of the vehicle (indicating taxis' state of hire or fare G07B13/045)} - B60W 60/005: {Handover processes (Handing over between remote control and on-board control or handing over between remote control arrangements G05D1/227)} - G05D 1/0016: {characterised by the operator's input device (input arrangements for computing systems in general G06F3/00)} - G05D 1/221: Remote-control arrangements - G05D 1/223: Command input arrangements on the remote controller, e.g. joysticks or touch screens - H04L 67/125: involving control of end-device applications over a network - H04N 5/64: Constructional details of receivers, e.g. cabinets or dust covers (furniture aspects {of television cabinets} A47B81/06) - H04N 7/183: {for receiving images from a single remote source} - B60K 2360/55: Remote control arrangements - B60K 2360/797: at the vehicle exterior - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A vehicle display device is provided, the vehicle display device including a communication section configured to receive an image of a remote driver remotely operating a vehicle, an external vehicle display section configured to display an image externally to the vehicle, and a display control section configured to cause the image of the remote driver received by the communication section to be displayed on the external vehicle display section. ## Claims ### Claim 1 A vehicle control method comprising: receiving a remote operation instruction for a vehicle from a remote operation terminal; receiving an image of a remote operator who performs remote operation of the vehicle using the remote operation terminal; and displaying the image of the remote operator on an external vehicle display section. ### Claim 2 (depends on 1) The vehicle control method of claim 1, further comprising causing the vehicle to travel to a predetermined location by autonomous travel when the remote operation by the remote operation terminal is ended. ### Claim 3 (depends on 1) The vehicle control method of claim 1, further comprising terminating displaying of the image of the remote operator when the remote operation by the remote operation terminal is ended. ### Claim 4 (depends on 1) The vehicle control method of claim 1, further comprising acquiring the image of the remote operator by an image-capture section provided to the remote operation terminal. ### Claim 5 (depends on 1) The vehicle control method of claim 1, wherein the image of the remote operator comprises an image of the remote operator that is performing remote operation of the vehicle. ### Claim 6 A non-transitory computer-readable storage medium storing a program that causes a computer to execute vehicle control processing, the vehicle control processing comprising: receiving a remote operation instruction for a vehicle from a remote operation terminal; receiving an image of a remote operator who performs remote operation of the vehicle using the remote operation terminal; and displaying the image of the remote operator on an external vehicle display section. ### Claim 7 (depends on 6) The non-transitory computer-readable storage medium storing the program that causes the computer to execute vehicle control processing of claim 6, wherein the image of the remote operator comprises an image of the remote operator that is performing remote operation of the vehicle. ### Claim 8 A vehicle comprising: a communication section configured to receive a remote operation instruction for the vehicle from a remote operation terminal, and an image of a remote operator remotely operating the vehicle using the remote operation terminal; an external vehicle display section configured to display an image externally to the vehicle; and a display control section configured to cause the image of the remote operator received by the communication section to be displayed on the external vehicle display section. ### Claim 9 (depends on 8) The vehicle of claim 8, further comprising a vehicle control section that causes the vehicle to travel to a predetermined location by autonomous travel when the remote operation by the remote operation terminal is ended. ### Claim 10 (depends on 8) The vehicle of claim 8, wherein the display control section is configured to terminate displaying of the image of the remote operator when the remote operation by the remote operation terminal is ended. ### Claim 11 (depends on 8) The vehicle of claim 8, wherein the image of the remote operator is acquired by an image-capture section provided to the remote operation terminal configured to remotely operate the vehicle. ### Claim 12 (depends on 8) The vehicle of claim 8, wherein the external vehicle display section is installed on a surface of a door panel of the vehicle. ### Claim 13 (depends on 8) The vehicle of claim 8, wherein the external vehicle display section is installed on a windshield or a side window of the vehicle. ### Claim 14 (depends on 8) The vehicle of claim 8, wherein the image of the remote operator comprises an image of the remote operator that is performing remote operation of the vehicle. ### Claim 15 (depends on 8) A vehicle control system comprising: the vehicle of claim 8; a first terminal configured to be used by a user; a second terminal that comprises an image-capture section and is configured to be used by the remote operator to remotely operate the vehicle; and a server that is connected to the first terminal and the second terminal so as to be in communication with the first terminal and the second terminal. ### Claim 16 (depends on 15) The vehicle control system of claim 15, wherein the server is configured to: receive a remote driving request for the vehicle from the first terminal, transmit the remote driving request to the second terminal, confer driving authority on the second terminal, and enable communication between the second terminal and the vehicle, and the second terminal is further configured to, after receiving the remote driving request and being conferred with driving authority from the server, image the remote operator using the image-capture section and transmit a captured image to the vehicle display device of the vehicle. ### Claim 17 A vehicle control method comprising: receiving a remote driving request for a vehicle from a first terminal used by a user; transmitting the remote driving request to a second terminal configured to perform remote operation of the vehicle by a remote operator, conferring driving authority on the second terminal, and enabling communication between the second terminal and the vehicle; transmitting a remote operation instruction to the vehicle from the second terminal; imaging the remote operator using an image-capture section of the second terminal, and transmitting a captured image to the vehicle; and causing a vehicle display device of the vehicle to display the received captured image on an external vehicle display section configured to display an image externally to the vehicle. ### Claim 18 A non-transitory computer-readable storage medium storing a program that causes a computer to execute vehicle control processing, the vehicle control processing comprising: receiving a remote driving request for a vehicle from a first terminal used by a user; transmitting the remote driving request to a second terminal configured to perform remote operation of the vehicle by a remote operator, conferring driving authority on the second terminal, and enabling communication between the second terminal and the vehicle; transmitting a remote operation instruction to the vehicle from the second terminal; imaging the remote operator using an image-capture section of the second terminal, and transmitting a captured image to the vehicle; and causing a vehicle display device of the vehicle to display the received captured image on an external vehicle display section configured to display an image externally to the vehicle.