# US 11,307,597 B2 · Tele-operation of autonomous cars to negotiate problem situations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,597 B2 - Document ID: US11307597B2 - Status: Granted - Kind: B2 - Application number: 16/890,506 - Filed: 2020-06-02 - Published: 2020-09-17 - Granted: 2022-04-19 - Earliest claimed: 2016-11-30 - Assignees: Nissan North America, Inc.; United States of America as represented by the Administrator of NASA - Applicants: Nissan North America, Inc.; United States of America as Represented by the Administrator of NASA - Inventors: Liam Pedersen; Maarten Sierhuis; Hans Utz; Mauro Della Penna; Terrence Fong; Mark Allan; Maria Bualat; Eric Schafer - Web page: https://patentgrep.com/patent/US11307597B2 ## CPC - G05D 1/0027: {involving a plurality of vehicles, e.g. fleet or convoy travelling (fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00)} - G05D 1/0291: {Fleet control (monitoring fleets in traffic control systems for road vehicles G08G1/127, G08G1/127)} - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - G05D 1/0212: {with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287)} - G05D 1/0276: {using signals provided by a source external to the vehicle (involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058)} - G05D 1/0287: {involving a plurality of land vehicles, e.g. fleet or convoy travelling (traffic control systems for road vehicles G08G1/00, particularly anticollision systems G08G1/16)} - G05D 1/0297: {by controlling means in a control room} - G06Q 50/40: Business processes related to the transportation industry (shipping G06Q10/083) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract Methods and systems for providing remote support and negotiating problem situations of autonomous operation of vehicles based on signal states and vehicle information are described. A system comprises a memory and a processor configured to execute instructions stored in the memory to: assign vehicles to support queues based on state data, generate a map display including locations of the vehicles, and generate a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data. ## Claims ### Claim 1 A system, comprising: a memory; and a processor configured to execute instructions stored in the memory to: assign vehicles to support queues based on state data, generate a map display including locations of the vehicles, and generate a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data. ### Claim 2 (depends on 1) The system of claim 1, wherein the processor is further configured to execute instructions stored in the memory to: receive a request from a vehicle for remote support; and transmit instruction data to the vehicle to provide the remote support. ### Claim 3 (depends on 1) The system of claim 1, wherein the processor is further configured to execute instructions stored in the memory to: responsive to autonomous operation of a vehicle operating outside of defined parameter values, transmit instruction data to the vehicle to modify the autonomous operation of the vehicle. ### Claim 4 (depends on 1) The system of claim 1, wherein the support queues comprise first level control stations, each first level control station receiving the state data for different vehicles. ### Claim 5 (depends on 4) The system of claim 4, wherein the vehicles are assigned to the first level control stations using a second level control station. ### Claim 6 (depends on 1) The system of claim 1, wherein the state data includes any of kinetic state data and location data for the vehicles. ### Claim 7 (depends on 1) The system of claim 1, wherein the processor is further configured to execute instructions stored in the memory to: responsive to an input signal to a first state indicator representing state data of a vehicle, generate a remote support interface display including image data associated with the vehicle. ### Claim 8 A method, comprising: assigning vehicles to support queues based on state data; generating a map display including locations of the vehicles; generating a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data; and tracking autonomous operation of the vehicles. ### Claim 9 (depends on 8) The method of claim 8, further comprising: receiving a request from a vehicle for remote support; and transmitting instruction data to the vehicle to provide the remote support. ### Claim 10 (depends on 8) The method of claim 8, further comprising: responsive to autonomous operation of a vehicle operating outside of defined parameter values, transmitting instruction data to the vehicle to modify the autonomous operation of the vehicle. ### Claim 11 (depends on 10) The method of claim 10, wherein the support queues comprise first level control stations, each first level control station receiving the state data for different vehicles. ### Claim 12 (depends on 11) The method of claim 11, further comprising: prior to transmitting the instruction data to the vehicle, assigning, by a second level control station, the vehicle from a first control station to a second control station based on a change in the state data for the vehicle. ### Claim 13 (depends on 8) The method of claim 8, further comprising: responsive to an input signal to a first state indicator representing state data of a vehicle, generating a remote support interface display including image data associated with the vehicle. ### Claim 14 (depends on 13) The method of claim 13, wherein the image data comprises any of an image of an individual within the vehicle and an image of an environment around the vehicle. ### Claim 15 (depends on 13) The method of claim 13, wherein the image data comprises an image of an obstruction in proximity to the vehicle, the method further comprising: determining a new route that avoids the obstruction and deviates from a current route; and transmitting the new route to the vehicle to modify autonomous operation of the vehicle. ### Claim 16 (depends on 15) The method of claim 15, further comprising: analyzing the state data of at least one of the vehicles other than the vehicle in proximity to the obstruction; and transmitting instruction data to the at least one of the vehicles to modify the autonomous operation of the at least one of the vehicles. ### Claim 17 (depends on 8) The method of claim 8, further comprising: ranking the vehicles based on the state data; and generating the state display by assigning each of the vehicles to a respective one of the support queues based on the ranking. ### Claim 18 (depends on 8) The method of claim 8, further comprising: assigning at least one vehicle from one of the support queues that is determined as being over-tasked to another of the support queues that is determined as not being over-tasked. ### Claim 19 (depends on 8) The method of claim 8, wherein the state data includes an issue state, the issue state indicating any of a normal state, an issue with an operator, an issue with a passenger, an issue with traffic, an issue with an external environment, a physical issue of the vehicle, and an issue related to a decision based on autonomous operation of the vehicle. ### Claim 20 A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium includes executable instructions that, when executed by a processor, facilitate performance of operations, the operations comprising: assigning vehicles to support queues based on state data; generating a map display including locations of the vehicles; generating a state display including the support queues, vehicle manager indicators corresponding to the support queues and state indicators corresponding to the state data; and tracking autonomous operation of the vehicles.