# US 12,005,913 B2 · Method of directly controlling autonomous vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,005,913 B2 - Document ID: US12005913B2 - Status: Granted - Kind: B2 - Application number: 17/396,489 - Filed: 2021-08-06 - Published: 2022-06-16 - Granted: 2024-06-11 - Earliest claimed: 2020-12-14 - Assignees: Hyundai Motor Company; Kia Corporation - Applicants: Hyundai Motor Company; Kia Corporation - Inventors: Myung-Ki Yeom; In-Su Kim - Web page: https://patentgrep.com/patent/US12005913B2 ## CPC - B60W 50/10: Interpretation of driver requests or demands - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/22: Display screens - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - B60W 60/0053: {from vehicle to occupant} - B60W 60/0054: {Selection of occupant to assume driving tasks} - B60K 35/20: Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor - B60K 2360/146: Instrument input by gesture - B60K 2360/151: Instrument output devices for configurable output - B60K 2360/175: Autonomous driving - B60K 2360/178: Warnings - B60K 2360/29: Holographic features - B60W 2050/146: {Display means} - B60W 2540/045: Occupant permissions - B60W 2540/12: Brake pedal position - B60W 2540/215: Selection or confirmation of options - B60W 2540/227: Position in the vehicle - B60W 2720/10: Longitudinal speed - B60W 2720/12: Lateral speed - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES ## Abstract A method of controlling vehicle driving, when a plurality of occupants is in the vehicle, by implementing components required for vehicle driving control as holographic images to ensure a maximum space in an autonomous vehicle, may include steps of requesting, by a first occupant currently having no driving control authority, a transfer of the authority by use of a holographic image, accepting, by a second occupant currently having the driving control authority, the transfer of the authority by use of the holographic image, selecting, by the first occupant, a mode by use of the holographic image based on the acceptance by the second occupant, and performing vehicle control in accordance with the mode selected by the first occupant, and the holographic images in the respective steps are displayed to the occupants wearing an HMD device. ## Claims ### Claim 1 A method of directly controlling an autonomous vehicle, the method comprising steps of: determining, by an autonomous vehicle control unit, whether there is a request of a first occupant currently having no driving control authority for the autonomous vehicle, for a transfer of the driving control authority by selecting a holographic image for requesting the transfer of the driving control authority; determining, by the autonomous vehicle control unit, whether there is acceptance of a second occupant currently having the driving control authority, for the request of the first occupant by selecting a holographic image for accepting the transfer of the driving control authority when the second occupant receives the request for the transfer of the driving control authority from the first occupant; determining, by the autonomous vehicle control unit, a vehicle control mode including a manual driving mode or an autonomous driving mode, in accordance with the first occupant's selecting the holographic image displaying the manual driving mode or the autonomous driving mode according to the acceptance by the second occupant; performing, by the autonomous vehicle control unit, in accordance with the vehicle control mode selected by the first occupant, driving control of the autonomous vehicle by displaying a holographic image required for the driving control of the autonomous vehicle, changing the holographic image in a predetermined pattern in accordance with a gesture of the first occupant, and performing, by the autonomous vehicle control unit, the driving control in accordance with the gesture of the first occupant; wherein the holographic images in the respective steps are implemented by a hologram device connected to the autonomous vehicle control unit required for the driving control, and the holographic images are displayed to the first and second occupants. ### Claim 2 (depends on 1) The method of claim 1, further including: verifying, by the autonomous vehicle control unit, whether the first occupant has a driving control ability, when the transfer of the driving control authority is requested by the first occupant. ### Claim 3 (depends on 1) The method of claim 1, wherein the holographic image for requesting the transfer of the driving control authority and the holographic image for accepting the transfer of the driving control authorityare movable from an initial position spaced apart, with a predetermined distance, from an occupant authorized to select the holographic image to a final position which is a position at which the authorized occupant is able to select the holographic image. ### Claim 4 (depends on 1) The method of claim 1, further including: determining, by the autonomous vehicle control unit, whether the first occupant is able to perform manual driving of the autonomous vehicle when the first occupant selects the manual driving mode. ### Claim 5 (depends on 1) The method of claim 1, further including: maintaining, by the autonomous vehicle control unit, a previous vehicle control mode when the first occupant does not select the vehicle control mode. ### Claim 6 (depends on 1) The method of claim 1, wherein the holographic image required for the driving control includes at least one of holographic images related to steering, acceleration, and braking of the autonomous vehicle. ### Claim 7 (depends on 1) The method of claim 1, further including: determining, by the autonomous vehicle control unit, whether in the autonomous driving mode, a follow-me mode in which the holographic image required for the driving control follows a position of the first occupant or a stationary mode in which the holographic image is fixed at a predetermined position in the autonomous vehicle, is selected. ### Claim 8 (depends on 7) The method of claim 7, further including: identifying, by the autonomous vehicle control unit, a position of the first occupant in the follow-me mode, and moving, by the autonomous vehicle control unit, an image of a brake button in accordance with the gesture of the first occupant, wherein the holographic image required for the driving control includes the image of the brake button. ### Claim 9 (depends on 7) The method of claim 7, further including: identifying, by the autonomous vehicle control unit, a position of the first occupant in the stationary mode; and fixing, by the autonomous vehicle control unit, an image of a brake button at a predetermined position, wherein the holographic image required for the driving control includes the image of the brake button. ### Claim 10 (depends on 7) The method of claim 7, further including: determining, by the autonomous vehicle control unit, whether the first occupant has selected an image of a brake button; and displaying, by the autonomous vehicle control unit, an amount to which the brake button is pushed in a holographic image indicating the amount to which the brake button is pushed, when the brake button is selected, wherein the holographic image required for the driving control includes the image of the brake button. ### Claim 11 (depends on 1) The method of claim 1, further including: performing, by the autonomous vehicle control unit in the autonomous driving mode, braking control or adjustment of vehicle speed in accordance with a magnitude of a request for braking of the autonomous vehicle. ### Claim 12 (depends on 11) The method of claim 11, further including: tightening, by the autonomous vehicle control unit in the autonomous driving mode, a safety belt for the first occupant when all occupants are seated in the autonomous vehicle and the request for braking is greater than a first threshold. ### Claim 13 (depends on 12) The method of claim 12, wherein when the request for braking is smaller than a second threshold, the autonomous driving control unit adjusts the vehicle speed in accordance with the request for braking, and wherein the second threshold is smaller than the first threshold. ### Claim 14 (depends on 11) The method of claim 11, further including: outputting, by the autonomous vehicle control unit in the autonomous driving mode, a warning signal when a predetermined number of all occupants are not seated in the autonomous vehicle and the request for braking is greater than a second threshold. ### Claim 15 (depends on 14) The method of claim 14, wherein when the request for braking is smaller than the second threshold, the autonomous driving control unit adjusts the vehicle speed in accordance with the request for braking. ### Claim 16 (depends on 11) The method of claim 11, further including: outputting, by the autonomous vehicle control unit in the autonomous driving mode, a warning signal when a predetermined number of all occupants are not seated in the autonomous vehicle and the request for braking is greater than a second threshold. ### Claim 17 (depends on 1) The method of claim 1, wherein the autonomous vehicle control unit includes: a processor; and a non-transitory storage medium on which a program for performing the method of claim 1 is recorded and executed by the processor. ### Claim 18 (depends on 1) A non-transitory computer readable storage medium storing a program for performing the method of claim 1 and executed by a processor.