# US 12,005,922 B2 · Toward simulation of driver behavior in driving automation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,005,922 B2 - Document ID: US12005922B2 - Status: Granted - Kind: B2 - Application number: 17/139,788 - Filed: 2020-12-31 - Published: 2022-06-30 - Granted: 2024-06-11 - Earliest claimed: 2020-12-31 - Assignees: Honda Motor Co., Ltd. - Applicants: Honda Motor Co., Ltd. - Inventors: Teruhisa Misu - Web page: https://patentgrep.com/patent/US12005922B2 ## CPC - B60W 60/001: {Planning or execution of driving tasks} - B60W 50/06: Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot - G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model - G06F 30/15: Vehicle, aircraft or watercraft design - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract In some examples, one or more characteristics of one or more driving scenes may be obtained. Based at least on the one or more characteristics, one or more behaviors of a simulated driver may be simulated via a machine learning model. An operation associated with one or more advanced driving assistance system (ADAS) functions may be performed based at least on the simulated one or more behaviors. ## Claims ### Claim 1 A method comprising: at an electronic device with one or more processors and memory: obtaining one or more characteristics including driver awareness characteristics of one or more driving scenes, the driver awareness characteristics indicating an awareness level of a driver towards a feature associated with driver intervention, the awareness level being based on an estimated gaze location of the driver; simulating, via a machine learning model, one or more behaviors of a simulated driver including the driver awareness characteristics, based at least on the one or more characteristics; and simulating, via a machine learning model, one or more behaviors of a simulated vehicle, based at least on the one or more characteristics; and performing an operation associated with one or more advanced driving assistance system (ADAS) functions based at least on simulated one or more behaviors comprising the one or more behaviors of the simulated driver and the one or more behaviors of the simulated vehicle. ### Claim 2 (depends on 1) The method of claim 1, wherein performing an operation associated with the one or more ADAS functions based at least on the simulated one or more behaviors comprises: improving the one or more ADAS functions based at least on the simulated one or more behaviors and the one or more characteristics of the one or more driving scenes. ### Claim 3 (depends on 2) The method of claim 2, wherein improving the one or more ADAS functions includes training one or more models associated with the one or more ADAS functions based on the simulated one or more behaviors and the one or more characteristics of the one or more driving scenes. ### Claim 4 (depends on 1) The method of claim 1, further comprising: updating one or more parameters of one or more models associated with the one or more ADAS functions based at least on one or more parameters, other than the one or more behaviors, of the machine learning model associated with the simulated driver. ### Claim 5 (depends on 1) The method of claim 1, wherein: obtaining the one or more characteristics of the one or more driving scenes includes detecting the one or more characteristics via one or more sensors of a vehicle while the vehicle is in operation, and performing the operation associated with the one or more ADAS functions comprises adjusting operation of the one or more ADAS functions of the vehicle based on the simulated one or more behaviors. ### Claim 6 (depends on 5) The method of claim 5, further comprising: determining one or more characteristics of one or more future driving scenes associated with the vehicle; and simulating, via the machine learning model, one or more future behaviors of the simulated driver using the one or more characteristics of the one or more future driving scenes, wherein adjusting the operation of the one or more ADAS functions is based on the simulated one or more future behaviors. ### Claim 7 (depends on 1) The method of claim 1, wherein the one or more driving scenes are synthetic driving scenes generated from one or more observed driving scenes. ### Claim 8 (depends on 1) The method of claim 1, wherein the machine learning model associated with the simulated driver comprises a generative adversarial network (GAN) model. ### Claim 9 (depends on 8) The method of claim 8, wherein the GAN model includes a re-constructor model configured to enforce a relationship between inputs and outputs of the GAN model. ### Claim 10 (depends on 1) The method of claim 1, wherein the one or more behaviors of the simulated driver that are simulated via the machine learning model include a score corresponding to the driver awareness characteristics of the simulated driver. ### Claim 11 (depends on 1) The method of claim 1, wherein the feature associated with driver intervention is a road object or hazard. ### Claim 12 (depends on 1) The method of claim 1, wherein the feature associated with driver intervention is an active function of one or more advanced driving assistance system (ADAS) functions. ### Claim 13 (depends on 1) The method of claim 1, wherein the one or more behaviors of the simulated driver that are simulated via the machine learning model include a likelihood of the simulated driver responding to the feature associated with driver intervention. ### Claim 14 A non-transitory computer-readable storage medium storing instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform a method comprising: obtaining one or more characteristics including driver awareness characteristics of one or more driving scenes, the driver awareness characteristics indicating an awareness level of a driver towards a feature associated with driver intervention, the awareness level being based on an estimated gaze location of the driver; simulating, via a machine learning model, one or more behaviors of a simulated driver including the driver awareness characteristics, based at least on the one or more characteristics; and simulating, via a machine learning model, one or more behaviors of a simulated vehicle, based at least on the one or more characteristics; and performing an operation associated with one or more advanced driving assistance system (ADAS) functions based at least on simulated one or more behaviors comprising the one or more behaviors of the simulated driver and the one or more behaviors of the simulated vehicle. ### Claim 15 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein performing the operation associated with the one or more ADAS functions based at least on the simulated one or more behaviors comprises: improving the one or more ADAS functions based at least on the simulated one or more behaviors and the one or more characteristics of the one or more driving scenes. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium of claim 15, wherein improving the one or more ADAS functions includes training one or more models associated with the one or more ADAS functions based on the simulated one or more behaviors and the one or more characteristics of the one or more driving scenes. ### Claim 17 (depends on 14) The non-transitory computer-readable storage medium of claim 14, the method further comprising: updating one or more parameters of one or more models associated with the one or more ADAS functions based at least on one or more parameters, other than the one or more behaviors, of the machine learning model associated with the simulated driver. ### Claim 18 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein: obtaining the one or more characteristics of the one or more driving scenes includes detecting the one or more characteristics via one or more sensors of a vehicle while the vehicle is in operation, and performing the operation associated with the one or more ADAS functions comprises adjusting operation of the one or more ADAS functions of the vehicle based on the simulated one or more behaviors. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, the method further comprising: determining one or more characteristics of one or more future driving scenes associated with the vehicle; and simulating, via the machine learning model, one or more future behaviors of the simulated driver using the one or more characteristics of the one or more future driving scenes, wherein adjusting the operation of the one or more ADAS functions is based on the simulated one or more future behaviors. ### Claim 20 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein the one or more driving scenes are synthetic driving scenes generated from one or more observed driving scenes. ### Claim 21 (depends on 14) The non-transitory computer-readable storage medium of claim 14, wherein the machine learning model associated with the simulated driver comprises a generative adversarial network (GAN) model. ### Claim 22 (depends on 21) The non-transitory computer-readable storage medium of claim 21, wherein the GAN model includes a re-constructor model configured to enforce a relationship between inputs and outputs of the GAN model. ### Claim 23 An electronic device comprising: one or more processors; and memory storing instructions, which when executed by the one or more processors, cause the electronic device to perform a method comprising: obtaining one or more characteristics including driver awareness characteristics of one or more driving scenes, the driver awareness characteristics indicating an awareness level of a driver towards a feature associated with driver intervention, the awareness level being based on an estimated gaze location of the driver; simulating, via a machine learning model, one or more behaviors of a simulated driver including the driver awareness characteristics, based at least on the one or more characteristics; and simulating, via a machine learning model, one or more behaviors of a simulated vehicle, based at least on the one or more characteristics; and performing an operation associated with one or more advanced driving assistance system (ADAS) functions based at least on simulated one or more behaviors comprising the one or more behaviors of the simulated driver and the one or more behaviors of the simulated vehicle. ### Claim 24 (depends on 23) The electronic device of claim 23, wherein performing the operation associated with the one or more ADAS functions based at least on the simulated one or more behaviors comprises: improving the one or more ADAS functions based at least on the simulated one or more behaviors and the one or more characteristics of the one or more driving scenes.