# US 12,594,959 B1 · Scene generation using probabilistic programming languages > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,594,959 B1 - Document ID: US12594959B1 - Status: Granted - Kind: B1 - Application number: 17/349,448 - Filed: 2021-06-16 - Granted: 2026-04-07 - Earliest claimed: 2021-06-16 - Assignees: Waymo LLC - Applicants: Waymo LLC - Inventors: Congrui Hetang - Web page: https://patentgrep.com/patent/US12594959B1 ## CPC - B60W 60/0011: {involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles} - B60W 40/04: Traffic conditions - B60W 40/06: Road conditions - B60W 60/001: {Planning or execution of driving tasks} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N 20/00: Machine learning - B60W 2552/00: Input parameters relating to infrastructure - B60W 2554/4041: Position - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A system includes a memory device, and a processing device, operatively coupled to the memory device, to receive a set of input data including a roadgraph and a distribution of a plurality of scene types associated with a driving context, the roadgraph including an autonomous vehicle driving path, perform stochastic sampling based on the distribution of the plurality of scene types to obtain a set of synthetic scenes for the driving context, and train a machine learning model to identify, for each synthetic scene of the set of synthetic scenes, a respective modified autonomous vehicle driving path. ## Claims ### Claim 1 A system comprising: a memory device; and a processing device, operatively coupled to the memory device, to: receive a set of input data including a roadgraph and a first distribution of a plurality of scene types associated with a driving context, the roadgraph representing an autonomous vehicle driving path, and the first distribution of the plurality of scene types comprising the plurality of scene types each associated with a weight indicating a probability of an occurrence of a respective scene type; generate a set of synthetic scenes for the driving context by performing stochastic sampling based on the first distribution of the plurality of scene types, wherein the processing device is to place a set of artifacts to define a synthetic scene within the roadgraph based on a second distribution of a variable associated with requirements for the driving context; train a machine learning model to identify, for each synthetic scene of the set of synthetic scenes, a respective modified autonomous vehicle driving path to be navigated by an autonomous vehicle; and cause an autonomous vehicle control system to modify, using the trained machine learning model, the autonomous driving path. ### Claim 2 (depends on 1) The system of claim 1, wherein each synthetic scene of the set of synthetic scenes is a synthetic construction zone. ### Claim 3 (depends on 1) The system of claim 1, wherein each respective modified autonomous vehicle path includes at least one of: a path shift, or a path merge into a second autonomous vehicle driving path represented by the roadgraph. ### Claim 4 (depends on 1) The system of claim 1, wherein the first distribution of the plurality of scene types comprises the plurality of scene types each associated with the weight indicating the probability of occurrence of the respective scene type under realistic driving conditions. ### Claim 5 (depends on 1) The system of claim 1, wherein the second distribution follows a probability density function. ### Claim 6 (depends on 1) The system of claim 1, wherein the stochastic sampling is performed by a probabilistic programming language module (PPLM) written in a probabilistic programming language and nested within a program written in a general purpose programming language. ### Claim 7 (depends on 1) The system of claim 1, wherein the set of synthetic scenes comprises a plurality of synthetic scenes, each synthetic scene of the plurality of synthetic scenes being associated with an intraclass variety. ### Claim 8 (depends on 1) The system of claim 1, wherein the processing device is further to: generate a set of training input data comprising a set of data frames from the set of synthetic scenes; and obtain a set of target output data for the set of training input data, wherein the machine learning model is trained using the set of training input data and the set of target output data. ### Claim 9 (depends on 8) The system of claim 8, wherein the set of target output data comprises at least one of: messages with injected markers or perception objects, or tensorflow examples. ### Claim 10 A method comprising: receiving, by a processing device, a set of input data including a roadgraph and a first distribution of a plurality of scene types associated with a driving context, the roadgraph representing an autonomous vehicle driving path, and the first distribution of the plurality of scene types comprising the plurality of scene types each associated with a weight indicating a probability of an occurrence of a respective scene type; generating, by the processing device, a set of synthetic scenes for the driving context by performing stochastic sampling based on the first distribution of the plurality of scene types, wherein the processing device is to place a set of artifacts to define a synthetic scene within the roadgraph based on a second distribution of a variable associated with requirements for the driving context; training, by the processing device, a machine learning model to identify, for each synthetic scene of the set of synthetic scenes, a respective modified autonomous vehicle driving path to be navigated by an autonomous vehicle; and causing an autonomous vehicle control system to modify, using the trained machine learning model, the autonomous driving path. ### Claim 11 (depends on 10) The method of claim 10, wherein each synthetic scene of the set of synthetic scenes is a synthetic construction zone. ### Claim 12 (depends on 10) The method of claim 10, wherein each respective modified autonomous vehicle path includes at least one of: a path shift, or a path merge into a second autonomous vehicle driving path represented by the roadgraph. ### Claim 13 (depends on 10) The method of claim 10, wherein the first distribution of the plurality of scene types comprises the plurality of scene types each associated with the weight indicating the probability of occurrence of the respective scene type under realistic driving conditions. ### Claim 14 (depends on 10) The method of claim 10, wherein the second distribution follows a probability density function. ### Claim 15 (depends on 10) The method of claim 10, wherein the stochastic sampling is performed by a probabilistic programming language module (PPLM) written in a probabilistic programming language and nested within a program written in a general purpose programming language. ### Claim 16 (depends on 10) The method of claim 10, wherein the set of synthetic scenes comprises a plurality of synthetic scenes, each synthetic scene of the plurality of synthetic scenes being associated with an intraclass variety. ### Claim 17 (depends on 10) The method of claim 10, further comprising: generating, by the processing device, a set of training input data comprising a set of data frames from the set of synthetic scenes; and obtaining a set of target output data for the set of training input data, wherein the machine learning model is trained using the set of training input data and the set of target output data. ### Claim 18 (depends on 17) The method of claim 17, wherein the set of target output data comprises at least one of: messages with injected markers or perception objects, or tensorflow examples. ### Claim 19 A non-transitory computer-readable storage medium having instructions stored thereon that, when executed by a processing device, cause the processing device to: receive a set of input data including a roadgraph and a first distribution of a plurality of scene types associated with a driving context, the roadgraph representing an autonomous vehicle driving path, and the first distribution of the plurality of scene types comprising the plurality of scene types each associated with a weight indicating a probability of an occurrence of a respective scene type; generate a set of synthetic scenes for the driving context by performing stochastic sampling based on the first distribution of the plurality of scene types, wherein the processing device is to place a set of artifacts to define a synthetic scene within the roadgraph based on a second distribution of a variable associated with requirements for the driving context; train a machine learning model to identify, for each synthetic scene of the set of synthetic scenes, a respective modified autonomous vehicle driving path to be navigated by an autonomous vehicle; and cause an autonomous vehicle control system to modify, using the trained machine learning model, the autonomous driving path. ### Claim 20 (depends on 19) The non-transitory computer-readable storage medium of claim 19, wherein each synthetic scene of the set of synthetic scenes is a synthetic construction zone.