Back to results
US 11,537,134 B1Granted
Generating environmental input encoding for training neural networks
AssigneeApple Inc.
InventorsJuergen Wiest, Moises Goldszmidt, Tobias Martin Gindele
Life of this patent
Estimating the term…
Abstract
An encoding of an environment for operating vehicles is obtained, comprising a combination of at least a representation of moving entities with a graph representation of static infrastructure elements. Using the encoding and a set of one or more observations of the environment state, a machine learning model is trained to produce a probabilistic representation of a set of predicted states of the environment. A trained version of the machine learning model is stored and deployed at one or more vehicles to help plan and control vehicle movements.
Claims20
A method, comprising:
- performing, at one or more computing devices:
- obtaining a first encoding of a vehicle operation environment, wherein the first encoding comprises a combination of:
- (a) a representation of a plurality of moving entities,
- (b) an infrastructure graph, distinct from the representation of the plurality of moving entities, wherein the infrastructure graph comprises a plurality of nodes representing respective static infrastructure components and one or more edges linking the nodes, wherein a particular edge of the one or more edges indicates a relationship between a pair of nodes of the plurality of nodes, and
- (c) an occupancy grid map;
- training, using at least the first encoding, a first machine learning model to produce a first probabilistic representation of a set of predicted states of the environment, wherein training the first machine learning model comprises:
- (a) a first phase of training using the infrastructure graph, wherein at least a portion of the first machine learning model is trained during the first phase to learn representations of one or more infrastructure components;
- and
- (b) a second phase of training, using at least the portion of the first machine learning model resulting from the first phase, to train the first machine learning model to predict actions of the plurality of moving entities using the representation of the plurality of moving entities;
- and
- storing a trained version of the first machine learning model, wherein output produced by the trained version is usable by one or more motion decision making components of a vehicle to generate motion control directives of the vehicle.
8 more elements
A system, comprising:
- a machine learning model trainer comprising one or more computing devices;
- wherein the machine learning model trainer is configured to:
- obtain a first encoding of a vehicle operation environment, wherein the first encoding comprises a combination of:
- (a) a representation of a plurality of moving entities,
- (b) an infrastructure graph, distinct from the representation of the plurality of moving entities, wherein the infrastructure graph comprises a plurality of nodes representing respective infrastructure components and one or more edges linking the nodes, wherein a particular edge of the one or more edges indicates a relationship between a pair of nodes of the plurality of nodes, and
- (c) an occupancy grid map;
- train, using at least the first encoding, a first machine learning model to produce a first probabilistic representation of a set of predicted states of the environment, wherein training the first machine learning model comprises:
- (a) a first phase of training using the infrastructure graph, wherein at least a portion of the first machine learning model is trained during the first phase to learn representations of one or more infrastructure components;
- and
- (b) a second phase of training, using at least the portion of the first machine learning model resulting from the first phase, to train the first machine learning model to predict actions of the plurality of moving entities using the representation of the plurality of moving entities;
- and
- store a trained version of the first machine learning model, wherein output produced by the trained version is usable by one or more motion decision making components of a vehicle to generate motion control directives of the vehicle.
9 more elements
A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors cause the one or more processors to:
- obtain a first encoding of a vehicle operation environment, wherein the first encoding comprises a combination of:
- a representation of a plurality of moving entities,
- an infrastructure graph, wherein the infrastructure graph comprises a plurality of nodes representing respective infrastructure components and one or more edges linking the nodes, and wherein a particular edge of the one or more edges indicates a relationship between a pair of nodes of the plurality of nodes, and
- an occupancy grid map;
- train, using at least the first encoding, a first machine learning model to produce a first probabilistic representation of a set of predicted states of the environment, wherein training the first machine learning model comprises:
- (a) a first phase of training using the infrastructure graph, wherein at least a portion of the first machine learning model is trained during the first phase to learn representations of one or more infrastructure components;
- and
- (b) a second phase of training, using at least the portion of the first machine learning model resulting from the first phase, to train the first machine learning model to predict actions of the plurality of moving entities using the representation of the plurality of moving entities;
- and
- store a trained version of the first machine learning model, wherein output produced by the trained version is usable by one or more motion decision making components of a vehicle to generate motion control directives of the vehicle.
7 more elements
Dependencies are inferred from the claim text.