Back to results
US 11,993,291 B2Granted
Neural networks for navigation of autonomous vehicles based upon predicted human intents
AssigneePerceptive Automata, Inc.
InventorMel McCurrie
Life of this patent
Estimating the term…
Abstract
A system uses neural networks to determine intents of traffic entities (e.g., pedestrians, bicycles, vehicles) in an environment surrounding a vehicle (e.g., an autonomous vehicle) and generates commands to control the vehicle based on the determined intents. The system receives images of the environment captured by sensors on the vehicle, and processes the images using neural network models to determine overall intents or predicted actions of the one or more traffic entities within the images. The system generates commands to control the vehicle based on the determined overall intents of the traffic entities.
Claims18
A method comprising:
- receiving a plurality of images, each image corresponding to a video frame captured by one or more sensors of a vehicle moving through traffic;
- processing the plurality of images using a first neural network model configured to generate, for a first image of the plurality of images, a first feature map indicating, for each pixel of the first image, a feature vector corresponding to an intent associated with the pixel;
- using a second neural network model, identifying one or more traffic entities within the first image based upon the first image and the first feature map of the first image, wherein a traffic entity represents an object moving through the traffic;
- determining, for each of the identified one or more traffic entities, an overall intent of the traffic entity, based upon an aggregation of the feature vectors corresponding to pixels encompassed by the traffic entity, wherein the overall intent represents an action that the traffic entity is expected to perform in the traffic;
- and
- generating one or more commands to control the vehicle based upon the determined overall intents of the one or more traffic entities.
3 more elements
A non-transitory computer readable medium storing instructions that when executed by a processor cause the processor to perform steps comprising:
- receiving a plurality of images, each image corresponding to a video frame captured by one or more sensors of a vehicle moving through traffic;
- processing the plurality of images using a first neural network model configured to generate, for a first image of the plurality of images, a first feature map indicating, for each pixel of the first image, a feature vector corresponding to an intent associated with the pixel;
- using a second neural network model, identifying one or more traffic entities within the first image based upon the first image and the first feature map of the first image, wherein a traffic entity represents an object moving through the traffic;
- determining, for each of the identified one or more traffic entities, an overall intent of the traffic entity, based upon an aggregation of the feature vectors corresponding to pixels encompassed by the traffic entity, wherein the overall intent represents an action that the traffic entity is expected to perform in the traffic;
- and
- generating one or more commands to control the vehicle based upon the determined overall intents of the one or more traffic entities.
3 more elements
A system comprising:
- a hardware processor;
- and
- a non-transitory computer readable medium storing instructions that when executed by the hardware processor cause the processor to perform steps comprising:
- receiving a plurality of images, each image corresponding to a video frame captured by one or more sensors of a vehicle moving through traffic;
- processing the plurality of images using a first neural network model configured to generate, for a first image of the plurality of images, a first feature map indicating, for each pixel of the first image, a feature vector corresponding to an intent associated with the pixel;
- using a second neural network model, identifying one or more traffic entities within the first image based upon the first image and the first feature map of the first image, wherein a traffic entity represents an object moving through the traffic;
- determining, for each of the identified one or more traffic entities, an overall intent of the traffic entity, based upon an aggregation of the feature vectors corresponding to pixels encompassed by the traffic entity, wherein the overall intent represents an action that the traffic entity is expected to perform in the traffic;
- and
- generating one or more commands to control the vehicle based upon the determined overall intents of the one or more traffic entities.
6 more elements
Dependencies are inferred from the claim text.