Inferring intent using computer vision
AssigneePony AI Inc.
InventorsKevin Sheu, Jie Mao
Life of this patent
Estimating the term…
Abstract
A system trains a model to infer an intent of an entity. The model includes one or more sensors to obtain frames of data, one or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the system to perform steps. A first step includes determining, in each frame of the frames, one or more bounding regions, each of the bounding regions enclosing an entity. A second step includes identifying a common entity, the common entity being present in bounding regions corresponding to a plurality of the frames. A third step includes associating the common entity across the frames. A fourth step includes training a model to infer an intent of the common entity based on data outside of the bounding regions.
Claims12
A system configured to train a model to infer an intent of an entity, comprising:
- one or more sensors configured to obtain frames of data;
- one or more processors;
- and
- a memory storing instructions that, when executed by the one or more processors, cause the system to perform:
- determining, in each frame of the frames, one or more bounding regions, each of the bounding regions enclosing an entity;
- and
- inferring, based on a trained model, an intent associated with the entity based on data outside of the bounding regions, wherein inferring an intent is based on semantic segmentation to predict a category or a classification associated with one or more pixels of the frames and based on instance segmentation to predict whether two pixels associated with a common category or a common classification belong to same or different instances.
A method comprising:
- obtaining, using one or more sensors, frames of data;
- determining, in each frame of the frames, one or more bounding regions, each of the bounding regions enclosing an entity;
- and
- inferring, based on a trained model, an intent associated with the entity based on data outside of the bounding regions, wherein inferring an intent is based on semantic segmentation to predict a category or a classification associated with one or more pixels of the frames and based on instance segmentation to predict whether two pixels associated with a common category or a common classification belong to same or different instances.
A system configured to train a model to infer an intent of an entity, comprising:
- one or more sensors configured to obtain frames of data;
- one or more processors;
- and
- a memory storing instructions that, when executed by the one or more processors, cause the system to perform:
- determining, in each frame of the frames, one or more bounding regions, each of the bounding regions enclosing an entity;
- and
- inferring, based on a trained model, an intent associated with the entity based on data outside of the bounding regions, wherein inferring an intent is based on a probability of a left turn signal of a vehicle being on, a probability of a right turn signal of the vehicle being on, and a probability of a brake light of the vehicle being on, wherein the trained model is trained based on cross entropy losses over the inferred intent, over left or right turn signals of the vehicle, and over the vehicle.
Dependencies are inferred from the claim text.