Vehicle placement on aerial views for vehicle control

A method for at least partially autonomously driving a vehicle that includes one or more processors and memory, the method comprising:

• obtaining a forward-facing view of a road captured by a front-facing camera of the vehicle, the forward-facing view of the road including one or more obstacle vehicles;
• applying a machine learning model to process the forward-facing view to determine a first location of each obstacle vehicle in a Frenet-Serret coordinate system of the road;
• obtaining an aerial view of the road;
• placing the one or more obstacle vehicles on the aerial view of the road;
• and
• measuring, by an on-vehicle sensor, a relative location of each obstacle vehicle with reference to the ego vehicle;
• and
• modifying the vehicle location of each obstacle vehicle based on the measured relative location;
• displaying the aerial view of the road on a display of the ego vehicle;
• and
• visualizing the one or more obstacle vehicles on the aerial view, further comprising, for each obstacle vehicle:
• determining a width or a height of the obstacle vehicle on the forward-facing view;
• estimating a length of the obstacle vehicle based on the width or height of the obstacle vehicle;
• and
• visualizing the obstacle vehicle at the vehicle location using the width and length of the obstacle vehicle;
• using the aerial view of the road to at least partially autonomously drive the vehicle.

A vehicle, comprising:

• one or more processors;
• and
• memory storing one or more programs configured for execution by the one or more processors, the one or more programs comprising instructions for:
• obtaining a forward-facing view of a road captured by a front-facing camera of the vehicle, the forward-facing view of the road including one or more obstacle vehicles;
• applying a machine learning model to process the forward-facing view to determine a first location of each obstacle vehicle in a Frenet-Serret coordinate system of the road, wherein determining the first location of each obstacle vehicle includes:
• determining a longitudinal distance from a starting point of a respective lane to the obstacle vehicle along the respective lane;
• and
• determining a lateral distance from a lane line of the respective lane to the obstacle vehicle along a direction substantially perpendicular to the respective lane;
• obtaining an aerial view of the road;
• placing the one or more obstacle vehicles on the aerial view of the road;
• and
• using the aerial view of the road to at least partially autonomously drive the vehicle.

A vehicle, comprising:

• one or more processors;
• and
• memory storing one or more programs configured for execution by the one or more processors, the one or more programs comprising instructions for:
• obtaining a forward-facing view of a road captured by a front-facing camera of the vehicle, the forward-facing view of the road including one or more obstacle vehicles;
• applying a machine learning model to process the forward-facing view to determine a first location of each obstacle vehicle in a Frenet-Serret coordinate system of the road, further comprising, for each obstacle vehicle:
• determining a longitudinal distance and a lateral distance of the respective obstacle vehicle with reference to an origin of the Frenet-Serret coordinate system, wherein the vehicle includes an ego vehicle, and the origin of the Frenet-Serret coordinate system is located substantially at the front-facing camera of the ego vehicle when the forward-facing view of the road is captured;
• obtaining an aerial view of the road;
• placing the one or more obstacle vehicles on the aerial view of the road;
• in accordance with a determination that the longitudinal distance of a first obstacle vehicle is less than a braking distance and a determination that the lateral distance of the first obstacle vehicle decreases within a lane cutting range, issuing a braking control command to control the vehicle that captured the forward-facing view to operate in a braking mode;
• and
• using the aerial view of the road to at least partially autonomously drive the vehicle in the braking mode.

A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of a vehicle, the one or more programs comprising instructions for:

• obtaining a forward-facing view of a road captured by a front-facing camera of the vehicle, the forward-facing view of the road including one or more obstacle vehicles;
• applying a machine learning model to process the forward-facing view to determine a first location of each obstacle vehicle in a Frenet-Serret coordinate system of the road, wherein determining the first location of each obstacle vehicle includes:
• determining a longitudinal distance from a starting point of a respective lane to the obstacle vehicle along the respective lane;
• and
• determining a lateral distance from a lane line of the respective lane to the obstacle vehicle along a direction substantially perpendicular to the respective lane;
• obtaining an aerial view of the road;
• placing the one or more obstacle vehicles on the aerial view of the road;
• and
• using the aerial view of the road to at least partially autonomously drive the vehicle.