Safe and secure end-to-end autonomous driving system

A processor-implemented method for autonomous driving, the processor-implemented method comprising:

• obtaining sensor data associated with an external environment of a vehicle;
• selecting, via a manager, a set of prompts for input to a machine learning model trained to assist with autonomous driving for the vehicle, based at least in part on the sensor data and on at least one of (i) an inference time of the trained machine learning model or (ii) one or more hardware metrics for the vehicle;
• generating, using the trained machine learning model and the set of prompts, a set of output tokens corresponding to the set of prompts, the set of output tokens being associated with at least one of (i) one or more adversarial objects in the environment or (ii) one or more safety events in the environment;
• outputting, via the manager, information associated with the set of output tokens for a user of the vehicle;
• receiving, via the manager, feedback from the user regarding the information associated with the set of output tokens;
• translating, via the manager, the feedback into an additional prompt for input to the trained machine learning model;
• and
• generating, using the trained machine learning model and the additional prompt, an additional set of output tokens.

A processing system comprising:

• means for obtaining sensor data associated with an external environment of a vehicle;
• means for selecting, via a manager, a set of prompts for input to a machine learning model trained to assist with autonomous driving for the vehicle, based at least in part on the sensor data and on at least one of (i) an inference time of the trained machine learning model or (ii) one or more hardware metrics for the vehicle;
• means for generating, using the trained machine learning model and the set of prompts, a set of output tokens corresponding to the set of prompts, the set of output tokens being associated with at least one of (i) one or more adversarial objects in the environment or (ii) one or more safety events in the environment;
• means for outputting, via the manager, information associated with the set of output tokens for a user of the vehicle;
• means for receiving, via the manager, feedback from the user regarding the information associated with the set of output tokens;
• means for translating, via the manager, the feedback into an additional prompt for input to the trained machine learning model;
• and
• means for generating, using the trained machine learning model and the additional prompt. an additional set of output tokens.