Systems and methods for performing commands in a vehicle using speech and image recognition

A method comprising:

• receiving, from one or more first sensors, data indicative of a voice command corresponding to a vehicular operation and a reference to a first occupant within an interior of a vehicle;
• receiving, from one or more second sensors, data indicative of an image of the interior of the vehicle;
• locating, using processing circuitry that implements at least one neural network, a positional region of the first occupant within the interior of the vehicle based at least in part on the image;
• retrieving, from a database, authentication data for a second occupant that issued the voice command;
• determining, based at least in part on the authentication data, the data indicative of the voice command, and the positional region of the first occupant, a component of the vehicle that the vehicular operation corresponds to;
• and
• causing the vehicular operation to be performed with respect to the component of the vehicle.

A system comprising:

• one or more first sensors configured to generate first sensor data indicative of a voice command from a first occupant in a vehicle, the voice command corresponding to a vehicular operation;
• one or more second sensors to generate second sensor data representative of one or more images of one or more fields of view of the one or more second sensors;
• and
• parallel processing circuitry configured to:
• determine, based at least in part on the voice command, that the voice command includes a reference to a second occupant of the vehicle;
• determine, using a neural network and based at least in part on the second sensor data, a position of the second occupant within the vehicle;
• determine, based at least in part on the position of the second occupant within the vehicle, a component associated with the vehicular operation;
• retrieve, from a database, authentication data for the first occupant;
• authenticate the first occupant to authorize the vehicular operation corresponding to the voice command;
• and
• responsive to the authentication, authorize the vehicular operation to be performed with respect to the component of the vehicle.

A system comprising:

• one or more sensors configured to:
• receive first sensor data indicative of an image of an interior of a vehicle;
• and
• receive second sensor data indicative of a voice command corresponding to a vehicular operation;
• and
• parallel processing circuitry configured to:
• determine, based at least in part on a neural network, that the image depicts an occupant and a component of the vehicle;
• determine, based at least in part on the image depicting the occupant and the component, that the component is associated with the vehicular operation;
• and
• based at least in part on the component being associated with the vehicular operation, cause the vehicular operation to be performed with respect to the component.

A non-transitory computer readable medium having instructions encoded thereon, that when executed by processing circuitry causes the processing circuitry to:

• receive, from one or more first sensors, first sensor data indicative of a voice command corresponding to a vehicular operation of a vehicle;
• determine, based at least in part on the voice command, an occupant that the voice command corresponds to;
• receive, from one or more second sensors, second sensor data representative of one or more images of one or more fields of view of the one or more second sensors;
• determine, using a neural network that the second sensor data represents the occupant located proximate to a component;
• determine, based at least in part on the second sensor data representing the occupant located proximate to the component, that the component is associated with the vehicular operation;
• and
• based at least in part on the component being associated with the vehicular operation, authorize the vehicular operation to be performed with respect to the component.

A non-transitory computer readable medium having instructions encoded thereon, that when executed by processing circuitry causes the processing circuitry to:

• receive, from one or more first sensors, data indicative of a voice command corresponding to a vehicular operation and a reference to a first occupant within an interior of a vehicle;
• receive, from one or more second sensors, data indicative of an image of the interior of the vehicle;
• determine, using processing circuitry that implements at least one neural network, a positional region of the first occupant within the interior of the vehicle based at least in part on the image;
• retrieve, from a database, authentication data for a second occupant that issued the voice command;
• determine, based at least in part on the authentication data, the data indicative of the voice command, and the positional region of the first occupant, a component of the vehicle that the vehicular operation corresponds to;
• and
• cause the vehicular operation to be performed with respect to the component of the vehicle.