Method and apparatus for generating speech outputs in a vehicle

An apparatus for multimodal speech communication with a user of a vehicle, the apparatus comprising:

• at least one microphone configured to record speech input by the user;
• at least one imaging sensor configured to record additional information in a passenger compartment of the vehicle;
• sensors configured to record a situation surrounding the vehicle;
• and
• a control device coupled to the at least one microphone and the at least one imaging sensor, wherein the control device is configured to
• analyze the recorded speech input, the recorded additional information, and, wherein the analysis of the recorded additional information comprises analyzing objects in the additional information and assigning the analyzed objects to at least one predetermined category of a plurality of predetermined categories;
• and
• generate a speech output based on the analysis of the recorded speech input and additional information, wherein the generated speech output is enriched with keywords or formulations matching one of the predetermined categories,
• wherein a degree of the enrichment of the generated speech output is adapted depending on the situation surrounding the vehicle,
• wherein the adaptation of the degree of enrichment ranges from no enrichment to at least one intermediate stage to a strong enrichment, and
• wherein the adaptation of the degree of enrichment is adapted based on a complexity of the surrounding situation,
• wherein the adaptation of the degrees of enrichment is less when the complexity of the surrounding situation is high than when the complexity of the surrounding situation is low.

A method comprising:

• recording, by a vehicle microphone, a speech input;
• recording, by an imaging sensor of the vehicle, a passenger compartment of the vehicle;
• identifying, by the vehicle, objects in the recording of the passenger compartment;
• assigning, by the vehicle, the identified objects to at least one predetermined category of a plurality of predetermined categories;
• determining, by semantic analysis of the at least one predetermined categories to which the identified objects are assigned and the recorded speech input, a connection between the at least one predetermined categories to which the identified objects are assigned and the recorded speech input;
• determining, by the vehicle, a situation surrounding the vehicle via sensors of the vehicle or via data regarding the situation surrounding the vehicle obtained from an external server;
• determining, based on the determined situation surrounding the vehicle, a degree of enrichment of a generated speech output, wherein the determined degree of enrichment is less when complexity of the situation surrounding the vehicle is high than when the complexity of the situation surrounding the vehicle is low;
• selecting, based on the recorded speech input, a speech output;
• selecting, based on the determined degree of enrichment and on the connection between the at least one predetermined categories to which the identified objects are assigned and the recorded speech input, keywords or formulations matching one of the predetermined categories;
• merging the speech output with the selected keywords or formulations to generate an enriched speech output;
• and
• audibly outputting, by the vehicle, the enriched speech output.