Dialog management for multiple users

A computer-implemented method comprising:

• receiving, by a user device operating in a first mode, first input audio data representing a first utterance initiated by a wakeword;
• processing the first utterance to determine a command to operate in a second mode corresponding to system participation in a conversation between at least two users;
• beginning operation in the second mode;
• receiving second input audio data representing a second utterance spoken by a first user;
• receiving first input image data representing the first user speaking the second utterance;
• processing the second input audio data and the first input image data to determine the first user is speaking to a second user;
• in response to determining the first user is speaking to the second user and to the operation in the second mode, using a first component to determine a system response to the second utterance is to be generated;
• in response to determining the system response to the second utterance is to be generated, processing the second input audio data to determine first output data responsive to the second utterance;
• and
• causing the user device to present the first output data.