Privacy-aware meeting room transcription from audio-visual stream

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• receiving an audio-visual signal comprising audio data and image data, the audio data corresponding to speech utterances from a plurality of participants in a speech environment and the image data representing faces of the plurality of participants in the speech environment;
• receiving a privacy request from a participant of the plurality of participants, the privacy request indicating a privacy condition associated with the participant in the speech environment;
• segmenting the audio data into a plurality of segments;
• for a respective segment of the plurality of segments of the audio data:
• determining from among the plurality of participants, an identity of a speaker of the respective segment of the audio data based on the image data;
• determining that the identity of the speaker of the respective segment comprises the participant associated with the privacy condition indicated by the received privacy request;
• and
• based on determining that the identity of the speaker of the respective segment comprises the participant, omitting transcribing the respective segment based on the privacy condition indicated by the received privacy request;
• and
• processing the plurality of segments of the audio data to determine a transcript for the audio data, the transcript comprising text corresponding to each segment of the plurality of segments other than the respective segment and omitting text corresponding to the respective segment.