Multicamera collaborative communication session system for dynamic detection and augmentation of visual aid display

A method comprising:

• receiving a plurality of video streams for a collaborative communication session associated with one or more collaborative communication session endpoint systems, the plurality of video streams including a first video stream, a second video stream, and a third video stream;
• selecting the first video stream of the plurality of video streams comprising video of an active speaker participant in the collaborative communication session;
• detecting, from the first video stream, speech that includes words spoken by the active speaker participant;
• determining, based on the words spoken by the active speaker participant, a category of speech content in the speech of the active speaker participant;
• determining, based on the third video stream of the plurality of video streams, a category of visual aid content associated with a visual aid;
• matching the category of the speech content with the category of the visual aid content;
• determining, based on matching the category of the speech content with the category of the visual aid content, that the active speaker participant is discussing the visual aid;
• selecting, in response to determining that the active speaker participant is discussing the visual aid, the second video stream of the plurality of video streams comprising video of the visual aid that includes the visual aid content;
• digitizing the visual aid content to generate digitized visual aid content;
• and
• providing the first video stream and the second video stream with the digitized visual aid content to the one or more collaborative communication session endpoint systems for concurrent display of the first video stream and the second video stream at the one or more collaborative communication session endpoint systems.

An apparatus comprising:

• a plurality of cameras of a collaborative communication session endpoint system,
• a network interface configured to communicate over a network;
• and
• one or more processors configured to perform operations comprising:
• controlling, during a collaborative communication session, a first camera of the plurality of cameras to provide room scanning of a room in which the collaborative communication session endpoint system is arranged;
• determining, based on a first video stream associated with the first camera, a category of visual aid content associated with a physical visual aid in the room;
• controlling a second camera of the plurality of cameras to image an active speaker participant in the collaborative communication session;
• detecting, from a second video stream associated with the second camera, speech that includes words being spoken by the active speaker participant;
• determining, from the words being spoken by the active speaker participant, a category of speech content in the speech of the active speaker participant;
• matching the category of the speech content with the category of the visual aid content;
• determining, based on matching the category of the speech content with the category of the visual aid content, that the active speaker participant is discussing a physical visual aid;
• controlling, based on determining that the active speaker participant is discussing the physical visual aid, a third camera of the plurality of cameras to image the physical visual aid;
• and
• digitizing content of the physical visual aid to generate a digitized visual aid for display on devices of participants of the collaborative communication session.