Video conference content auto-retrieval and focus based on learned relevance

A non-transitory computer-readable medium comprising instructions, which when executed by a processing device cause the processing device to perform operations comprising:

• determining, based on a first plurality of media streams provided by a plurality of client devices participating in a video conference, a first subject being discussed during the video conference at a first point in time, the first plurality of media streams associated with the first point in time;
• providing one or more first inputs to a machine learning model, the one or more first inputs comprising a first plurality of transcriptions each corresponding to one of the first plurality of media streams provided by the plurality of client devices participating in the video conference, and an indication of a request to identify one or more first media streams that are relevant to the first subject;
• determining, based on one or more first outputs of the machine learning model, a first media stream that is most relevant to the first subject, wherein the first media stream comprises at least one of audio data or video data created by a first client device of the plurality of client devices during the video conference;
• providing the determined first media stream for presentation on one or more of the plurality of client devices participating in the video conference while the first subject is being discussed during the video conference;
• and
• responsive to determining, at a second point in time during the video conference, that the first subject has changed to a second subject:
• providing one or more second inputs to the machine learning model, the one or more second inputs comprising a second plurality of transcriptions each corresponding to one of a second plurality of media streams of the plurality of client devices that are associated with the second point in time, and an indication of a request to identify one or more second media streams that are relevant to the second subject;
• determining, based on one or more second outputs of the machine learning model, a second media stream that is most relevant to the second subject, wherein the second media stream comprises at least one of audio data or video data created by a second client device of the plurality of client devices during the video conference;
• and
• providing the determined second media stream for presentation on the plurality of client devices while the second subject is being discussed during the video conference.

A system comprising:

• a memory;
• and
• a processing device, coupled to the memory, to perform operations comprising:
• determining, based on a first plurality of media streams provided by a plurality of client devices participating in a video conference, a first subject being discussed during the video conference at a first point in time, the first plurality of media streams associated with the first point in time;
• providing one or more first inputs to a machine learning model, the one or more first inputs comprising a first plurality of transcriptions each corresponding to one of the first plurality of media streams provided by the plurality of client devices participating in the video conference, and an indication of a request to identify one or more first media streams that are relevant to the first subject;
• determining, based on one or more first outputs of the machine learning model, a first media stream that is most relevant to the first subject, wherein the first media stream comprises at least one of audio data or video data created by a first client device of the plurality of client devices during the video conference;
• providing the determined first media stream for presentation on one or more of the plurality of client devices participating in the video conference while the first subject is being discussed during the video conference;
• and
• responsive to determining, at a second point in time during the video conference, that the first subject has changed to a second subject:
• providing one or more second inputs to the machine learning model, the one or more second inputs comprising a second plurality of transcriptions each corresponding to one of a second plurality of media streams of the plurality of client devices that are associated with the second point in time, and an indication of a request to identify one or more second media streams that are relevant to the second subject;
• determining, based on one or more second outputs of the machine learning model, a second media stream that is most relevant to the second subject, wherein the second media stream comprises at least one of audio data or video data created by a second client device of the plurality of client devices during the video conference;
• and
• providing the determined second media stream for presentation on the plurality of client devices while the second subject is being discussed during the video conference.