# US 2023/0066504 A1 · Automated adaptation of video feed relative to presentation content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0066504 A1
- Document ID: US20230066504A1
- Status: Published application
- Kind: A1
- Application number: 17/458,003
- Filed: 2021-08-26
- Published: 2023-03-02
- Earliest claimed: 2021-08-26
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Fatima Zohra DAHA; Robert Fernand GORDAN; Amit SRIVASTAVA; Joshua Alexander DOCTORS
- Web page: https://patentgrep.com/patent/US20230066504A1

## CPC

- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- H04N 7/15: Conference systems
- G06F 16/4393: {Multimedia presentations, e.g. slide shows, multimedia albums}
- G06N 20/00: Machine learning
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- H04L 65/4015: {where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference}
- H04N 5/265: Mixing
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

The present disclosure relates to application of artificial intelligence (AI) processing that adapts one or more video feeds relative to presentation content. Trained AI processing automatically generates a combined representation comprising one or more video feeds and presentation content. An exemplary combined representation is the result of contextual analysis by one or more trained AI models that are adapted to consider how to adapt presentation of a video feed relative to displayable presentation content (or visa-versa). A combined representation of one or more video feeds and presentation content is automatically generated (and subsequently rendered) based on a result of contextual evaluation of data associated with a video feed and data attributes of presentation content. A combined representation may comprise a modification of the one or more video feeds, objects of presentation content or a combination thereof. Further examples extend to management of combined representations through an adapted graphical user interface.

## Claims

### Claim 1

A computer-implemented method comprising:
detecting a video feed that is presented concurrently with presentation content of a presentation application or service;
automatically applying a trained artificial intelligence (AI) model that is adapted to generate a combined representation of the video feed with the presentation content of the presentation application or service, wherein the trained AI model executes processing operations that comprise:
generating contextual determinations for combining the video feed with the presentation content based on analysis of the video feed, wherein the contextual determinations comprise a cropping determination indicating whether the video feed is cropped to focus on a head of at least one user,
selecting a level of adaptation of the video feed to focus on the head of the at least one user based on analysis of contextual determinations including the cropping determination,
selecting a placement configuration for the video feed relative to the presentation content based on a comparative analysis of the data associated with the level of adaption of the video feed and data attributes of objects within the presentation content, and
automatically generating a combined representation of the video feed and the presentation content, wherein the combined representation overlays the video feed over the presentation content according to the placement configuration and the video feed focuses on the head of the at least one user.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising: causing rendering of the combined representation within a graphical user interface of the presentation application or service.

### Claim 3

A computer-implemented method comprising:
detecting a video feed that is presented concurrently with presentation content of a presentation application or service;
automatically applying a first trained artificial intelligence (AI) model that is adapted to generate contextual determinations for combining the video feed with the presentation content based on analysis of the video feed, wherein the contextual determinations comprise a cropping determination indicating whether the video feed is cropped to focus on a head of at least one user;
propagating the contextual determinations, generated by the first trained AI model, to a second trained AI model that is adapted to manage cropping of the video feed for combination with the presentation content;
selecting, using the second trained AI model, a level of adaptation of the video feed to focus on the head of the at least one user based on analysis of contextual determinations including the cropping determination;
propagating, data associated with the level of adaptation of the video feed to a third trained AI model that is adapted to manage placement of the video feed relative to the presentation content;
selecting, using the third trained AI model, a placement configuration for the video feed relative to the presentation content based on a comparative analysis of the data associated with the level of adaption of the video feed and data attributes of objects within the presentation content; and
automatically generating a combined representation of the video feed and the presentation content, wherein the combined representation overlays the video feed over the presentation content according to the placement configuration and the video feed focuses on the head of the at least one user.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, further comprising: causing rendering of the combined representation within a graphical user interface of the presentation application or service.

### Claim 5 (depends on 3)

The computer-implemented method of claim 3, wherein the selecting of the placement configuration for the video feed comprises selecting, based on the comparative analysis of the data associated with the level of adaption of the video feed and the data attributes of objects within the presentation content, both of a size for display of the video feed and a placement position for overlaying the video feed over the presentation content.

### Claim 6 (depends on 3)

The computer-implemented method of claim 3, wherein the first trained AI model executed image segmentation of at least one frame of the video feed to determine one or more regions of interest in the at least one frame of the video feed, and wherein the cropping determination determines whether the video feed is cropped to display the head of the at least one user based on analysis of the one or more regions of interest in the at least one frame.

### Claim 7 (depends on 3)

The computer-implemented method of claim 3, wherein the contextual determinations, generated by the first trained AI model, further comprise: a size determination indicating a size of the video feed, a video resolution determination indicating a video resolution of the video feed, and a frame rate determination indicating a frame rate of the video feed.

### Claim 8 (depends on 3)

The computer-implemented method of claim 3, wherein the level of adaptation of the video feed, selected by the second trained AI model, bypasses cropping of the video feed when the cropping determination indicates that the video feed is cropped to focus on the head of the at least one user.

### Claim 9 (depends on 3)

The computer-implemented method of claim 3, wherein the level of adaptation of the video feed, selected by the second trained AI model, requires cropping of the video feed when the cropping determination indicates that the video feed is not cropped to focus on the head of the at least one user, and wherein the second trained AI model executes processing to crop the video feed to focus on the head of the at least one user.

### Claim 10 (depends on 3)

The computer-implemented method of claim 3, wherein the presentation content is a slide of slide-based presentation content, and wherein the placement configuration for the video is selected for the slide of the slide-based presentation content.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, further comprising: detecting a change to display of the slide-based presentation content where the presentation content changes from display of the slide to display of a second slide of the slide-based presentation content; in response to detecting the change to display of the slide-based presentation content, selecting, using the third trained AI model, an updated placement configuration for the video feed relative to the presentation content of the second slide based on a comparative analysis of the data associated with the level of adaption of the video feed and data attributes of objects within the second slide; and automatically updating the combined representation of the video feed and the presentation content so that the video feed the second slide according to the updated placement configuration.

### Claim 12

A system comprising:
at least one processor; and
a memory, operatively connected with the at least one processor, storing computer-executable instructions that, when executed by the at least one processor, causes the at least one processor to execute a method that comprises:
detecting a video feed that is presented concurrently with presentation content of a presentation application or service;
automatically applying a first trained artificial intelligence (AI) model that is adapted to generate contextual determinations for combining the video feed with the presentation content based on analysis of the video feed, wherein the contextual determinations comprise a cropping determination indicating whether the video feed is cropped to focus on a head of at least one user;
propagating the contextual determinations, generated by the first trained AI model, to a second trained AI model that is adapted to manage cropping of the video feed for combination with the presentation content;
selecting, using the second trained AI model, a level of adaptation of the video feed to focus on the head of the at least one user based on analysis of contextual determinations including the cropping determination;
propagating, data associated with the level of adaptation of the video feed to a third trained AI model that is adapted to manage placement of the video feed relative to the presentation content;
selecting, using the third trained AI model, a placement configuration for the video feed relative to the presentation content based on a comparative analysis of the data associated with the level of adaption of the video feed and data attributes of objects within the presentation content; and
automatically generating a combined representation of the video feed and the presentation content, wherein the combined representation overlays the video feed over the presentation content according to the placement configuration and the video feed focuses on the head of the at least one user.

### Claim 13 (depends on 12)

The system of claim 12, wherein the method, executed by the at least one processor, further comprises: causing rendering of the combined representation within a graphical user interface of the presentation application or service.

### Claim 14 (depends on 12)

The system of claim 12, wherein the selecting of the placement configuration for the video feed comprises selecting, based on the comparative analysis of the data associated with the level of adaption of the video feed and the data attributes of objects within the presentation content, both of a size for display of the video feed and a placement position for overlaying the video feed over the presentation content.

### Claim 15 (depends on 12)

The system of claim 12, wherein the first trained AI model executed image segmentation of at least one frame of the video feed to determine one or more regions of interest in the at least one frame of the video feed, and wherein the cropping determination determines whether the video feed is cropped to display the head of the at least one user based on analysis of the one or more regions of interest in the at least one frame.

### Claim 16 (depends on 12)

The system of claim 12, wherein the contextual determinations, generated by the first trained AI model, further comprise: a size determination indicating a size of the video feed, a video resolution determination indicating a video resolution of the video feed, and a frame rate determination indicating a frame rate of the video feed.

### Claim 17 (depends on 12)

The system of claim 12, wherein the level of adaptation of the video feed, selected by the second trained AI model, bypasses cropping of the video feed when the cropping determination indicates that the video feed is cropped to focus on the head of the at least one user.

### Claim 18 (depends on 12)

The system of claim 12, wherein the level of adaptation of the video feed, selected by the second trained AI model, requires cropping of the video feed when the cropping determination indicates that the video feed is not cropped to focus on the head of the at least one user, and wherein the second trained AI model executes processing to crop the video feed to focus on the head of the at least one user.

### Claim 19 (depends on 12)

The system of claim 12, wherein the presentation content is a slide of slide-based presentation content, and wherein the placement configuration for the video is selected for the slide of the slide-based presentation content.

### Claim 20 (depends on 19)

The system of claim 19, wherein the method, executed by the at least one processor, further comprises: detecting a change to display of the slide-based presentation content where the presentation content changes from display of the slide to display of a second slide of the slide-based presentation content; in response to detecting the change to display of the slide-based presentation content, selecting, using the third trained AI model, an updated placement configuration for the video feed relative to the presentation content of the second slide based on a comparative analysis of the data associated with the level of adaption of the video feed and data attributes of objects within the second slide; and automatically updating the combined representation of the video feed and the presentation content so that the video feed the second slide according to the updated placement configuration.
