# US 2023/0274076 A1 · Intelligent Inset Window Placement in Content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274076 A1
- Document ID: US20230274076A1
- Status: Published application
- Kind: A1
- Application number: 17/682,885
- Filed: 2022-02-28
- Published: 2023-08-31
- Earliest claimed: 2022-02-28
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Adam P Williams; David A Underwood; Amy W Hung
- Web page: https://patentgrep.com/patent/US20230274076A1

## CPC

- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 40/106: Display of layout of documents; Previewing
- G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V 10/764: using classification, e.g. of video objects
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Disclosed herein are techniques for providing secondary content within primary content via an adaptive inset window. Adaptations of the adaptive inset window may include specifying an activation status, size, shape, and/or location of an adaptive inset window to be presented in the primary content. The adaptation may be defined based upon a salient region identified based upon object within the primary content.

## Claims

### Claim 1

A tangible, non-transitory, computer-readable medium, comprising computer-readable instructions that, when executed by one or more processors of a computer, cause the computer to:
identify objects in a first viewable portion of digital content;
determine a salient region of the first viewable portion of the digital content based upon the objects in the first viewable portion of the digital content;
determine an activation status, size, shape, location, or any combination thereof of an inset window configured to cause the inset window to avoid, at least partially, the salient region of the first viewable portion of the digital content; and
cause the inset window to be rendered in the first viewable portion of the digital content in accordance with the activation status, the size, the shape, the location, or the any combination thereof.

### Claim 2 (depends on 1)

The tangible, non-transitory, computer-readable medium of claim 1, comprising computer-readable instructions that, when executed by the one or more processors of the computer, cause the computer to:
determine that no suitable presentation region exists in the first viewable portion of the digital content for the inset window based upon the salient region of the first viewable portion of the digital content;
determine the activation status to be a status indicating that the inset window should not be rendered in the first viewable portion of the digital content, based upon determining that no suitable presentation region in the first viewable portion of the digital content exists; and
causing refraining of rendering the inset window in the first viewable portion of the digital content based upon the activation status.

### Claim 3 (depends on 1)

The tangible, non-transitory, computer-readable medium of claim 1, comprising computer-readable instructions that, when executed by the one or more processors of the computer, cause the computer to:
identify objects in a second viewable portion of the digital content;
determine a salient region of the second viewable portion of the digital content based upon the objects in the second viewable portion of the digital content;
determine a second activation status, second size, second shape, second location, or any combination thereof of an inset window configured to cause the inset window to avoid, at least partially, the salient region of the second viewable portion of the digital content;
transition from rendering the first viewable portion to rendering in the second viewable portion; and
cause the inset window to be rendered in the second viewable portion of the digital content in accordance with the second activation status, the second size, the second shape, the second location, or the any combination thereof.

### Claim 4 (depends on 3)

The tangible, non-transitory, computer-readable medium of claim 3, wherein:
the salient region of the first viewable portion of the digital content and the salient region of the second viewable portion of the digital content are different; and
the activation status, the size, the shape, the location, or any combination thereof are different than the second activation status, the second size, the second shape, the second location, or the any combination thereof.

### Claim 5 (depends on 1)

The tangible, non-transitory, computer-readable medium of claim 1, wherein:
the digital content comprises a slide presentation; and
the first viewable portion of the digital content comprises a first slide of the slide presentation.

### Claim 6 (depends on 5)

The tangible, non-transitory, computer-readable medium of claim 5, comprising computer-readable instructions that, when executed by the one or more processors of the computer, cause the computer to:
cause the inset window to be rendered in the first slide of the slide presentation in accordance with the activation status, the size, the shape, the location, or the any combination thereof when the slide presentation is in a presentation mode.

### Claim 7 (depends on 1)

The tangible, non-transitory, computer-readable medium of claim 1, comprising computer-readable instructions that, when executed by the one or more processors of the computer, cause the computer to:
when one or more of the objects includes an animation effect, determine the salient region based at least in part upon the animation effect.

### Claim 8 (depends on 7)

The tangible, non-transitory, computer-readable medium of claim 7, comprising computer-readable instructions that, when executed by the one or more processors of the computer, cause the computer to:
identify the salient region at particular intervals of the animation effect; and
adjust the activation status, size, shape, location, or any combination thereof of the inset window based at least in part upon the salient region at the particular intervals of the animation effect.

### Claim 9 (depends on 7)

The tangible, non-transitory, computer-readable medium of claim 7, wherein the computer-readable medium comprises computer-readable instructions that, when executed by the one or more processors of the computer, cause the computer to:
determine the activation status such that the inset window is not rendered during the animation of the objects in the first viewable portion of the digital content and is rendered after completion of the animation of the objects in the first viewable portion of the digital content;
determine the activation status such that the inset window is rendered during a portion of the animation of the objects in the first viewable portion of the digital content, but not during an entire portion of the animation of the objects; or
both.

### Claim 10 (depends on 1)

The tangible, non-transitory, computer-readable medium of claim 1, wherein the computer-readable medium comprises computer-readable instructions that, when executed by the one or more processors of the computer, cause the computer to:
classify one or more of the objects as possible non-salient area objects that may include non-salient content that the inset window can overlap;
identify, using machine learning, an important area of the possible non-salient area objects; and
determine the salient region based upon the important area of the possible non-salient area objects, the salient region comprising less than a whole area of the possible non-salient area objects.

### Claim 11 (depends on 10)

The tangible, non-transitory, computer-readable medium of claim 10, wherein the one or more objects are classified as possible non-salient area objects based upon their object type, wherein their object type comprises: a text box, an image, or both.

### Claim 12 (depends on 1)

The tangible, non-transitory, computer-readable medium of claim 1, comprising computer-readable instructions that, when executed by the one or more processors of the computer, cause the computer to:
identify that the salient region of the first viewable portion of the digital content comprises no portion of the first viewable portion of the digital content; and
in response to identifying that the salient region of the first viewable portion of the digital content comprises no portion of the first viewable portion of the digital content, determine the activation status as active and the size and the shape such that the inset window encompasses an expanded portion of the first viewable portion of the digital content.

### Claim 13 (depends on 1)

The tangible, non-transitory, computer-readable medium of claim 1, wherein:
the digital content comprises a spreadsheet;
the first viewable portion of the digital content comprises a first sheet of the spreadsheet; and
wherein the salient region is determined based upon populated cells within the first sheet.

### Claim 14 (depends on 1)

The tangible, non-transitory, computer-readable medium of claim 1, wherein the inset window comprises a video view of a presenter of the digital content, supplemental content corresponding to the first viewable portion, or both.

### Claim 15

A computer-implemented method, comprising:
identifying objects in a first viewable portion of digital content;
determining a salient region of the first viewable portion of the digital content based upon the objects in the first viewable portion of the digital content;
determining an activation status, size, shape, location, or any combination thereof of an inset window configured to cause the inset window to avoid, at least partially, the salient region of the first viewable portion of the digital content; and
causing the inset window to be rendered in the first viewable portion of the digital content in accordance with the activation status, the size, the shape, the location, or the any combination thereof.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15, comprising:
identifying objects in a second viewable portion of the digital content;
determining a salient region of the second viewable portion of the digital content based upon the objects in the second viewable portion of the digital content;
determining a second activation status, second size, second shape, second location, or any combination thereof of an inset window configured to cause the inset window to avoid, at least partially, the salient region of the second viewable portion of the digital content;
transitioning from rendering the first viewable portion to rendering in the second viewable portion; and
causing the inset window to be rendered in the second viewable portion of the digital content in accordance with the second activation status, the second size, the second shape, the second location, or the any combination thereof.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, comprising:
identifying a portion-to-portion constraint for rendering the inset window between rendering the inset window in the first viewable portion and rendering the inset window in the second viewable portion; and
causing the inset window to be rendered in the second viewable portion only in accordance with the portion-to-portion constraint.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, wherein the portion-to-portion constraint comprises: a size limitation, a location limitation, a shape limitation, or any combination thereof with respect to rendering the inset window in the first viewable portion and rendering the inset window in the second viewable portion.

### Claim 19

An electronic device, comprising:
a display;
a memory storing computer-readable instructions;
one or more processors configured to execute the computer-readable instructions, causing the one or more processors to:
identify objects in a first viewable portion of digital content to be displayed on the display;
determine a salient region of the first viewable portion of the digital content based upon the objects in the first viewable portion of the digital content;
determine an activation status, size, shape, location, or any combination thereof of an inset window configured to cause the inset window to avoid, at least partially, the salient region of the first viewable portion of the digital content; and
cause the inset window to be rendered in the first viewable portion of the digital content, on the display, in accordance with the activation status, the size, the shape, the location, or the any combination thereof.

### Claim 20 (depends on 19)

The electronic device of claim 19, wherein the one or more processors are configured to execute the computer-readable instructions, causing the one or more processors to:
identify a requested default constraint for a presentation of the inset window;
determine that the requested default constraint cannot be facilitated based upon the objects in the first viewable portion of the digital content to be displayed on the display;
identify a modification to the objects in the first viewable portion of the digital content to be displayed on the display that will enable the requested default constraint to be met; and
provide, via a graphical user interface, a prompt proposing the modification to the objects in the first viewable portion of the digital content to be displayed on the display.
