# US 9,437,038 B1 · Simulating three-dimensional views using depth relationships among planes of content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,437,038 B1
- Document ID: US9437038B1
- Status: Granted
- Kind: B1
- Application number: 14/038,548
- Filed: 2013-09-26
- Granted: 2016-09-06
- Earliest claimed: 2013-09-26
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Kevin Robert Costello; Christopher Wayne Lockhart
- Web page: https://patentgrep.com/patent/US9437038B1

## CPC

- G06T 15/20: Perspective computation
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06T 15/08: Volume rendering
- G06T 15/503: {Blending, e.g. for anti-aliasing}
- G06T 15/60: Shadow generation
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Approaches enable image content (e.g., still or video content) to be displayed in such a way that the image content will appear, to a viewer, to include portions with different locations in physical space, with the relative positioning of those portions being determined at least in part upon a current relative position and/or orientation of the viewer with respect to the device, as well as changes in that relative position and/or orientation. For example, relationship pairs for image content capable of being displayed on a display screen can be determined. Based on the relationship pairs, a node hierarchy that includes position information for planes of content that include the image content can be determined. The position information can be to render a view of the image content based on the relative position, direction, and/or orientation between the viewer and device to provide a two- or three-dimensional representation of that image content that is appropriate for that viewing angle, giving the impression of a three-dimensional view or display even when the display is in two dimensions.

## Claims

### Claim 1

A computing device, comprising:
a display screen;
a camera;
at least one computing device processor;
a memory device including instructions that, when executed by the at least one computing device processor, enable the computing device to:
determine relationship information between related content capable of being displayed on a display screen of a computing device, the relationship information indicates a relative arrangement of the related content;
determine, based at least in part on the relationship information, a node hierarchy for a plurality of nodes that includes position information for a first node of the plurality of nodes, the position information includes lateral position information and depth position information;
associate a first plane of content of a plurality of planes of content with the first node of the plurality of nodes based at least on the relationship information and a depth position of the first node;
display at a first lateral position on the display screen, first content included in the first plane of content;
display at a second lateral position on the display screen, second content included in a second plane of content;
acquire, using the camera, at least one image of a viewer of the computing device;
determine, by analyzing the at least one image, a location of the viewer with respect to the computing device;
determine a first lateral offset position for the first plane of content based at least in part on the location of the viewer;
determine a second lateral offset position for the second plane of content based at least in part on the location of the viewer;
display, on the display screen, the first content at the first lateral offset position; and
display, on the display screen, the second content at the second lateral offset position.

### Claim 2 (depends on 1)

The computing device of claim 1, wherein the related content includes at least first related content and second related content, wherein the instructions, when executed, further cause the computing device to:
determine a change in position of the first related content caused by at least one of an animation of the first content or a scrolling of an interface displaying the first content;
update first position information for one or more nodes associated with one of the plurality of planes of content associated with the first content; and
update second position information for at least one other node of the plurality of nodes associated with at least one other plane of content based at least upon the change in position of the first content, the at least one other plane of content associated with the second content, wherein the at least one other plane of content is determined based at least in part upon the relationship information.

### Claim 3 (depends on 1)

The computing device of claim 1, wherein each plane of the plurality of planes of content corresponds to a node of the plurality of nodes of a hierarchy, wherein the plurality of nodes of the hierarchy include at least one parent node and one or more child nodes depending from the at least one parent node, the depth position of the one or more child nodes being relative to at least one of the display screen or the at least one parent node.

### Claim 4 (depends on 1)

The computing device of claim 1, wherein the instructions, when executed, further cause the computing device to:
render at least one shadow based at least in part on an intersection of light from a virtual light source with the first plane of content.

### Claim 5

A computer implemented method, comprising:
determining relationship information between first content and second content capable of being displayed on a display screen of a computing device, the first content associated with a first interface plane of a plurality of interface planes and the second content associated with a second interface plane of the plurality of interface planes;
determining, based at least in part on the relationship information, a node hierarchy for a plurality of nodes that includes a first node associated with the first interface plane and a second node associated with the second interface plane, the first node having first position information that includes lateral position information and depth position information;
determining a first lateral offset position for the first interface plane based at least on a position of the computing device relative to a viewer of the display screen;
determining a second lateral offset position for the second interface plane based at least in part on the position of the computing device relative to the viewer of the display screen;
displaying, on the display screen, the first content based at least in part on the depth position information of the first node and the first lateral offset position; and
displaying, on the display screen, the second content based at least in part on the second lateral offset position.

### Claim 6 (depends on 5)

The computer implement method of claim 5, further comprising:
determining a change in position of the first content caused by at least one of an animation of the first content or a scrolling of an interface displaying the first content;
updating the first position information for the first node based at least on the change in position; and
updating second position information for the second node associated with the second interface plane based at least on the change in position of the first content.

### Claim 7 (depends on 6)

The computer implement method of claim 6, further comprising:
determining the second node associated with the second interface plane based at least in part on relationship information between the first content and the second content.

### Claim 8 (depends on 5)

The computer implemented method of claim 5, wherein displaying the first content further includes:
displaying the first content to appear to be closer to a viewer of the computing device than content associated with at least one other interface plane, wherein the at least one other interface plane is associated with at least one other node having a depth position different than the depth position of the first node.

### Claim 9 (depends on 5)

The computer implemented method of claim 5, wherein displaying the first content further includes:
determining an adjusted appearance of at least one of a size, shape, color, shading, or blur for at least one of the plurality of interface planes according to a location of the viewer.

### Claim 10 (depends on 5)

The computer implemented method of claim 5, further comprising:
determining a change in position of the computing device from a first location to a second location relative to the viewer of the display screen; and
determining the first lateral offset position for the first interface plane based at least on the change in position of the computing device from the first location to the second location relative to the viewer to display occluded portions of the second content.

### Claim 11 (depends on 5)

The computer implemented method of claim 5, further comprising:
assigning the first interface plane to the depth position of the first node of the plurality of nodes;
assigning the second interface plane to a second depth position associated with a second node of the plurality of nodes, the depth position of the first node different from the second depth position of the second node;
determining a change in the position of the computing device relative to the viewer of the display screen;
determining the first lateral offset position for the first interface plane further based on the change in the position of the computing device relative to the viewer of the display screen;
determining the second lateral offset position for the second interface plane further based on the change in the position of the computing device relative to the viewer of the display screen; and
displaying, on the display screen, the second content further based at least in part on the second depth position associated with the second node.

### Claim 12 (depends on 5)

The computer implemented method of claim 5, rendering, based at least on a draw order, the first interface plane of the plurality of interface planes with a first depth position before rendering the second interface plane of the plurality of interface planes with a second depth position, the first interface plane appearing closer to a viewer of the computing device than the second interface plane.

### Claim 13 (depends on 5)

The computer implemented method of claim 5, further comprising:
determining, based at least in part on the location of the viewer, coefficients for a set of transformation equations to adjust a scale and a translation for a respective interface plane with respect to a determined pivot point associated with a center of the display screen; and
determining an adjusted appearance of for the first content associated with the first interface plane using the set of transformation equations.

### Claim 14 (depends on 5)

The computer implemented method of claim 5, wherein each interface plane of the plurality of interface planes corresponds to a node of the plurality of nodes of the node hierarchy, wherein the plurality of nodes include at least one parent node and one or more child nodes depending from the at least one parent node, and wherein the plurality of nodes are provided via an operating system of the computing device.

### Claim 15 (depends on 5)

The computer implemented method of claim 5, further including:
rendering at least one shadow based at least in part on an intersection of light from a virtual light source with the first interface plane.

### Claim 16

A non-transitory computer readable storage medium storing one or more sequences of instructions executable by one or more processors to perform a set of operations comprising:
determining relationship information between first content and second content capable of being displayed on a display screen of a computing device, the first content being associated with a first interface plane of a plurality of interface planes and the second content being associated with a second interface plane of the plurality of interface planes;
determining, based at least in part on the relationship information, a node hierarchy for a plurality of nodes that includes a first node associated with the first interface plane and a second node associated with the second interface plane, the first node of the plurality of nodes having position information that includes lateral position information and depth position information;
determining a first lateral offset position for the first interface plane based at least on a position of the computing device relative to a viewer of the display screen;
determining a second lateral offset position for the second interface plane based at least in part on the position of the computing device relative to the viewer of the display screen; and
displaying, on the display screen, the first content based at least in part on the depth position information of the first node and the first lateral offset position for the first interface plane; and
displaying, on the display screen, the second content based at least in part on the second lateral offset position for the second interface plane.

### Claim 17 (depends on 16)

The non-transitory computer readable storage medium of claim 16, further comprising instructions executed by the one or more processors to perform the operations of:
determining a change in position of the first content caused by at least one of an animation of the first content or a scrolling of an interface displaying the first content;
updating the first lateral offset position for the first interface plane based at least on the change in position; and
updating the second lateral offset position for the second interface plane based at least on the change in position.

### Claim 18 (depends on 16)

The non-transitory computer readable storage medium of claim 16, further comprising instructions executed by the one or more processors to perform the operations of:
determining a change in position of the computing device from a first location to a second location relative to the viewer of the display screen; and
determining the first lateral offset position for the first interface plane based at least on change in position of the computing device from the first location to the second location relative to the viewer to display occluded potions of the second content.

### Claim 19 (depends on 16)

The non-transitory computer readable storage medium of claim 16, wherein a first element associated with the first interface plane of the plurality of interface planes is capable of partially obscuring a second element associated with the second interface plane of the plurality of interface planes.