# US 10,482,623 B2 · Placement of augmented reality objects using a bounding shape

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,482,623 B2
- Document ID: US10482623B2
- Status: Granted
- Kind: B2
- Application number: 16/128,201
- Filed: 2018-09-11
- Published: 2019-06-27
- Granted: 2019-11-19
- Earliest claimed: 2017-12-21
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Geoffrey Dagley; Jason Hoover; Qiaochu Tang; Stephen Wylie; Sunil Vasisht; Micah Price
- Web page: https://patentgrep.com/patent/US10482623B2

## CPC

- G06T 7/73: using feature-based methods
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 11/65: on geographic maps
- G06T 15/20: Perspective computation
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 2207/30204: Marker
- G06T 2210/12: Bounding box
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system may include a camera, a display, one or more memories, and one or more processors communicatively coupled to the one or more memories. The system may output a bounding shape for presentation on the display. The bounding shape may be superimposed on an image being captured by the camera and presented on the display. The bounding shape may bound an object in the image. The system may determine 3D coordinates of an intersection point associated with the bounding shape. The intersection point may be a point where a projection of the bounding shape into 3D space intersects with a horizontal plane identified in the image. The system may determine 2D coordinates for presentation of an augmented reality object on the display based on the 3D coordinates of the intersection point, and may superimpose a representation of the augmented reality object on the image based on the 2D coordinates.

## Claims

### Claim 1

A system, comprising:
a camera;
a display;
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, to:
output a bounding shape for presentation on the display,
the bounding shape being superimposed on an image being captured by the camera, and
a base of the bounding shape to be aligned with a base of a target object in the image;
determine three-dimensional (3D) coordinates of an intersection point associated with the bounding shape;
determine 3D coordinates of an augmented reality object based on the determined 3D coordinates of the intersection point associated with the bounding shape,
the augmented reality object being offset based upon at least one of:
a height of the target object in the image, or
a height of the bounding shape;
determine two-dimensional (2D) coordinates for presentation of the augmented reality object based on the determined 3D coordinates of the augmented reality object; and
superimpose the augmented reality object, according to the 2D coordinates, on the image, as the image is rendered for display.

### Claim 2 (depends on 1)

The system of claim 1, where the 3D coordinates of the intersection point are determined based on an indication that the bounding shape is bounding the target object in the image.

### Claim 3 (depends on 1)

The system of claim 1, where the one or more processors are further to:
apply object recognition to change a size of the bounding shape to bound the target object in the image.

### Claim 4 (depends on 1)

The system of claim 1, where the intersection point is a point where a projection of the bounding shape into 3D space intersects with a horizontal plane,
the horizontal plane being designated as a ground plane.

### Claim 5 (depends on 4)

The system of claim 4, where the one or more processors, when designating the horizontal plane as the ground plane, are to:
identify the horizontal plane;
determine a size of the horizontal plane; and
determine that the size of the horizontal plane meets a threshold.

### Claim 6 (depends on 4)

The system of claim 4, where the intersection point is a point where a projection of a lower boundary, of the bounding shape, into 3D space intersects with the horizontal plane identified in the image.

### Claim 7 (depends on 1)

The system of claim 1, where the bounding shape is a bounding box.

### Claim 8

A method, comprising:
outputting, by a device, a bounding shape for presentation on a display of the device,
the bounding shape being superimposed on an image being captured by the device and presented on the display of the device,
the bounding shape bounding an object in the image, and
a base of the bounding shape to be aligned with a base of the object in the image;
determining, by a device, three-dimensional (3D) coordinates of an intersection point associated with a bounding shape,
the intersection point being a point where a projection of the bounding shape into 3D space intersects with a horizontal plane identified in the image;
determining, by the device, 3D coordinates of an augmented reality object based on the determined 3D coordinates of the intersection point associated with the bounding shape,
the augmented reality object being offset based upon a height of the object in the image;
determining, by the device, two-dimensional (2D) coordinates for presentation of the augmented reality object based on the determined 3D coordinates of the augmented reality object; and
superimposing, by the device, the augmented reality object, according to the 2D coordinates, on the image, as the image is rendered for display.

### Claim 9 (depends on 8)

The method of claim 8, where the height of the object is determined based on a height of the bounding shape.

### Claim 10 (depends on 9)

The method of claim 9, where the 2D coordinates for presentation of the augmented reality object are generated by applying an offset to 2D coordinates that represent a position of a lower point of the bounding shape on the display of the device.

### Claim 11 (depends on 8)

The method of claim 8, where the bounding shape is a bounding box.

### Claim 12 (depends on 8)

The method of claim 8, where the intersection point is a point where a projection of a lower boundary, of the bounding shape, into 3D space intersects with the horizontal plane identified in the image.

### Claim 13 (depends on 8)

The method of claim 8, where the 2D coordinates for presentation of the augmented reality object are determined based on a height of the bounding shape.

### Claim 14 (depends on 8)

The method of claim 8, where the horizontal plane is designated as a ground plane.

### Claim 15

A non-transitory computer-readable medium storing instructions, the instructions comprising:
one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to:
output a bounding shape for presentation on a display of the device,
the bounding shape being superimposed on an image being captured by the device and presented on the display,
the bounding shape bounding an object in the image, and
a base of the bounding shape to be aligned with a base of the object in the image;
determine three-dimensional (3D) coordinates of an intersection point associated with the bounding shape;
determine 3D coordinates of an augmented reality object based on the determined 3D coordinates of the intersection point associated with the bounding shape,
the 3D coordinates associated with placement of the augmented reality object being generated by applying an offset to the 3D coordinates of the intersection point;
determine 2D coordinates for presentation of the augmented reality object based on the determined 3D coordinates of the augmented reality object; and
superimpose the augmented reality object, according to the 2D coordinates, on the image, as the image is rendered for display.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, where the offset is determined based on a height of the object,
the height of the object being determined based on a height of the bounding shape.

### Claim 17 (depends on 15)

The non-transitory computer-readable medium of claim 15, where the 3D coordinates of the intersection point are determined based on an indication that the bounding shape is bounding the object.

### Claim 18 (depends on 15)

The non-transitory computer-readable medium of claim 15, where the intersection point is a point where a projection of the bounding shape into 3D space intersects with a horizontal plane identified in the image.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, where the horizontal plane is identified as a ground plane.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, where the 2D coordinates for presentation of the augmented reality object are determined based on the offset.