# US 2019/0378341 A1 · Reconstruction of 3d model with immersive experience

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378341 A1
- Document ID: US20190378341A1
- Status: Published application
- Kind: A1
- Application number: 16/006,263
- Filed: 2018-06-12
- Published: 2019-12-12
- Earliest claimed: 2018-06-12
- Assignees: eBay Inc.
- Applicants: eBay Inc.
- Inventors: Jinrong Xie; Qiaosong Wang; Robinson Piramuthu
- Web page: https://patentgrep.com/patent/US20190378341A1

## CPC

- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 3/4092: Image resolution transcoding, e.g. by using client-server architectures
- G06T 15/04: Texture mapping
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 21/23439: {for generating different versions}
- H04N 21/25808: {Management of client data (terminal profiles in network data switching protocols H04L67/303)}
- H04N 21/8146: {involving graphical data, e.g. 3D object, 2D graphics}
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/004: Annotating, labelling
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system receives image data associated with an item, where the image data comprising a view of the item from two or more angles; determines physical attributes of the item; generates a base model of the item; samples the base model to generate one or more sampled models, each of the one or more sampled models comprising a subset of the geometric data, the subset of the geometric data determined based on one or more device characteristics of one or more user devices that interface with the system; receives device characteristics of a user device associated with a request from the user device for the item; selects, based on the received device characteristics, a sampled model of the item; and transmits a data object comprising the selected sampled model to the user device to cause the user device to generate a three-dimensional rendering of the item.

## Claims

### Claim 1

A system having a back-end device to:
receive image data associated with an item, the image data comprising a view of the item from two or more angles;
determine, automatically using the image data, physical attributes of the item;
generate, automatically using the physical attributes, a base model of the item, the base model comprising geometric data to generate a three-dimensional rendering of the item;
sample the geometric data to generate one or more sampled models, each of the one or more sampled models comprising a subset of the geometric data, the subset of the geometric data determined based on one or more device characteristics of one or more user devices that interface with the back-end processing device;
receive a device characteristic of a user device associated with a request from the user device for the item;
select, based on the received device characteristics, a sampled model of the item from a database comprising the one or more sampled models; and
transmit a data object comprising the selected sampled model to the user device to cause the user device to generate a three-dimensional rendering of the item using the data object and a graphics processing hardware circuit of user device.

### Claim 2 (depends on 1)

The system of claim 1, wherein to determine physical attributes of the rte is to determine identify texture data of the item using the image data.

### Claim 3 (depends on 1)

The system of claim 1, wherein the geometric data to generate a three-dimensional rendering of the item comprises geometric mesh of three-dimensional points determined using the texture data.

### Claim 4 (depends on 3)

The system of claim 3, wherein the subset of the geometric data is a subset of the three-dimensional points.

### Claim 5 (depends on 1)

The system of claim 1, wherein the one or more device characteristics includes at least one of a communication bandwidth between the back-end device and the user devices, a display resolution of the user devices, a processing capability of the user devices, and a display interface of the user devices.

### Claim 6 (depends on 1)

The system of claim 1, wherein to identify, based on the received device characteristics, a sampled model of the item from a database comprising the one or more sampled models is to sample the geometric to generate the sampled model when the database does not include a sampled model matching the received device characteristics.

### Claim 7 (depends on 1)

The system of claim 1, wherein to sample the geometric data to generate one or more sampled models is to select two or more three-dimensional points of the geometric data to generate the one or more sampled models having a threshold number of geometric mesh triangles or a threshold quality level based on device characteristics of one or more user devices.

### Claim 8 (depends on 1)

The system of claim 1, wherein to sample the geometric data to generate one or more sampled models is to select two or more three-dimensional points of the geometric data to generate the one or more sampled models to preserve a boundary or a topology of the base model based a user provided parameter.

### Claim 9 (depends on 1)

The system of claim 1, where the image data comprises one or more digital images of the item.

### Claim 10 (depends on 1)

The system of claim 1, wherein the image data comprises a video of the item.

### Claim 11

A method of operating a front-end processing device, the method comprising:
transmitting a request for a three-dimensional model of an item to a backend processing device;
receiving, from the backend processing device after transmitting the request, a data object comprising geometric data of the three-dimensional model and texture data for rending the three-dimensional model, the geometric data selected to have a resolution determined by the device characteristics of the frond-end processing device; and
generating a rendering of three-dimensional model on a graphic display device using a graphics processing hardware circuit of the front-end processing device and the geometric data and the texture data.

### Claim 12 (depends on 11)

The method of claim 11, wherein generating the rendering of the three-dimensional model comprises:
generating a single rendering of the three-dimensional model when the graphic display device is a standard display device; and
generating a binocular rendering of the three-dimensional model when the graphic display device is at least one of an augmented reality display device and a virtual reality display device.

### Claim 13 (depends on 11)

The method of claim 11, the method further comprising:
receiving a request to modify the rendered three-dimensional model;
modifying at the geometric data in response to the request; and
regenerating the rendering of the three-dimensional model using the graphics processing hardware based on the modified geometric data.

### Claim 14 (depends on 11)

The method of claim 11, the method further comprising:
rendering a control menu on the graphic display device;
receiving a selection of a control from the control menu in response to a user visually focusing on the control for a threshold time period; and
executing an operation associated with the selected control.

### Claim 15 (depends on 14)

The method of claim 14, wherein the control menu comprises:
a control to replace the rendered of the three-dimensional model of the item with a rendering of another three-dimensional model of another item;
a control to display additional information about the item; and
a control to change an orientation of the rendered three-dimensional model.

### Claim 16 (depends on 15)

The method of claim 15, wherein the selected control is the control to display additional information about the item, and executing an operation associated with the selected control comprises displaying a list of attributes associated with the item.

### Claim 17 (depends on 15)

The method of claim 15, wherein the selected control is the control to change the orientation of the rendered three-dimensional model, and executing an operation associated with the selected control comprises:
modifying the geometric data to of the three-dimensional model using graphics processing hardware to change the orientation of the three-dimensional mode; and
regenerating, using graphics processing hardware, the rendering of the three-dimensional model using the using the modified geometric data and the texture data.

### Claim 18 (depends on 19)

The method of claim 19, further comprising:
rendering thumbnail image of one or more objects on the graphic display device; and
wherein requesting the item includes receiving a user selection of the from the thumbnail image of the one or more objects.

### Claim 19

A non-transitory computer-readable storage medium storing a set of instructions that, when executed by at least one processor of a machine; cause the machine to perform operations comprising:
receiving image data associated with an item, the image data comprising a view of the item from two or more angles;
determining, automatically using the image data, physical attributes of the item;
generating, automatically using the physical attributes, a base model of the item, the base model comprising geometric data to generate a three-dimensional rendering of the item;
sampling the geometric data to generate one or more sampled models, each of the one or more sampled models comprising a subset of the geometric data, the subset of the geometric data determined based on one or more device characteristics of one or more user devices that interface with the back-end processing device;
receiving a device characteristic of a user device associated with a request from the user device for the item;
identifying, based on the received device characteristics, a sampled model of the item from a database comprising the one or more sampled models; and
transmitting a data object comprising the identified sampled model to the user device to cause the user device to generate a three-dimensional rendering of the item using the data object and a graphics processing hardware circuit of user device.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, operations further comprising:
sampling the geometric data to generate one or more sampled models by selecting two or more three-dimensional points of the geometric data to generate the one or more sampled models having a threshold number of geometric mesh triangles or a threshold quality level based on device characteristics of one or more user devices.
