Back to results
US 10,475,250 B1Granted
Virtual item simulation using detected surfaces
AssigneeHouzz, Inc.
InventorsXiaoyi Huang, John DeCorato, Yi Wu
Life of this patent
Estimating the term…
Abstract
Disclosed are various embodiments for simulating one or more virtual objects (e.g., renders) based on characteristics of detected surfaces. Mask data may be generated which can be used to delineate borders depicted in a given image. The mask data can be refined based on a quantity of depicted image features (e.g., edges). The mask data can be used to generate one or more planes on which virtual items can be applied and rendered.
Claims20
A method comprising:
- receiving, on a client device, a selection of a portion of an image displayed on the client device;
- determining, on the client device, orientation information of a surface depicted in the selected portion of the image;
- storing, on the client device, an image mask generated using a neural network, the image mask assigning a portion of pixels in the image into surface category corresponding to the depicted surface, the image masking comprising an interface between the portion of pixels in the surface category and other pixels of the image that are not in the surface category;
- rendering, on the client device, a three-dimensional virtual item on a finite plane in a three-dimensional virtual environment, the finite plane oriented using the orientation information, the finite plane having a side terminating at the interface of the image mask;
- dynamically updating, on the client device, a display depicting the rendered 3D virtual item overlaid on a video sequence such that an orientation of the rendered 3D virtual item with respect to the finite plane is maintained as the client device moves.
2 more elements
A system comprising:
- one or more processors of a client device;
- and
- a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
- receiving a selection of a portion of an image displayed on a client device;
- determining orientation information of a surface depicted in the selected portion of the image;
- storing, in the memory, an image mask generated using a neural network, the image mask assigning a portion of pixels in the image into surface category corresponding to the depicted surface, the image masking comprising an interface between the portion of pixels in the surface category and other pixels of the image that are not in the surface category;
- rendering a three-dimensional virtual item on a finite plane in a three-dimensional virtual environment, the finite plane oriented using the orientation information, the finite plane having a side terminating at the interface of the image mask;
- dynamically updating a display depicting the rendered 3D virtual item overlaid on a video sequence such that an orientation of the rendered 3D virtual item with respect to the finite plane is maintained as the system physically moves.
5 more elements
A machine-readable storage device embodying instructions that, when executed by a device, cause the device to perform operations comprising:
- receiving a selection of a portion of an image displayed on the device;
- determining orientation information of a surface depicted in the selected portion of the image;
- storing, in the memory, an image mask generated using a neural network, the image mask assigning a portion of pixels in the image into surface category corresponding to the depicted surface, the image masking comprising an interface between the portion of pixels in the surface category and other pixels of the image that are not in the surface category;
- rendering a three-dimensional virtual item on a finite plane in a three-dimensional virtual environment, the finite plane oriented using the orientation information, the finite plane having a side terminating at the interface of the image mask;
- dynamically updating a display depicting the rendered 3D virtual item overlaid on a video sequence such that an orientation of the rendered 3D virtual item with respect to the finite plane is maintained as the device physically moves.
2 more elements
Dependencies are inferred from the claim text.