# US 12,182,931 B2 · System and method for rendering clothing on a two-dimensional image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,931 B2 - Document ID: US12182931B2 - Status: Granted - Kind: B2 - Application number: 17/649,758 - Filed: 2022-02-02 - Published: 2022-07-14 - Granted: 2024-12-31 - Earliest claimed: 2021-01-13 - Applicants: Snap 'N Fit, LLC - Inventors: Hein Hundal; Rob Reitzen; Larissa Posner; Andrew Raffensperger - Web page: https://patentgrep.com/patent/US12182931B2 ## CPC - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06Q 30/0621: {by configuring or customising goods or services} - G06Q 30/0643: {graphically representing goods, e.g. 3D product representation} - G06T 7/13: Edge detection - G06T 11/60: Creating or editing images; Combining images with text - G06T 15/205: {Image-based rendering} - G06T 15/506: {Illumination models} - G06T 17/205: {Re-meshing} - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 2207/30124: Fabrics; Textile; Paper - G06T 2207/30196: Human being; Person - G06T 2210/16: Cloth - G06T 2219/2021: Shape modification - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR ## Abstract A system is disclosed for rending clothing on a two-dimensional image. The system may include a processor or a computer-readable storage medium storing executable instructions thereon. In response to the processor executing the executable instructions, the processor may be configured to receive at least one depth frame of a photography subject. The processor can generate a virtual avatar based on the at least one depth frame, wherein the virtual avatar may include a digital three-dimensional mesh. The processor can receive a digital three-dimensional model of an article of clothing. The processor can manipulate the digital three-dimensional model of the article of clothing to fit on the virtual avatar, render the digital three-dimensional model of the article of clothing into a two-dimensional image, or overlay the two-dimensional image onto a two-dimensional version of the at least one depth frame. ## Claims ### Claim 1 A system for rending clothing on a two-dimensional image, comprising: a processor; and a computer-readable storage medium storing executable instructions thereon, wherein in response to the processor executing the executable instructions, the processor is configured to receive at least one depth frame of a photography subject, generate a virtual avatar based on the at least one depth frame, wherein the virtual avatar includes a digital three-dimensional mesh, receive a digital three-dimensional model of an article of clothing, manipulate the digital three-dimensional model of the article of clothing to fit on the virtual avatar, render the digital three-dimensional model of the article of clothing into a two-dimensional image, and overlay the two-dimensional image onto a two-dimensional version of the at least one depth frame; wherein manipulating the digital three-dimensional model of the article of clothing to fit on the virtual avatar comprises determining a smooth bijection between a three-dimensional space of the model of the article of clothing and a three-dimensional space of the virtual avatar. ### Claim 2 (depends on 1) The system of claim 1, wherein the at least one depth frame comprises a red-green-blue-depth (RGB-D) image. ### Claim 3 (depends on 1) The system of claim 1, wherein the digital three-dimensional mesh comprises a three-dimensional triangulated mesh representing at least a portion of the photography subject. ### Claim 4 (depends on 1) The system of claim 1, wherein the photography subject comprises at least a portion of a human body. ### Claim 5 (depends on 4) The system of claim 4, wherein the processor being configured to generate the virtual avatar comprises inputting, into the virtual avatar, at least one of: an angle of a head of the human body relative to a remainder of the human body; an angle between an upper arm of the human body and a shoulder of the human body; or an angle between the upper arm of the human body and a lower arm of the human body. ### Claim 6 (depends on 4) The system of claim 4, wherein the processor being configured to generate the virtual avatar comprises inputting, into the virtual avatar, at least one of: a difference between an orientation of shoulders of the human body and an orientation of hips of the human body; a curvature of a spine of the human body; an angle between the spine of the human body and an upper leg of the human body; or an angle between the upper leg of the human body and a lower leg of the human body. ### Claim 7 (depends on 4) The system of claim 4, wherein the processor being configured to generate the virtual avatar comprises inputting, into the virtual avatar, a length of a bone. ### Claim 8 (depends on 4) The system of claim 4, wherein the processor being configured to generate the virtual avatar comprises modifying a first portion of the virtual avatar relative to a proportion of a second portion of the virtual avatar. ### Claim 9 A computer-implemented method for rendering clothing on a two-dimensional image, comprising: receiving at least one depth frame of a photography subject; generating a virtual avatar based on the at least one depth frame, wherein the virtual avatar includes a digital three-dimensional mesh; receiving a digital three-dimensional model of an article of clothing; manipulating the digital three-dimensional model of the article of clothing to fit on the virtual avatar; calculating lighting data based on content included in the at least one depth frame; rendering the three-dimensional model of the article of clothing into a two-dimensional image based on the lighting data; and overlaying the two-dimensional image onto a two-dimensional version of the at least one depth frame; wherein manipulating the digital three-dimensional model of the article of clothing to fit on the virtual avatar comprises determining a smooth bijection between a three-dimensional space of the model of the article of clothing and a three-dimensional space of the virtual avatar. ### Claim 10 (depends on 9) The method of claim 9, wherein: receiving the at least one depth frame comprises receiving a plurality of depth frames; and the plurality of depth frames include at least one close-up image. ### Claim 11 (depends on 9) The method of claim 9, wherein calculating lighting data comprises at least one of: estimating a temperature of a lighting of at the least one depth frame; estimating an amount of ambient lighting of at the least one depth frame; estimating a direction of the lighting of at the least one depth frame; or estimating an intensity of direct lighting of at the least one depth frame. ### Claim 12 (depends on 9) The method of claim 9, wherein rendering the three-dimensional model of the article of clothing into a two-dimensional model comprises edge detection of the at least one depth frame. ### Claim 13 (depends on 12) The method of claim 12, wherein the edge detection comprises executing an artificial neural network on the two-dimensional version of the at least one depth frame. ### Claim 14 (depends on 9) The method of claim 9, further comprising receiving a body measurement. ### Claim 15 (depends on 14) The method of claim 14, wherein the body measurement comprises at least one of: a height of the photography subject; a waist size of the photography subject; or a neck size of the photography subject. ### Claim 16 (depends on 9) The method of claim 9, further comprising overlaying a portion of the depth frame over the two-dimensional image, wherein the portion of the depth frame includes a portion of the photography subject. ### Claim 17 A system for-rending rendering an item on a two-dimensional image, comprising: a processor; and a computer-readable storage medium storing executable instructions thereon, wherein in response to the processor executing the executable instructions, the processor is configured to receive at least one depth frame of a photography subject, generate a virtual avatar based on the at least one depth frame, wherein the virtual avatar includes a digital three-dimensional mesh, receive a digital three-dimensional model of an item, manipulate the digital three-dimensional model of the item to fit on the virtual avatar, calculate lighting data based on content included in the at least one depth frame, render the three-dimensional model of the item into a two-dimensional image based on the lighting data, and overlay the two-dimensional image onto a two-dimensional version of the at least one depth frame; wherein the virtual avatar comprises a plurality of frustums and a plurality of ellipsoids. ### Claim 18 (depends on 17) The system of claim 17, wherein the photography subject comprises a piece of furniture.