# US 10,997,779 B2 · Method of generating an image file of a 3D body model of a user wearing a garment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,997,779 B2 - Document ID: US10997779B2 - Status: Granted - Kind: B2 - Application number: 16/488,788 - Filed: 2018-02-27 - Published: 2020-02-27 - Granted: 2021-05-04 - Earliest claimed: 2017-02-27 - Assignees: Metail Limited - Applicants: Metail Limited - Inventors: Yu Chen; Sukrit Shankar; Dongjoe Shin; David Chalmers; Jim Downing; Tom Adeyoola - Web page: https://patentgrep.com/patent/US10997779B2 ## CPC - G06T 15/205: {Image-based rendering} - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 7/11: Region-based segmentation - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 15/04: Texture mapping - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30196: Human being; Person - G06T 2210/16: Cloth - G06T 2219/2004: Aligning objects, relative positioning of parts - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A computer-implemented method of generating an image file of a 3D body model of a user wearing a garment, comprising: (i) receiving one or more two dimensional images of a model wearing a garment, which images provide a view of an outer surface of the garment; (ii) for each two dimensional image, segmenting an image of the garment to produce a set of segmented garment images; (iii) using the set of segmented garment images to generate a complete 3D garment model; (iv) receiving a 3D body model of a user; (v) simulating the complete 3D garment model worn on the 3D body model of the user and, (vi) generating an image file of the 3D body model of the user wearing the complete 3D garment model, using the simulated complete 3D garment model worn on the 3D body model of the user. ## Claims ### Claim 1 A computer-implemented method of generating an image file of a 3D body model of a user wearing a garment, the method comprising the computer-implemented steps of: (i) receiving one or more two dimensional images of a model wearing a garment, wherein in at least one two dimensional image the model is in an occluded pose, wherein the one or more two dimensional images of the model wearing the garment provide a view of an outer surface of the garment; (ii) for each two dimensional image, segmenting an image of the garment to produce a set of segmented garment images; (iii) using the set of segmented garment images to generate a complete 3D garment model; (iv) receiving a 3D body model of a user; (v) simulating the complete 3D garment model worn on the 3D body model of the user and, (vi) generating an image file of the 3D body model of the user wearing the complete 3D garment model, using the simulated complete 3D garment model worn on the 3D body model of the user. ### Claim 2 (depends on 1) The method of claim 1, wherein the view of an outer surface of the garment is an incomplete view of an outer surface of the garment, and wherein step (iii) includes filling-in a 3D garment model to generate the complete 3D garment model. ### Claim 3 (depends on 1) The method of claim 1, the method including: deriving a set of texture samples of the garment from the set of segmented garment images; using the set of segmented garment images to generate an incomplete 3D garment model; filling-in the incomplete 3D garment model to generate a complete 3D garment model, including filling-in a texture of the complete 3D garment model using the texture samples, to create a coherent texture of the complete 3D garment model, the complete 3D garment model including the coherent texture of the complete 3D garment model. ### Claim 4 (depends on 1) The method of claim 1, in which any occluded sections of the garment within the one or more two dimensional images of the model wearing the garment are identified and recreated automatically to produce a non-occluded 3D garment model. ### Claim 5 (depends on 1) The method of claim 1, in which at least one of a patch based algorithm, and a diffusion based algorithm, is used. ### Claim 6 (depends on 1) The method of claim 1, including predicting automatically a visibility of and locations of one or more garment landmarks and/or splines on the model within an image. ### Claim 7 (depends on 1) The method of claim 1, including acquiring a plurality of model images of the model where one or a plurality of viewing angles and one or a plurality of body poses, are used. ### Claim 8 (depends on 1) The method of claim 1, including using a deep neural network model for generating the 3D garment model, or for generating the complete 3D garment model. ### Claim 9 (depends on 1) The method of claim 1, including acquiring the one or more two dimensional images of the model wearing the garment from at least one of: an internet web site, a digital camera device, a computer aided design, a streaming data feed. ### Claim 10 (depends on 1) The method of claim 1, including using a data processing arrangement comprising a first subsystem which creates a digital garment model of a garment from 2D images of the garment, and a second subsystem which simulates the digital garment model onto a 3D body avatar of a consumer. ### Claim 11 (depends on 10) The method of claim 10, including using the data processing arrangement to receive inputs from the user, wherein the inputs include one or more consumer body metrics, and using the second subsystem to generate automatically a 3D body avatar representing the consumer's body shape. ### Claim 12 (depends on 10) The method of claim 10, including using the second subsystem to create the 3D body avatar representing the user's body shape using a 3D statistical body model with an embedded skeleton and a regression application. ### Claim 13 (depends on 10) The method of claim 10, including using the data processing system to receive inputs from the user to define a preferred body pose, and to configure the second subsystem to create automatically an identical body pose for the 3D body avatar. ### Claim 14 (depends on 10) The method of claim 10, including using the second subsystem to select automatically one or more digital garment models from a garment database which match a consumer's garment choice input. ### Claim 15 (depends on 14) The method of claim 14, including using the second subsystem to simulate automatically a drape and a fit of the digital garment model onto the 3D body avatar. ### Claim 16 (depends on 15) The method of claim 15, including using the second subsystem to simulate the drape and the fit of the digital garment model onto the 3D body avatar using finite-element analysis. ### Claim 17 (depends on 15) The method of claim 15, including using the second subsystem to composite automatically multiple individual garment models in a correct layering order and then to display the 3D body avatar in an outfit of one or more selected garments. ### Claim 18 (depends on 10) The method of claim 10, including the step of the first subsystem recognizing body poses in the one or more two dimensional images of the model wearing the garment, and automatically matching a nearest pose in a pre-defined pose pool and selecting appropriate body poses for digitization. ### Claim 19 (depends on 1) The method of claim 1, in which for each two dimensional image, the segmenting of an image of the garment to produce a segmented garment image is performed automatically. ### Claim 20 A system arranged to generate an image file of a 3D body model of a user wearing a garment, the system including a processor configured to: (i) receive one or more two dimensional images of a model wearing a garment, wherein in at least one two dimensional image the model is in an occluded pose, wherein the one or more two dimensional images of the model wearing the garment provide a view of an outer surface of the garment; (ii) for each two dimensional image, segment an image of the garment to produce a set of segmented garment images; (iii) use the set of segmented garment images to generate a complete 3D garment model; (iv) receive a 3D body model of a user; (v) simulate the complete 3D garment model being worn on the 3D body model of the user and, (vi) generate an image file of the 3D body model of the user wearing the complete 3D garment model, using the simulated complete 3D garment model being worn on the 3D body model of the user. ### Claim 21 A computer program product, executable on a processor to perform a method of generating an image file of a 3D body model of a user wearing a garment, the computer program product executable on the processor to: (i) receive one or more two dimensional images of a model wearing a garment, wherein in at least one two dimensional image the model is in an occluded pose, wherein the one or more two dimensional images of the model wearing the garment provide a view of an outer surface of the garment; (ii) for each two dimensional image, segment an image of the garment to produce a set of segmented garment images; (iii) use the set of segmented garment images to generate a complete 3D garment model; (iv) receive a 3D body model of a user; (v) simulate the complete 3D garment model worn on the 3D body model of the user and, (vi) generate an image file of the 3D body model of the user wearing the complete 3D garment model, using the simulated complete 3D garment model worn on the 3D body model of the user.