# US 11,443,491 B2 · 3D object camera customization system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,491 B2 - Document ID: US11443491B2 - Status: Granted - Kind: B2 - Application number: 17/249,092 - Filed: 2021-02-19 - Published: 2021-06-10 - Granted: 2022-09-13 - Earliest claimed: 2019-06-28 - Assignees: Snap Inc. - Applicants: Snap Inc. - Inventors: Samuel Edward Hare; Andrew James McPhee; Maxim Maximov Lazarov; Wentao Shang; Kyle Goodrich; Tony Mathew - Web page: https://patentgrep.com/patent/US11443491B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 15/04: Texture mapping - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - H04L 51/04: Real-time or near real-time messaging, e.g. instant messaging [IM] - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2219/2016: Rotation, translation, scaling - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems and methods are provided for capturing by a camera of a user device, a first image depicting a first environment of the user device; overlaying a first virtual object on a portion of the first image depicting the first environment; modifying a surface of the first virtual object using content captured by the user device; storing a second virtual object comprising the first virtual object with the modified surface; and generating for display the second virtual object on a portion of a second image depicting a second environment. ## Claims ### Claim 1 A method comprising: capturing, by a camera of a client device, a first image depicting a first environment of the client device and a user of the client device; overlaying, by one or more processors of the client device, a first virtual object on a portion of the first image depicting the first environment, the first virtual object comprising a virtual garment positioned on top of the user depicted in the first image; and modifying a surface of the first virtual object such that a portion of the virtual garment depicts a face of the user or surroundings of the user. ### Claim 2 (depends on 1) The method of claim 1, further comprising: storing a second virtual object comprising the first virtual object with the modified surface; and generating, for display, the second virtual object on a portion of a second image depicting a second environment. ### Claim 3 (depends on 1) The method of claim 1, wherein modifying the surface comprises: identifying a first pixel of the portion of the image; retrieving a pixel value of the first pixel; and copying the retrieved pixel value to a surface pixel of the first virtual object. ### Claim 4 (depends on 1) The method of claim 1, wherein the first virtual object is displayed within a camera feed captured using a messaging application of the client device of a second environment. ### Claim 5 (depends on 1) The method of claim 1, wherein the first virtual object is stored in response to receiving input confirming a position of the first virtual object in the first image. ### Claim 6 (depends on 1) The method of claim 1, wherein the client device is a first client device, and further comprising sending the first virtual object to a second client device via a messaging application of the first client device. ### Claim 7 (depends on 1) The method of claim 1, wherein a second image depicting a second environment is captured by a second client device, and further comprising causing generating for display the first virtual object on the portion of the second image depicting the second environment, using a messaging application of the second client device, in response to receiving the first virtual object from the first client device. ### Claim 8 (depends on 1) The method of claim 1 further comprising: receiving user input comprising a plurality of characters of a string; and generating a second virtual object comprising the string as each character in the string is received in the user input, the second virtual object comprising a three-dimensional caption that includes a surface modified to represent one or more visual components of the first environment depicted in the first image. ### Claim 9 (depends on 1) The method of claim 1, wherein: modifying the surface of the first virtual object comprises presenting a second video on the first virtual object. ### Claim 10 (depends on 9) The method of claim 9, wherein the client device is a first client device, and further comprising: causing capturing a third video using a camera of a second client device; and presenting the first virtual object with the second video on top of the third video on the second client device. ### Claim 11 (depends on 1) The method of claim 1, wherein the camera of the client device comprises a front-facing camera of the client device, wherein the first image is a first of a plurality of images of a first video captured by the front-facing camera of the client device, wherein the face of the user is captured by the front-facing camera, further comprising: displaying the first virtual object comprising the modified surface depicting the face of the user captured by the front-facing camera within a second video captured by a rear-facing camera of the client device. ### Claim 12 (depends on 1) The method of claim 1, wherein the virtual garment comprises a virtual hat, wherein the virtual hat is positioned on top of a head of the user depicted in the first image. ### Claim 13 A system comprising: one or more processors; a memory storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising: capturing, by a camera of a client device, a first image depicting a first environment of the client device and a user of the client device; overlaying, by one or more processors of the client device, a first virtual object on a portion of the first image depicting the first environment, the first virtual object comprising a virtual garment positioned on top of the user depicted in the first image; and modifying a surface of the first virtual object such that a portion of the virtual garment depicts a face of the user or surroundings of the user. ### Claim 14 (depends on 13) The system of claim 13, wherein the operations further comprise: storing a second virtual object comprising the first virtual object with the modified surface; and generating, for display, the second virtual object on a portion of a second image depicting a second environment. ### Claim 15 (depends on 13) The system of claim 13, wherein modifying the surface comprises: identifying a first pixel of the portion of the image; retrieving a pixel value of the first pixel; and copying the retrieved pixel value to a surface pixel of the first virtual object. ### Claim 16 (depends on 13) The system of claim 13, wherein the first virtual object is displayed within a camera feed captured using a messaging application of the client device of a second environment. ### Claim 17 (depends on 13) The system of claim 13, wherein the first virtual object is stored in response to receiving input confirming a position of the first virtual object in the first image. ### Claim 18 (depends on 13) The system of claim 13, wherein the client device is a first client device, and further comprising sending the first virtual object to a second client device via a messaging application of the first client device. ### Claim 19 (depends on 13) The system of claim 13, wherein the virtual garment comprises a virtual hat, wherein the virtual hat is positioned on top of a head of the user depicted in the first image. ### Claim 20 A non-transitory machine-readable medium storing non-transitory instructions which, when executed by one or more processors of a machine, cause the machine to perform operations comprising: capturing, by a camera of a client device, a first image depicting a first environment of the client device and a user of the client device; overlaying, by one or more processors of the client device, a first virtual object on a portion of the first image depicting the first environment, the first virtual object comprising a virtual garment positioned on top of the user depicted in the first image; and modifying a surface of the first virtual object such that a portion of the virtual garment depicts a face of the user or surroundings of the user.