# US 12,602,862 B2 · Light normalization in combined 3D user representations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,862 B2 - Document ID: US12602862B2 - Status: Granted - Kind: B2 - Application number: 18/389,319 - Filed: 2023-11-14 - Published: 2024-07-04 - Granted: 2026-04-14 - Earliest claimed: 2022-12-29 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Borja Morales Hernando; Michael S. Hutchinson - Web page: https://patentgrep.com/patent/US12602862B2 ## CPC - G06T 15/506: {Illumination models} - G06T 15/04: Texture mapping - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 2219/2004: Aligning objects, relative positioning of parts - G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Various implementations disclosed herein include devices, systems, and methods that adjust a combined user representation via light normalization techniques. For example, a process may include obtaining a first user representation of at least a first portion of a user generated via a first technique based on the user in a first lighting condition in a first physical environment. The process may further include obtaining a second user representation of at least a second portion of the user, the second user representation being generated by generating and delighting an initial user representation based on a lighting representation of a second physical environment having a second lighting condition, and producing the second user representation by relighting the delighted initial user representation based on the first lighting condition. The process may further include generating combined user representation based on the first user representation and the second user representation. ## Claims ### Claim 1 A method comprising: at a processor of a device: obtaining a first user representation of at least a first portion of a user, wherein the first user representation is generated via a first technique based on first image data obtained via a first set of sensors in a first physical environment and the first user representation represents the user in a first lighting condition; obtaining a second user representation of at least a second portion of the user, wherein the second user representation was generated by: generating an initial user representation of the user based on second image data obtained via a second set of sensors in a second physical environment having a second lighting condition; delighting the initial user representation based on a lighting representation of the second physical environment; and producing the second user representation by relighting the delighted initial user representation based on the first lighting condition; and generating a combined user representation based on the first user representation and the second user representation. ### Claim 2 (depends on 1) The method of claim 1, wherein the first user representation comprises texture data produced via a machine learning model trained using training data obtained via one or more sensors in one or more environments having in the first lighting condition. ### Claim 3 (depends on 2) The method of claim 2, wherein the first lighting condition is provided in the one or more environments using a plurality of lights positioned at fixed positional relationships to provide even light distribution across faces of training subjects. ### Claim 4 (depends on 1) The method of claim 1, wherein the lighting representation of the second physical environment comprises an omnidirectional image representation of the second lighting condition of the second physical environment. ### Claim 5 (depends on 4) The method of claim 4, wherein the omnidirectional image representation of the second lighting condition of the second physical environment is generated by a machine learning model based on the second image data obtained via sensor data. ### Claim 6 (depends on 1) The method of claim 1, wherein relighting the second user representation comprises matching lighting attributes of the second lighting condition to lighting attributes of the first user representation. ### Claim 7 (depends on 1) The method of claim 1, wherein the first lighting condition of the first physical environment is different than the second lighting condition of the second physical environment. ### Claim 8 (depends on 1) The method of claim 1, further comprising: providing a view of an adjusted combined user representation in a three-dimensional (3D) environment, wherein the adjusted combined user representation is produced by adjusting the combined user representation based on at least one of one or more color attributes or one or more light attributes of the 3D environment. ### Claim 9 (depends on 1) The method of claim 1, wherein the first physical environment is different than the second physical environment. ### Claim 10 (depends on 1) The method of claim 1, wherein the first portion includes a representation of the face and hair of the user and the second portion represents other parts of the user different from the first portion. ### Claim 11 (depends on 1) The method of claim 1, wherein the combined user representations is a three-dimensional (3D) user representation. ### Claim 12 A device comprising: a non-transitory computer-readable storage medium; and one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: obtaining a first user representation of at least a first portion of a user, wherein the first user representation is generated via a first technique based on first image data obtained via a first set of sensors in a first physical environment and the first user representation represents the user in a first lighting condition; obtaining a second user representation of at least a second portion of the user, wherein the second user representation was generated by: generating an initial user representation of the user based on second image data obtained via a second set of sensors in a second physical environment having a second lighting condition; delighting the initial user representation based on a lighting representation of the second physical environment; and producing the second user representation by relighting the delighted initial user representation based on the first lighting condition; and generating a combined user representation based on the first user representation and the second user representation. ### Claim 13 (depends on 12) The device of claim 12, wherein the first user representation comprises texture data produced via a machine learning model trained using training data obtained via one or more sensors in one or more environments having in the first lighting condition. ### Claim 14 (depends on 13) The device of claim 13, wherein the first lighting condition is provided in the one or more environments using a plurality of lights positioned at fixed positional relationships to provide even light distribution across faces of training subjects. ### Claim 15 (depends on 12) The device of claim 12, wherein the lighting representation of the second physical environment comprises an omnidirectional image representation of the second lighting condition of the second physical environment. ### Claim 16 (depends on 15) The device of claim 15, wherein the omnidirectional image representation of the second lighting condition of the second physical environment is generated by a machine learning model based on the second image data obtained via sensor data. ### Claim 17 (depends on 12) The device of claim 12, wherein relighting the second user representation comprises matching lighting attributes of the second lighting condition to lighting attributes of the first user representation. ### Claim 18 (depends on 12) The device of claim 12, wherein the first lighting condition of the first physical environment is different than the second lighting condition of the second physical environment. ### Claim 19 (depends on 12) The device of claim 12, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, further cause the one or more processors to perform operations comprising: providing a view of an adjusted combined user representation in a three-dimensional (3D) environment, wherein the adjusted combined user representation is produced by adjusting the combined user representation based on at least one of one or more color attributes or one or more light attributes of the 3D environment. ### Claim 20 A non-transitory computer-readable storage medium, storing program instructions executable on a device to perform operations comprising: obtaining a first user representation of at least a first portion of a user, wherein the first user representation is generated via a first technique based on first image data obtained via a first set of sensors in a first physical environment and the first user representation represents the user in a first lighting condition; obtaining a second user representation of at least a second portion of the user, wherein the second user representation was generated by: generating an initial user representation of the user based on second image data obtained via a second set of sensors in a second physical environment having a second lighting condition; delighting the initial user representation based on a lighting representation of the second physical environment; and producing the second user representation by relighting the delighted initial user representation based on the first lighting condition; and generating a combined user representation based on the first user representation and the second user representation.