# US 11,308,669 B1 · Shader for graphical objects > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,669 B1 - Document ID: US11308669B1 - Status: Granted - Kind: B1 - Application number: 17/007,673 - Filed: 2020-08-31 - Granted: 2022-04-19 - Earliest claimed: 2019-09-27 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Shem Nguyen; Austin Caleb Germer - Web page: https://patentgrep.com/patent/US11308669B1 ## CPC - G06T 11/60: Creating or editing images; Combining images with text - G06T 7/507: from shading (G06T7/586 takes precedence) - G06T 11/00: Two-dimensional [2D] image generation - G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) - G06T 11/40: Filling planar surfaces by adding surface attributes, e.g. adding colours or textures - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A device includes a non-transitory memory and one or more processors coupled with the non-transitory memory. The device performs a method that includes detecting, in a set of one or more images, appearance features that define an appearance of an object represented in the set of one or more images. The appearance includes one or more shading effects associated with the object. The method includes generating a feature map for the object that encodes the detected appearance features of the object. The method includes displaying, based on the feature map, a computer-generated graphical object with one or more simulated shading effects that are within a degree of similarity to the one or more shading effects associated with the object. ## Claims ### Claim 1 A method comprising: at a device including a display, a non-transitory memory and one or more processors coupled with the display and the non-transitory memory: detecting, in a set of one or more images, appearance features that define an appearance of an object represented in the set of one or more images, wherein the appearance includes one or more hand-drawn shading effects associated with the object; generating a feature map for the object that encodes the detected appearance features of the object; and displaying, based on the feature map, a computer-generated graphical environment including a computer-generated graphical object with one or more simulated shading effects that are generated by varying a simulated light source of the computer-generated graphical environment and the one or more simulated shading effects are within a degree of similarity to the one or more hand-drawn shading effects associated with the object. ### Claim 2 (depends on 1) The method of claim 1, wherein displaying the computer-generated graphical object comprises performing deferred neural rendering in order to display the computer-generated graphical object. ### Claim 3 (depends on 1) The method of claim 1, further comprising: providing the feature map as an input to a deferred neural renderer that generates the one or more simulated shading effects for the computer-generated graphical object. ### Claim 4 (depends on 3) The method of claim 3, wherein the deferred neural renderer includes an object-specific deferred neural renderer that generates simulated shading effects for computer-generated graphical objects that are within a degree of similarity to the object. ### Claim 5 (depends on 3) The method of claim 3, wherein the deferred neural renderer includes a generalized deferred neural renderer that generates simulated shading effects for another computer-generated graphical object that is beyond a degree of similarity to the object. ### Claim 6 (depends on 1) The method of claim 1, wherein varying the simulated light source includes changing a direction of the simulated light source in order to generate the one or more simulated shading effects. ### Claim 7 (depends on 1) The method of claim 1, wherein varying the simulated light source includes changing an intensity of the simulated light source in order to generate the one or more simulated shading effects. ### Claim 8 (depends on 1) The method of claim 1, wherein varying the simulated light source includes changing a color of the simulated light source in order to generate the one or more simulated shading effects. ### Claim 9 (depends on 1) The method of claim 1, wherein varying the simulated light source includes changing a type of the simulated light source in order to generate the one or more simulated shading effects. ### Claim 10 (depends on 9) The method of claim 9, wherein changing the type of the simulated light source includes switching between a simulated ambient light source, a simulated directional light source, a simulated point light source, a simulated spotlight light source, a simulated area light source and a simulated volumetric light source. ### Claim 11 (depends on 1) The method of claim 1, wherein the appearance features indicate one or more textures of the object. ### Claim 12 (depends on 1) The method of claim 1, wherein the appearance features include neural textures that are detected by a neural network system. ### Claim 13 (depends on 1) The method of claim 1, wherein detecting the appearance features includes providing the set of one or more images as an input to a machine-learned model that extracts the appearance features from the set of one or more images. ### Claim 14 (depends on 1) The method of claim 1, wherein detecting the appearance features comprises applying a constant sampling rate to a first image in the set of one or more images. ### Claim 15 (depends on 1) The method of claim 1, wherein detecting the appearance features comprises varying a sampling rate across a first image in the set of one or more images based on a depth associated with pixels in the first image. ### Claim 16 (depends on 1) The method of claim 1, wherein the appearance features include neural textures of the object, and the feature map includes a neural texture map of the object. ### Claim 17 (depends on 1) The method of claim 1, further comprising: determining respective texel-to-pixel mappings for the images in the set of one or more images; and detecting the appearance features based on the respective texel-to-pixel mappings. ### Claim 18 (depends on 1) The method of claim 1, wherein the one or more hand-drawn shading effects include hatching, smoothing or stippling that is associated with the object. ### Claim 19 A device comprising: one or more processors; a non-transitory memory; one or more displays; and one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to: detect, in a set of one or more images, appearance features that define an appearance of an object represented in the set of one or more images, wherein the appearance includes hatching, smoothing or stippling that is associated with the object and neural textures that are detected by a neural network system; generate a feature map for the object that encodes the detected appearance features of the object; and display, based on the feature map, a computer-generated graphical object with simulated hatching that is within a degree of similarity to the hatching, simulated smoothing that is within a degree of similarity to the smoothing or simulated stippling that is within a degree of similarity to the stippling. ### Claim 20 (depends on 19) The device of claim 19, wherein the one or more simulated shading effects are generated by varying a simulated light source of a computer-generated graphical environment. ### Claim 21 (depends on 20) The device of claim 20, wherein varying the simulated light source comprises changing a direction of the simulated light source, changing an intensity of the simulated light source, changing a color of the simulated light source or changing a type of the simulated light source. ### Claim 22 (depends on 19) The device of claim 19, wherein displaying the computer-generated graphical object comprises performing deferred neural rendering in order to display the computer-generated graphical object. ### Claim 23 A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device with a display, cause the device to: detect, in a set of one or more images, appearance features that define an appearance of an object represented in the set of one or more images, wherein the appearance includes hatching, smoothing or stippling that is associated with the object and neural textures that are detected by a neural network system; generate a feature map for the object that encodes the detected appearance features of the object; and display, based on the feature map, a computer-generated graphical object with simulated hatching that is within a degree of similarity to the hatching, simulated smoothing that is within a degree of similarity to the smoothing or simulated stippling that is within a degree of similarity to the stippling. ### Claim 24 (depends on 23) The non-transitory memory of claim 23, wherein the one or more simulated shading effects are generated by varying a simulated light source of a computer-generated graphical environment. ### Claim 25 (depends on 24) The non-transitory memory of claim 24, wherein varying the simulated light source comprises changing a direction of the simulated light source, changing an intensity of the simulated light source, changing a color of the simulated light source or changing a type of the simulated light source. ### Claim 26 (depends on 23) The non-transitory memory of claim 23, wherein displaying the computer-generated graphical object comprises performing deferred neural rendering in order to display the computer-generated graphical object.