# US 11,443,485 B2 · Rendering device, rendering method, 3D model generating method, neural network model generating method, and non-transitory computer readable medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,485 B2 - Document ID: US11443485B2 - Status: Granted - Kind: B2 - Application number: 17/100,515 - Filed: 2020-11-20 - Published: 2021-03-11 - Granted: 2022-09-13 - Earliest claimed: 2018-05-24 - Assignees: Preferred Networks, Inc. - Applicants: Preferred Networks, Inc. - Inventors: Takahiro Ando - Web page: https://patentgrep.com/patent/US11443485B2 ## CPC - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06N 3/084: Backpropagation, e.g. using gradient descent - G06T 15/00: Three-dimensional [3D] image rendering - G06T 15/80: Shading - G06N 5/046: Forward inferencing; Production systems - G06T 15/005: {General purpose rendering architectures} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A rendering device includes at least one memory and at least one processor. The at least one processor acquires information about projection data in a 2D space from information about a region of a 3D model; stores the information about the projection data in association with the information about the region in the at least one memory; and generates the projection data based on the information about the projection data. The associated information includes information associating an identifier given to a part of regions obtained by dividing the 3D model with information about a position where the part of the regions is projected in the 2D space. ## Claims ### Claim 1 A projection data generating device for generating projection data in a 2D space, comprising: at least one memory; and at least one processor configured to: acquire information about the projection data in the 2D space from information about a 3D model; store, in the at least one memory, (i) the information about the projection data in the 2D space and (ii) information associating identification information given to a part of the 3D model with a pixel where the part of the 3D model is projected in the 2D space; and generate the projection data in the 2D space based on the information about the projection data in the 2D space. ### Claim 2 (depends on 1) The projection data generating device according to claim 1, wherein the at least one processor is configured to generate the projection data using deferred shading. ### Claim 3 (depends on 2) The projection data generating device according to claim 2, wherein the at least one memory is a G buffer (Geometry buffer). ### Claim 4 (depends on 1) The projection data generating device according to claim 1, wherein the at least one processor is configured to generate the projection data by a differentiable arithmetic operation. ### Claim 5 (depends on 1) The projection data generating device according to claim 1, wherein the at least one processor is configured to generate the projection data by a neural network. ### Claim 6 (depends on 5) The projection data generating device according to claim 5, wherein the at least one processor is configured to update the neural network using information indicating an error between supervised data and the projection data. ### Claim 7 (depends on 1) The projection data generating device according to claim 1, wherein the part of the 3D model has a shape of a polygon. ### Claim 8 (depends on 1) The projection data generating device according to claim 1, wherein the information about the projection data includes information about at least one of color, normal vector, albedo, depth, distance, reflection, emission, diffusion, ambient occlusion, lightmap, or data to be used for lighting calculation. ### Claim 9 (depends on 1) The projection data generating device according to claim 1, wherein the at least one processor is configured to perform a backpropagation by propagating information indicating an error between supervised data and the projection data back to the 3D model using the associating information. ### Claim 10 (depends on 9) The projection data generating device according to claim 9, wherein the at least one processor is configured to update the 3D model based on the backpropagation. ### Claim 11 (depends on 1) The projection data generating device according to claim 1, wherein the at least one processor is configured to generate the projection data using at least one of a shading or a rendering. ### Claim 12 (depends on 11) The projection data generating device according to claim 11, wherein the at least one processor is configured to update parameters for the at least one of the shading or the rendering by using an error between supervised data and the projection data. ### Claim 13 (depends on 11) The projection data generating device according to claim 11, wherein the at least one processor is configured to generate the projection data by using a forward shading. ### Claim 14 (depends on 1) The projection data generating device according to claim 1, wherein the information about the 3D model is at least one of identifier data, or coordinate data of the part of the 3D model. ### Claim 15 (depends on 1) The projection data generating device according to claim 1, wherein the at least one processor is further configured to store, in the at least one memory, information associating identification information given to another part of the 3D model with another pixel where the another part of the 3D model is projected in the 2D space. ### Claim 16 A method for generating projection data in a 2D space comprising: acquiring, by at least one processor, information about the projection data in the 2D space from information about a 3D model; storing, by the at least one processor in at least one memory, (i) the information about the projection data in the 2D space and (ii) information associating identification information given to a part of the 3D model with a pixel where the part of the 3D model is projected in the 2D space; and generating, by the at least one processor, the projection data in the 2D space based on the information about the projection data in the 2D space. ### Claim 17 (depends on 16) The method according to claim 16, further comprising: performing, by the at least one processor, a backpropagation by propagating information indicating an error between supervised data and the projection data back to the 3D model using the associating information. ### Claim 18 (depends on 17) The method according to claim 17, further comprising: updating, by the at least one processor, the 3D model based on the backpropagation. ### Claim 19 A non-transitory computer readable medium storing therein a program which, when executed by at least one processor performs a method comprising: acquiring, by at least one processor, information about projection data in a 2D space from information about a 3D model; storing, by the at least one processor in at least one memory, (i) the information about the projection data in the 2D space and (ii) information associating identification information given to a part of the 3D model with a pixel where the part of the 3D model is projected in the 2D space; and generating, by the at least one processor, the projection data in the 2D space based on the information about the projection data in the 2D space. ### Claim 20 (depends on 19) The non-transitory computer readable medium according to claim 19, wherein the method further comprises: performing, by the at least one processor, a backpropagation by propagating information indicating an error between supervised data and the projection data back to the 3D model using the associating information. ### Claim 21 (depends on 20) The non-transitory computer readable medium according to claim 20, wherein the method further comprises: updating, by the at least one processor, the 3D model based on the backpropagation.