# US 11,113,869 B2 · Techniques for generating visualizations of ray tracing images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,869 B2 - Document ID: US11113869B2 - Status: Granted - Kind: B2 - Application number: 16/731,634 - Filed: 2019-12-31 - Published: 2020-04-30 - Granted: 2021-09-07 - Earliest claimed: 2018-03-02 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Austin Neil Kinross; Shawn Lee Hargreaves; Amar Patel; Thomas Lee Davidson - Web page: https://patentgrep.com/patent/US11113869B2 ## CPC - G06T 15/06: Ray-tracing - G06T 15/005: {General purpose rendering architectures} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T 2210/12: Bounding box - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Examples described herein generally relate to generating a visualization of an image. A proprietary structure that specifies ray tracing instructions for generating the image using ray tracing is intercepted from a graphics processing unit (GPU) or a graphics driver. The proprietary structure can be converted, based on assistance information, to a visualization structure for generating the visualization of the image. The visualization of the image can be generated from the visualization structure. ## Claims ### Claim 1 A method for providing a structure to a visualization application, comprising: obtaining, by a graphics processing unit (GPU) and based on a request from a different application executing on a central processing unit (CPU), a structure that specifies ray tracing instructions for generating an image for the different application; and providing, by the GPU and based on a request from the visualization application, the structure to the visualization application, wherein the visualization application is a separate application from the different application and is executing on the CPU or a different CPU to separately generate the image. ### Claim 2 (depends on 1) The method of claim 1, wherein providing the structure to the visualization application is based on accessing, by the GPU, a memory to store the structure for generating the image. ### Claim 3 (depends on 1) The method of claim 1, wherein providing the structure to the visualization application comprises calling a function registered by the visualization application for receiving the structure. ### Claim 4 (depends on 1) The method of claim 1, further comprising performing, by the GPU, ray tracing of the image based on receiving the structure from a graphics driver. ### Claim 5 A method for providing a visualization structure to a visualization application, comprising: receiving, by a graphics driver and from the visualization application, a structure proprietary to the graphics driver that specifies ray tracing instructions for generating an image by a graphics processing unit (GPU) based on a request from a different application executing on a central processing unit (CPU); converting the structure to the visualization structure for use by the visualization application; and providing, to the visualization application, the visualization structure, wherein the visualization application is a separate application from the different application and is executing on the CPU or a different CPU to separately generate the image. ### Claim 6 (depends on 5) The method of claim 5, further comprising: generating, by the graphics driver and from an acceleration structure received from the different application, the structure proprietary to the graphics driver; and providing the structure to the GPU for ray tracing the image, wherein converting the structure to the visualization structure includes generating the visualization structure as at least a portion of the acceleration structure. ### Claim 7 (depends on 6) The method of claim 6, wherein converting the structure to the visualization structure comprises optimizing away one or more features of the acceleration structure. ### Claim 8 (depends on 7) The method of claim 7, wherein the one or more features include one or more object layers defined by the acceleration structure that are not displayed in ray tracing the image. ### Claim 9 (depends on 5) The method of claim 5, further comprising receiving assistance information of one or more intersection shaders for the structure, wherein converting the structure to the visualization structure comprises applying the one or more intersection shaders to the structure to generate the visualization structure. ### Claim 10 (depends on 9) The method of claim 9, wherein applying the one or more intersection shaders comprises applying a shape or orientation to objects of the image in bounding boxes. ### Claim 11 (depends on 9) The method of claim 9, wherein receiving the assistance information comprises receiving the assistance information from the visualization application. ### Claim 12 A computing device for providing a structure to a visualization application, comprising: a memory for storing a structure that specifies ray tracing instructions for generating an image; and at least one graphics processing unit (GPU) coupled to the memory, wherein the at least one GPU is configured to: obtain, based on a request from a different application executing on a central processing unit (CPU), a structure that specifies ray tracing instructions for generating an image for the different application; and providing, based on a request from the visualization application, the structure to the visualization application, wherein the visualization application is a separate application from the different application and is executing on the CPU or a different CPU to separately generate the image. ### Claim 13 (depends on 12) The computing device of claim 12, wherein the at least one GPU is configured to provide the structure to the visualization application based on accessing, by the GPU, the memory to store the structure for generating the image. ### Claim 14 (depends on 12) The computing device of claim 12, wherein the at least one GPU is configured to provide the structure to the visualization application at least in part by calling a function registered by the visualization application for receiving the structure. ### Claim 15 (depends on 12) The computing device of claim 12, wherein the at least one GPU is further configured to provide perform ray tracing of the image based on receiving the structure from a graphics driver. ### Claim 16 A computing device for providing a visualization structure to a visualization application, comprising: a memory storing one or more parameters or instructions for executing an operating system and a graphics driver for communicating with a graphics processing unit (GPU); and at least one processor coupled to the memory, wherein the at least one processor is configured to: receive, from the visualization application, a structure proprietary to the graphics driver that specifies ray tracing instructions for generating an image by the GPU based on a request from a different application executing on a central processing unit (CPU); convert the structure to the visualization structure for use by the visualization application; and provide, to the visualization application, the visualization structure, wherein the visualization application is a separate application from the different application and is executing on the CPU or a different CPU to separately generate the image. ### Claim 17 (depends on 16) The computing device of claim 16, wherein the at least one processor is further configured to: generate, from an acceleration structure received from the different application, the structure proprietary to the graphics driver; and provide the structure to the GPU for ray tracing the image, wherein the at least one processor is configured to convert the structure to the visualization structure at least in part by generating the visualization structure as at least a portion of the acceleration structure. ### Claim 18 (depends on 17) The computing device of claim 17, wherein the at least one processor is configured to convert the structure to the visualization structure at least in part by optimizing away one or more features of the acceleration structure. ### Claim 19 (depends on 18) The computing device of claim 18, wherein the one or more features include one or more object layers defined by the acceleration structure that are not displayed in ray tracing the image. ### Claim 20 (depends on 16) The computing device of claim 16, wherein the at least one processor is further configured to receive assistance information of one or more intersection shaders for the structure, wherein the at least one processor is configured to convert the structure to the visualization structure at least in part by applying the one or more intersection shaders to the structure to generate the visualization structure. ### Claim 21 (depends on 20) The computing device of claim 20, wherein the at least one processor is configured to apply the one or more intersection shaders at least in part by applying a shape or orientation to objects of the image in bounding boxes. ### Claim 22 (depends on 20) The computing device of claim 20, wherein the at least one processor is configured to receive the assistance information from the visualization application.