# US 2021/0279942 A1 · Method of rendering a volume and a surface embedded in the volume > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279942 A1 - Document ID: US20210279942A1 - Status: Published application - Kind: A1 - Application number: 17/154,568 - Filed: 2021-01-21 - Published: 2021-09-09 - Earliest claimed: 2020-03-09 - Assignees: Siemens Healthcare GmbH - Applicants: Siemens Healthcare GmbH - Inventors: Kaloian Petkov; Christoph Vetter; Rishabh Shah - Web page: https://patentgrep.com/patent/US20210279942A1 ## CPC - G06T 15/08: Volume rendering - G06T 7/50: Depth or shape recovery - G06T 15/04: Texture mapping - G06T 15/06: Ray-tracing - G06T 15/30: Clipping - G06T 15/80: Shading - G06T 2210/41: Medical - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Computer-implemented methods for rendering a volumetric dataset and a surface embedded in the volumetric dataset are described. One method includes performing a volume rendering process to generate a volume rendering of the volume. Based on the volume rendering process, depths of respective locations in the volume rendering are determined, and the depths stored in association with the respective locations. A surface rendering process generates a surface rendering of a surface using the depths and respective locations. The volume rendering and the surface rendering are combined into a combined rendering of the volume and surface. In another method, depths of the surface at respective locations with respect to the volume are determined and the depths are stored in association with the locations. A volume rendering process includes a shading process which uses the depths of the surface and the respective locations. ## Claims ### Claim 1 A computer-implemented method for rendering a volumetric dataset and a surface embedded in the volumetric dataset, the method comprising: performing a volume rendering process, the volume rendering process generating a volume rendering of the volumetric dataset; determining, based on the volume rendering process, depths of respective locations in the volume rendering and storing the depths in association with the respective locations; performing a surface rendering process, the surface rendering process generating a surface rendering of the surface, the surface rendering process comprising using the depths and respective locations; and combining the volume rendering and the surface rendering into a combined rendering of the volumetric dataset and the surface. ### Claim 2 (depends on 1) The method according to claim 1 wherein the surface rendering process comprises, based on a comparison of a depth of a fragment of the surface with a depth stored in association with a location associated with the fragment of the surface, clipping the fragment of the surface from the surface rendering. ### Claim 3 (depends on 1) The method according to claim 1 wherein the surface rendering process comprises, based on a comparison of a depth of a fragment of the surface with a depth stored in association with a location associated with the fragment of the surface, selecting a rendering style or parameter for use in rendering the fragment by the surface rendering process. ### Claim 4 (depends on 1) The method according to any of claim 1 wherein the volume rendering process comprises a physically-based rendering process and the depths are obtained by sampling paths traced during the physically-based rendering process. ### Claim 5 (depends on 4) The method according to claim 4, wherein the depths are obtained by determining, for each pixel in the volume rendering, an average depth value from a plurality of paths sampled to render the pixel by the physically-based rendering process. ### Claim 6 (depends on 1) The method according to claim 1 wherein determining the depths of the respective locations in the volume rendering comprises using a ray casting process to obtain the depths. ### Claim 7 (depends on 6) The method according to claim 6: wherein using the ray casting process to obtain the depths comprises: casting a plurality of rays through the volumetric dataset, each ray being associated with a location of the respective locations; determining, along each of the rays, at least one depth value associated with a predetermined change in a value related to the volumetric dataset, and storing the at least one depth values in a data structure; and selecting, according to predetermined criteria, a pre-determined number of the at least one depth values at each respective location to be used as initial values for the depths at each of the respective locations. ### Claim 8 (depends on 7) The method according to claim 7: wherein determining the depths of each respective location comprises modifying the initial values using data obtained from a physically-based rendering process. ### Claim 9 (depends on 1) The method according to claim 1, wherein determining the depths comprises determining, for at least some of the respective locations in the volume rendering, a plurality of depths per location and storing each of the plurality of depths in association with a location of the locations. ### Claim 10 (depends on 1) The method according to claim 1 further comprising selecting, for a given fragment of the surface, a surface rendering style or parameter for use in the surface rendering process used to render the fragment based on a comparison of a depth of the fragment with a depth of the depths at a location in the volume rendering associated with the fragment. ### Claim 11 (depends on 10) The method according to claim 10, wherein selecting comprises selecting, for the fragment, the surface rendering style or parameter for use in the surface rendering process used to render the fragment based on whether the comparison of the depth of the fragment with the depth at the location in the volume rendering associated with the fragment indicates that the fragment is occluded by or is not occluded by an object in the volumetric dataset. ### Claim 12 (depends on 9) The method according to claim 9 further comprising selecting, for a given fragment of the surface, a surface rendering style or parameter for use in the surface rendering process used to render the fragment based on whether a comparison of a depth associated with the fragment with a depth of the depths at a location associated with the fragment indicates that the fragment is in front of, inside or behind an object in the volumetric dataset. ### Claim 13 A computer-implemented method for rendering a volumetric dataset and a surface embedded in the volumetric dataset, the method comprising: determining depths of the surface at respective locations with respect to a viewing point to be used to render the volumetric dataset and storing the depths in association with the respective locations; and performing a volume rendering process to render the volumetric dataset, the volume rendering process comprising applying a shading process using the depths of the surface and the respective locations. ### Claim 14 (depends on 13) The computer-implemented method according to claim 13, wherein the shading process comprises a modification process to modify one or more optical properties of a first point in the volumetric dataset based on a comparison of a depth of the first point with a depth of the surface stored in association with a location associated with the first point. ### Claim 15 (depends on 14) The computer-implemented method according to claim 14, wherein the modification process is based on one or more properties of a fragment of the surface having the depth stored in association with the location associated with the first point. ### Claim 16 (depends on 15) The computer-implemented method according to claim 15, wherein the one or more properties of the fragment of the surface comprise a colour, an opacity, and/or a normal direction. ### Claim 17 (depends on 16) The computer-implemented method according to claim 16, wherein modifying the one or more optical properties of the first point comprises performing a blending process to obtain a modified colour for the first point based on a colour of the first point and a colour of the fragment of the surface. ### Claim 18 (depends on 17) The computer-implemented method according to claim 17, wherein modifying the one or more optical properties of the first point comprises determining one or more parameters of the blending process based on a property of the fragment of the surface, such as a property indicating that the fragment of the surface is at an edge of the surface. ### Claim 19 (depends on 16) The computer-implemented method according to claim 16, wherein the one or more optical properties of the first point comprise an opacity, and wherein modifying the one or more optical properties of the first point comprises modifying the opacity of the first point. ### Claim 20 (depends on 13) The computer-implemented method according to claim 13, wherein the method comprises performing a surface rendering process to produce a surface rendering of the surface, and wherein the surface rendering process comprises the determining of the depths of the surface. ### Claim 21 (depends on 20) The computer-implemented method according to claim 20, wherein the surface rendering process comprises a rasterization-based process. ### Claim 22 (depends on 21) The computer-implemented method according to claim 21, wherein the determining the depths of the surface at the respective locations comprises performing a depth peeling process. ### Claim 23 (depends on 21) The computer-implemented method according to claim 21, wherein the determining the depths of the surface at the respective locations comprises generating a fragment linked list. ### Claim 24 (depends on 21) The computer-implemented method according to claim 21, wherein the surface rendering process comprises rasterizing the surface for deferred shading during the volume rendering process. ### Claim 25 (depends on 13) The computer-implemented method according to claim 13, wherein the volume rendering process comprises a ray casting process or a path tracing process. ### Claim 26 (depends on 25) The computer-implemented method according to claim 25, wherein the shading process comprises computing shading for points along each of one or more rays cast during the ray casting process or along each of one or more primary rays used during the path tracing process. ### Claim 27 (depends on 13) The computer-implemented method according to claim 13, wherein determining the depths of the surface at the respective locations comprises determining, for at least some of the respective locations, a plurality of depths per location and storing each of the plurality of depths in association with a location of the locations. ### Claim 28 An apparatus comprising: a processor; and a non-transitory memory storing therein a set of machine-readable instructions which when executed by the processor cause the processor to: generate a volume rendering of a volumetric dataset; determine, based on the volume rendering, depths of respective locations in the volume rendering; generate a surface rendering of a surface using the depths and respective locations; and combine the volume rendering and the surface rendering into a combined rendering of the volumetric dataset and the surface.