# US 2016/0364902 A1 · High quality embedded graphics for remote visualization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2016/0364902 A1 - Document ID: US20160364902A1 - Status: Published application - Kind: A1 - Application number: 14/734,003 - Filed: 2015-06-09 - Published: 2016-12-15 - Earliest claimed: 2015-06-09 - Assignees: Siemens Aktiengesellschaft - Applicants: Siemens Aktiengesellschaft - Inventors: Wei Hong; John Paulus, JR.; Feng Qiu; Daphne Yu - Web page: https://patentgrep.com/patent/US20160364902A1 ## CPC - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - G06T 15/00: Three-dimensional [3D] image rendering - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2210/41: Medical - G06T 2219/004: Annotating, labelling - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Apparatuses and methods for rendering and compositing two-dimensional images for remote visualization using a multi-pass rendering technique are provided. A server computer renders a three-dimensional volume separately from one or more three-dimensional graphical objects to be embedded in the rendered image of the volume. The server compresses and transmits the separately rendered images to a client computer. The client computer decompresses the rendered images and generates a composite image using the rendered images. If a user manipulates, adds or deletes an embedded object, the client computer generates a new composite image using the previously rendered volume image. In the case of a new or manipulated embedded graphical object, the server only renders the manipulated or new object. ## Claims ### Claim 1 A method for rendering embedded graphics for remote visualization, the method comprising: rendering, by a server, a two-dimensional volume rendered image from three-dimensional data captured for a volume; rendering, by the server, a two-dimensional graphic rendered image to be embedded in the two-dimensional volume rendered image from three-dimensional data for a graphic; compressing, by the server, the volume rendered image and the graphic rendered image; and transmitting, to a workstation over a network, the compressed volume rendered image and the compressed graphic rendered image. ### Claim 2 (depends on 1) The method of claim 1 further comprising: receiving, by the workstation, the compressed volume rendered image and compressed the graphic rendered image; decompressing, by the workstation, the compressed volume rendered image and compressed the graphic rendered image; generating, by the workstation, a composite rendered image from the volume rendered image and the graphic rendered image; and displaying, by the workstation, the composite rendered image. ### Claim 3 (depends on 1) The method of claim 1 wherein the volume rendered image and the graphic rendered image comprise graphics hardware frame buffer objects. ### Claim 4 (depends on 3) The method of claim 3 wherein the graphics hardware frame buffer objects for the graphic rendered image comprise an image set, the image set comprising: color pixel information of the graphic rendered image; a depth buffer comprising relative distance information for each pixel of the graphic rendered image; and a stencil buffer comprising identification of which pixels of the graphic rendered image intersect with pixels of the volume rendered image. ### Claim 5 (depends on 2) The method of claim 2 wherein rendering the graphic rendered image comprises: rendering, based on the stencil buffer, non-empty pixels for pixels that intersect with pixels of the volume rendered image; and rendering, based on the stencil buffer, empty pixels for pixels that do not intersect with pixels of the volume rendered image. ### Claim 6 (depends on 5) The method of claim 5 further comprising: receiving, by the workstation, the volume rendered image and the graphic rendered image; generating, by the workstation, a composite rendered image from the volume rendered image and the graphic rendered image; and displaying, by the workstation, the composite rendered image. ### Claim 7 (depends on 6) The method of claim 6 wherein generating the composite rendered image comprises: replacing pixels of the volume rendered image with non-empty pixels from the graphic rendered image. ### Claim 8 (depends on 6) The method of claim 6 wherein generating the compositing rendered image comprises: comparing the depth buffer entry for each non-empty pixel from the graphic rendered image with the depth buffer entry for each corresponding pixel for the volume rendered image; and replacing pixels of the volume rendered image with non-empty pixels from the graphic rendered image with smaller depth buffer values than the corresponding pixels of the volume rendered image. ### Claim 9 (depends on 2) The method of claim 2 further comprising: receiving, by the workstation, a request to delete the graphic rendered image; and displaying, by the workstation in response to receiving the request, the volume rendered image without the graphic rendered image. ### Claim 10 A system for rendering embedded graphics for remote visualization, the system comprising: a server configured to: render a two-dimensional volume rendered image from three-dimensional data captured for a volume; render a two-dimensional graphic rendered image to be embedded in the two-dimensional volume rendered image from three-dimensional data for a graphic; compress the volume rendered image and the graphic rendered image; transmit, over a network, the compressed volume rendered image and compressed the graphic rendered image; a workstation configured to: receive, over the network, the compressed volume rendered image and the compressed graphic rendered image; decompress the compressed volume rendered image and the compressed graphic rendered image; generate a composite rendered image from the volume rendered image and the graphic rendered image; and display the composite rendered image. ### Claim 11 (depends on 10) The system of claim 10 wherein the volume rendered image and the graphic rendered image comprise graphics hardware frame buffer objects. ### Claim 12 (depends on 11) The system of claim 11 wherein the graphics hardware frame buffer objects for the graphic rendered image comprise graphic image sets, the graphic image sets comprising: color pixel information of the graphic rendered image; a depth buffer comprising relative distance information for each pixel of the graphic rendered image; and a stencil buffer comprising identification of which pixels of the graphic rendered image intersect with pixels of the volume rendered image. ### Claim 13 (depends on 12) The system of claim 12 wherein the graphic rendered image comprises: non-empty pixels for pixels that intersect with pixels of the volume rendered image based on the stencil buffer; and empty pixels for pixels that do not intersect with pixels of the volume rendered image based on the stencil buffer. ### Claim 14 (depends on 13) The system of claim 13 wherein the composite rendered image comprises: the volume rendered image with pixels replaced by the non-empty pixels from the graphic rendered image. ### Claim 15 (depends on 10) The system of claim 10 wherein the composite rendered image comprises: the volume rendered image with pixels replaced by non-empty pixels from the graphic rendered image with smaller depth buffer values than the corresponding pixels of the volume rendered image. ### Claim 16 (depends on 10) The system of claim 10 wherein the workstation is further configured to: receive a request to delete the graphic rendered image; and display, in response to receiving the request, the volume rendered image without the graphic rendered image. ### Claim 17 A method for rendering high quality embedded graphics for remote, the method comprising: scanning a patient to capture three-dimensional volume data representing the patient; rendering a two-dimensional volume rendered image from the three-dimensional volume data representing the patient; rendering a plurality of two-dimensional graphic rendered images to be embedded in the two-dimensional volume rendered image from three-dimensional data for a plurality of graphics, wherein the graphics comprise a display point, a line graphic, a curve, a surface, volume shape, or a combination thereof; generating a composite rendered image from the volume rendered image and the plurality of graphic rendered images; and displaying the composite rendered image. ### Claim 18 (depends on 17) The method of claim 17 wherein rendering the plurality graphic rendered images comprises: rendering non-empty pixels for pixels that intersect with pixels of the volume rendered image; and rendering empty pixels for pixels that do not intersect with pixels of the volume rendered image. ### Claim 19 (depends on 17) The method of claim 17 wherein generating the composite rendered image comprises: replacing pixels of the volume rendered image with non-empty pixels from the plurality of graphic rendered images. ### Claim 20 (depends on 17) The method of claim 17 further comprising: receiving a request to delete one of the plurality of graphic rendered images; re-generating the composite rendered image from the volume rendered image and the plurality of graphic rendered images, wherein the composite rendered image is re-generated without one of the plurality of graphic rendered images; and displaying the composite rendered image.