# US 7,612,773 B2 · Apparatus and method for rendering for display forward-looking image data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,612,773 B2 - Document ID: US7612773B2 - Status: Granted - Kind: B2 - Application number: 11/437,687 - Filed: 2006-05-22 - Published: 2007-11-22 - Granted: 2009-11-03 - Earliest claimed: 2006-05-22 - Inventors: Paul A. Magnin; John W. Goodnow; Russell W. Bowden; Graham A. Wright; Alexander J. Dick; Perry Radau; John J. Graham - Web page: https://patentgrep.com/patent/US7612773B2 ## CPC - A61B 8/12: in body cavities or body tracts, e.g. by using catheters - A61B 5/0062: {Arrangements for scanning} - A61B 5/0084: {for introduction into the body, e.g. by catheters (A61B5/1459 takes precedence)} - A61B 5/743: {Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots} - A61B 8/483: {involving the acquisition of a 3D volume of data} - G06T 3/04: Context-preserving transformations, e.g. by using an importance map (panospheric to cylindrical image transformations G06T3/12) - G06T 15/10: Geometric effects - G06T 19/003: {Navigation within 3D models or images} - A61B 5/0066: {Optical coherence imaging} - G06T 2210/41: Medical - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An apparatus and method for rendering for display forward-looking image data. The apparatus includes rendering for display image data from a forward-looking conical section of tissue collected by an image data collection device, comprising a data input unit configured to receive the image data; and an image data processing unit configured to rasterize the image data onto a projected three-dimensional geometric model to produce a rasterized image, thereby giving the appearance of three-dimensions when displayed. The method includes a method of rendering for display image data from a forward-looking conical section of tissue collected by an image data collecting device, comprising receiving the image data; and rasterizing the image data onto a projected three-dimensional geometric model to produce a rasterized image, thereby giving the appearance of three dimensions when displayed. ## Claims ### Claim 1 A method of rendering for display image data from a forward-looking conical section of tissue collected by an image data collecting device, comprising: receiving the image data from the forward-looking conical section; generating a nutated projected three-dimensional geometric model of the forward-looking conical section, wherein the generating step comprises applying a nutation definition to tilt a three-dimensional geometric model corresponding to the forward-looking conical section; and rasterizing the image data in accordance with the nutated projected three-dimensional geometric model to produce a rasterized image for display on a graphical display. ### Claim 2 (depends on 1) The method of claim 1, further comprising: organizing the received image data in the form of a data table, including an angle and corresponding image data for each scan line of the image data. ### Claim 3 (depends on 1) The method of claim 1, wherein rasterizing the image data in accordance with the projected three-dimensional geometric model comprises orienting scan lines of the image data in a correct geometric perspective on a surface of the three-dimensional geometric model. ### Claim 4 (depends on 3) The method of claim 3, wherein the three-dimensional geometric model comprises a cone. ### Claim 5 (depends on 4) The method of claim 4, wherein the cone is approximated by a plurality of parametric bicubic surface patches. ### Claim 6 (depends on 5) The method of claim 5, wherein the plurality of parametric bicubic surface patches comprise at least one of Hermite surfaces, Bezier surfaces, or B-spline surfaces. ### Claim 7 (depends on 4) The method of claim 4, wherein the cone is modeled using implicit surface equations. ### Claim 8 (depends on 4) The method of claim 4, wherein the cone is modeled using explicit surface equations. ### Claim 9 (depends on 3) The method of claim 3, wherein the three-dimensional geometric model comprises a polyhedron. ### Claim 10 (depends on 9) The method of claim 9, wherein the three-dimensional geometric model comprises a conical polyhedron. ### Claim 11 (depends on 10) The method of claim 10, wherein the conical polyhedron comprises a number of triangles equivalent to a number of scan lines of the image data. ### Claim 12 (depends on 11) The method of claim 11, wherein the conical polyhedron comprises a number of triangles equivalent to a number of scan lines of the image data for a complete 360 degree rotation of the image data collection device. ### Claim 13 (depends on 11) The method of claim 11, wherein the number of triangles comprises at least 400 triangles. ### Claim 14 (depends on 10) The method of claim 10, wherein the conical polyhedron comprises a number of trapezoids equivalent to a number of scan lines of the image data such that an apex of the conical polyhedron is open. ### Claim 15 (depends on 1) The method of claim 1, wherein the nutation definition specifies an angle for nutating an axis of the three-dimensional geometric model. ### Claim 16 (depends on 15) The method of claim 15, wherein nutating the axis of the three-dimensional geometric model comprises nutating the axis of the three-dimensional geometric model at least one of a constant rate or a variable rate. ### Claim 17 (depends on 15) The method of claim 15, wherein nutating the axis of the three-dimensional geometric model comprises at least one of gradually or abruptly increasing the magnitude of nutation of the axis of the three-dimensional geometric model as scan lines of new image data are processed after a predetermined time period in which no new scan lines are added. ### Claim 18 (depends on 15) The method of claim 15, wherein nutating the axis of the three-dimensional geometric model comprises at least one of gradually or abruptly reducing the magnitude of nutation of the axis of the three-dimensional geometric model after a predetermined time period in which no new scan lines were added. ### Claim 19 (depends on 15) The method of claim 15, wherein nutating the axis of the three-dimensional geometric model comprises nutating the axis of the three-dimensional geometric model via manipulation of a user interface. ### Claim 20 (depends on 19) The method of claim 19, wherein the user interface comprises a computer pointing device. ### Claim 21 (depends on 19) The method of claim 19, wherein the user interface comprises an angle encoder. ### Claim 22 (depends on 15) The method of claim 15, wherein nutating the axis of the three-dimensional geometric model comprises nutating the axis of the three-dimensional geometric model away from scan lines corresponding to most recently acquired image data to emphasize the scan lines of the most recently acquired image data and de-emphasize via foreshortening lines diametrically opposed to the scan lines of the most recently acquired image data. ### Claim 23 (depends on 1) The method of claim 1, further comprising: initializing a three-dimensional geometric model; initializing a projection matrix; and projecting the three-dimensional geometric model using the projection matrix to create the nutated projected three-dimensional geometric model. ### Claim 24 (depends on 23) The method of claim 23, further comprising: initializing a nutation matrix; and nutating an axis of the three-dimensional geometric model using the nutation matrix prior to projecting the three-dimensional geometric model using the projection matrix to create the nutated projected three-dimensional geometric model. ### Claim 25 (depends on 1) The method of claim 1, further comprising: adding lighting, shading an/or texture before, during, or after rasterizing the image data. ### Claim 26 (depends on 1) The method of claim 1, further comprising: adding markers and/or icons before, during, or after rasterizing the image data. ### Claim 27 (depends on 26) The method of claim 26, wherein the respective added markers and/or icons represent a location of one or more auxiliary diagnostic and therapeutic devices and/or where their effects will occur. ### Claim 28 (depends on 26) The method of claim 26, wherein the respective added markers and/or icons represent depth markers. ### Claim 29 (depends on 1) The method of claim 1, further comprising: displaying the rasterized image. ### Claim 30 (depends on 29) The method of claim 29, further comprising: displaying the rasterized image for a predetermined period of time. ### Claim 31 (depends on 30) The method of claim 30, further comprising: at least one of gradually or abruptly removing the rasterized image after the predetermined period of time. ### Claim 32 Apparatus for rendering for display image data from a forward-looking conical section of tissue collected by an image data collection device, comprising: a data input unit configured to receive the image data from the forward-looking conical section; and an image data processing unit including a computer-readable medium having computer-executable instructions configured to: generate a nutated projected three-dimensional geometric model of the forward-looking conical section by applying a nutation definition to a three-dimensional geometric model, and rasterize the image data in accordance with the nutated projected three-dimensional geometric model to produce a rasterized image for display on a graphical display. ### Claim 33 (depends on 32) The apparatus of claim 32, wherein the data input unit organizes the image data in the form of a data table, including an angle and corresponding image data for each scan line of the image data. ### Claim 34 (depends on 32) The apparatus of claim 32, wherein the image data processing unit orients scan lines of the image data in a correct geometric perspective on a surface of the three-dimensional geometric model. ### Claim 35 (depends on 34) The apparatus of claim 34, wherein the three-dimensional geometric model comprises a cone. ### Claim 36 (depends on 35) The apparatus of claim 35, wherein the cone is approximated by a plurality of parametric bicubic surface patches. ### Claim 37 (depends on 36) The apparatus of claim 36, wherein the plurality of parametric bicubic surface patches comprise at least one of Hermite surfaces, Bezier surfaces, or B-spline surfaces. ### Claim 38 (depends on 35) The apparatus of claim 35, wherein the cone is modeled using implicit surface equations. ### Claim 39 (depends on 35) The apparatus of claim 35, wherein the cone is modeled using explicit surface equations. ### Claim 40 (depends on 34) The apparatus of claim 34, wherein the three-dimensional geometric model comprises a polyhedron. ### Claim 41 (depends on 40) The apparatus of claim 40, wherein the three-dimensional model comprises a conical polyhedron. ### Claim 42 (depends on 41) The apparatus of claim 41, wherein the conical polyhedron comprises a number of triangles equivalent to a number of scan lines of the image data. ### Claim 43 (depends on 42) The apparatus of claim 42, wherein the conical polyhedron comprises a number of triangles equivalent to a number of scan lines of the image data for a complete 360 degree rotation of the image data collection device. ### Claim 44 (depends on 42) The apparatus of claim 42, wherein the number of triangles comprises at least 400 triangles. ### Claim 45 (depends on 42) The apparatus of claim 42, wherein the conical polyhedron comprises a number of trapezoids equivalent to a number of scan lines of the image data such that an apex of the conical polyhedron is open. ### Claim 46 (depends on 32) The apparatus of claim 32, wherein the image data processing unit comprises: a three-dimensional geometric model initialization unit configured to initialize a three-dimensional geometric model; a projection matrix initialization unit configured to initialize a projection matrix; and a three-dimensional geometric model projection unit configured to project the three-dimensional geometric model using the projection matrix. ### Claim 47 (depends on 46) The apparatus of claim 46, wherein the image data processing unit further comprises: a nutation matrix initialization unit configured to initialize a nutation matrix in accordance with the nutation definition; and a three-dimensional geometric model axis nutation unit configured to nutate an axis of the three-dimensional geometric model using the nutation matrix prior to projecting the three-dimensional model using the projection matrix to produce the nutated projected three-dimensional geometric model. ### Claim 48 (depends on 47) The apparatus of claim 47, further comprising: a user manipulation device configured to control nutation of the axis of the three-dimensional geometric model. ### Claim 49 (depends on 48) The apparatus of claim 48, wherein the user manipulation device comprises a computer pointing device. ### Claim 50 (depends on 48) The apparatus of claim 48, wherein the user manipulation device comprises an angle encoder. ### Claim 51 (depends on 47) The apparatus of claim 47, wherein the image data processing unit further comprises: an image data rasterization unit configured to rasterize the image data in accordance with the nutated projected three-dimensional geometric model. ### Claim 52 (depends on 51) The apparatus of claim 51, wherein the image data rasterization unit is further configured to add lighting, shading, and/or texture before, during, or after rasterizing the image data to enhance the appearance of three-dimensions when displayed. ### Claim 53 (depends on 51) The apparatus of claim 51, wherein the image data rasterization unit is further configured to add markers and/or icons before, during, or after rasterizing the image data. ### Claim 54 (depends on 32) An imaging system comprising the rendering apparatus of claim 32. ### Claim 55 (depends on 54) The imaging system of claim 54, wherein the imaging system comprises an IVUS system. ### Claim 56 (depends on 32) A display apparatus comprising the rendering apparatus of claim 32. ### Claim 57 An imaging apparatus that scans and creates an image of a conical forward-looking section of tissue, comprising: a transducer that is rotated in a forward-looking orientation for transmitting a signal and receiving image data resulting from interaction of the transmitted signal and tissues it encounters; electronic circuitry that processes the received image data; and an image data processing unit including a computer-readable medium having computer-executable instructions configured to: generate a nutated projected three-dimensional geometric model of the forward-looking conical section by applying a nutation definition to a three-dimensional geometric model, and rasterize the image data in accordance with the nutated projected three-dimensional geometric model to produce a rasterized image for display on a graphical display. ### Claim 58 (depends on 57) The apparatus of claim 57, wherein the three-dimensional geometric model comprises a three-dimensional cone model. ### Claim 59 (depends on 58) The imaging apparatus of claim 58, wherein the processing unit further comprises: computer-executable instructions for adding three-dimensional lighting, shading, and/or texture visual effects to the three-dimensional cone model. ### Claim 60 (depends on 58) The imaging apparatus of claim 58, wherein the processing unit further comprises: computer-executable instructions that facilitate nutating an axis of the three-dimensional cone model. ### Claim 61 (depends on 58) The imaging apparatus of claim 58, wherein the processing unit further comprises: computer-executable instructions that facilitate superimposing one or more markers and/or icons on the three-dimensional cone for accurately indicating a location of one or more auxiliary diagnostic and therapeutic devices and/or where they will have effect. ### Claim 62 (depends on 61) The imaging apparatus of claim 61, wherein the processing unit further comprises computer executable instructions facilitating superimposing depth markers. ### Claim 63 (depends on 58) The imaging apparatus of claim 58, further comprising: an interventional device, wherein the transducer is configured to image a portion of tissue while the interventional device operates on or near the tissue being imaged. ### Claim 64 (depends on 58) The imaging apparatus of claim 58, wherein the processing unit comprises computer-executable instructions facilitating persisting displaying the image data on the graphical display for a finite period of time, after which the image is either gradually or abruptly removed. ### Claim 65 A method for displaying image data that has been acquired by an image data collecting device from a conical forward-looking section of tissue, comprising: receiving the image data from the forward-looking conical section; and displaying the image data by orienting scan lines of the image data in their correct geometric perspective on a surface of a three-dimensional geometric model, the displaying step comprising: generating a nutated projected three-dimensional geometric model of the forward-looking conical section, wherein the generating step comprises applying a nutation definition to tilt a three-dimensional geometric model corresponding to the forward-looking conical section; and rasterizing the image data in accordance with the nutated projected three-dimensional geometric model to produce a rasterized image for display on a graphical display. ### Claim 66 (depends on 65) The method of claim 65, wherein the three-dimensional geometric model comprises a three-dimensional cone model. ### Claim 67 (depends on 66) The method of claim 66, wherein the nutation definition specifies an angle for nutating an axis of the three-dimensional cone model. ### Claim 68 (depends on 67) The method of claim 67, further comprising: nutating an axis of the three-dimensional cone model in real-time in response to new image scan lines being acquired during the receiving step. ### Claim 69 (depends on 67) The method of claim 67, further comprising: nutating the axis of the three-dimensional cone away from most recently acquired scan lines so as to emphasize the most recently acquired scan lines and deemphasize, via foreshortening, lines diametrically opposed to the most recently acquired scan lines. ### Claim 70 (depends on 67) The method of claim 67, further comprising: nutating the axis of the three-dimensional cone in real-time at least one of a constant or variable rate independent of new image lines being added. ### Claim 71 (depends on 67) The method of claim 67, further comprising: nutating the axis of the three-dimensional cone in response to manipulation of a user interface control device. ### Claim 72 (depends on 71) The method of claim 71, where the user interface control device comprises a computer pointing device. ### Claim 73 (depends on 71) The method of claim 71, where the user interface control device comprises an angle encoder. ### Claim 74 (depends on 67) The method of claim 67, further comprising: reducing the magnitude of the nutation angle of the three-dimensional cone model in response to no new image lines being added after a certain time period. ### Claim 75 (depends on 67) The method of claim 67, further comprising: gradually or abruptly increasing the magnitude of nutation of the three-dimensional cone in response to new image lines being added after a certain time period in which no new image lines were added. ### Claim 76 (depends on 67) The method of claim 67, further comprising: lighting, shading, and/or texturing the image of the three-dimensional cone in real-time as the cone nutates. ### Claim 77 (depends on 67) The method of claim 67, further comprising: superimposing a location of one or more auxiliary diagnostic and therapeutic devices or a location of one or more actions on the image of the three-dimensional cone. ### Claim 78 (depends on 65) The method of claim 65, further comprising: retaining the image data on the display for a predetermined period of time and thereafter removing the image data.