# US 11,122,250 B2 · Three-dimensional image projection apparatus, three-dimensional image projection method, and three-dimensional image projection control program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,122,250 B2 - Document ID: US11122250B2 - Status: Granted - Kind: B2 - Application number: 16/292,370 - Filed: 2019-03-05 - Published: 2019-06-27 - Granted: 2021-09-14 - Earliest claimed: 2016-09-28 - Assignees: FUJIFILM Corporation - Applicants: FUJIFILM Corporation - Inventors: Hirotaka Ito - Web page: https://patentgrep.com/patent/US11122250B2 ## CPC - H04N 13/167: Synchronising or controlling image signals - A61B 6/03: Computed tomography [CT] - A61B 90/361: {Image-producing devices, e.g. surgical cameras} - A61B 90/37: {Surgical systems with images on a monitor during operation} - A61B 90/39: {Markers, e.g. radio-opaque or breast lesions markers} - G03B 17/54: with projector - G03B 21/14: Details - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 20/20: in augmented reality scenes - G06V 20/64: Three-dimensional [3D] objects - H04N 9/3185: {Geometric adjustment, e.g. keystone or convergence (optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14)} - H04N 9/3194: {including sensor feedback} - H04N 13/363: using image projection screens (volumetric display H04N13/388) - A61B 2090/364: {Correlation of different images or relation of image positions in respect to the body} - A61B 2090/372: {Details of monitor hardware} - A61B 2090/3983: {Reference marker arrangements for use with image guided surgery} - G03B 15/14: for taking photographs during medical operations - G06T 2207/30004: Biomedical image processing - G06T 2207/30204: Marker - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A three-dimensional image projection apparatus includes: a screen recognition unit 12 that recognizes a position of a screen which is randomly provided and a posture of the screen in a three-dimensional space; a projection unit 30 that projects a projection image of a three-dimensional image of a subject on the screen on the basis of the position of the screen recognized by the screen recognition unit 12; and a three-dimensional image processing unit 11 that changes parameters of three-dimensional image processing performed for the three-dimensional image, on the basis of the posture of the screen in the three-dimensional space which is recognized by the screen recognition unit 12. ## Claims ### Claim 1 A three-dimensional image projection apparatus comprising: a processor configured to recognize an inclination of a screen in a three-dimensional space, wherein the screen is a flat plate and configured to be movable three-dimensionally without changing a shape of the flat plate by a user; a projection unit that projects a projection image of a three-dimensional image of an organ of a subject on the screen on the basis of the position of the screen; and the processor is configured to change parameters of three-dimensional image processing performed for the three-dimensional image, on the basis of a change in the inclination of the screen in the three-dimensional space to perform a rotation process for the projection image of the three-dimensional image projected on the screen. ### Claim 2 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the processor is configured to change the parameters of the three-dimensional image processing on the basis of the change in the posture of the screen in the three-dimensional space to change the posture of the projection image of the three-dimensional image. ### Claim 3 (depends on 2) The three-dimensional image projection apparatus according to claim 2, wherein the processor is configured to change the parameters of the three-dimensional image processing on the basis of the change in the posture of the screen in the three-dimensional space to perform a rotation process for the three-dimensional image. ### Claim 4 (depends on 3) The three-dimensional image projection apparatus according to claim 3, wherein the rotation process is a rotation process using, as a rotation axis, a straight line that extends in a plane of a coordinate system of the three-dimensional image which corresponds to a projection surface of the screen. ### Claim 5 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein, in a case in which the screen is moved, the processor is configured to recognize a position of the screen after the movement, and the projection unit projects the projection image on the screen after the movement. ### Claim 6 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the processor is configured to detect a marker provided on the screen to recognize the position of the screen and the posture of the screen in the three-dimensional space. ### Claim 7 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the processor is configured to detect the screen to recognize the position of the screen and the posture of the screen in the three-dimensional space. ### Claim 8 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the projection unit projects a projection image of an operation marker on the screen, the processor is configured to recognize a change in the projection image of the operation marker, and the processor is configured to change the parameters of the three-dimensional image processing in a case in which the processor is configured to recognize the change in the projection image of the operation marker. ### Claim 9 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the processor is configured to change the parameters of the three-dimensional image processing to change transparency of the three-dimensional image. ### Claim 10 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the projection unit projects an index indicating a projection range of the projection unit. ### Claim 11 (depends on 1) The three-dimensional image projection apparatus according to claim 1, further comprising: an imaging unit that captures an image of the screen, wherein the processor is configured to recognize the position of the screen and the posture of the screen in the three-dimensional space on the basis of the image captured by the imaging unit. ### Claim 12 (depends on 11) The three-dimensional image projection apparatus according to claim 11, wherein an arrangement relationship between the projection unit and the imaging unit is fixed in advance. ### Claim 13 (depends on 12) The three-dimensional image projection apparatus according to claim 12, wherein the projection unit and the imaging unit have an arrangement relationship in which optical axes of the projection unit and the imaging unit are aligned with each other and angles of view of the projection unit and the imaging unit are matched with each other. ### Claim 14 (depends on 12) The three-dimensional image projection apparatus according to claim 12, wherein the imaging unit and the projection unit are configured such that a position or a posture of the imaging unit and the projection unit is changeable. ### Claim 15 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the projection unit projects the projection images on a plurality of the screens which are provided in one projection range. ### Claim 16 (depends on 15) The three-dimensional image projection apparatus according to claim 15, wherein the projection unit projects different projection images on the plurality of screens. ### Claim 17 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the projection unit projects, as the projection image of the three-dimensional image, a projection image of a simulation image for an operation or a projection image of an image including treatment information on the screen. ### Claim 18 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the projection unit projects vital information on the screen. ### Claim 19 A three-dimensional image projection method comprising: recognizing an inclination of a screen in a three-dimensional space, wherein the screen is a flat plate and configured to be movable three-dimensionally without changing a shape of the flat plate by a user; projecting a projection image of a three-dimensional image of an organ of a subject on the screen on the basis of the position of the screen; and changing parameters of three-dimensional image processing performed for the three-dimensional image, on the basis of a change in the inclination of the screen in the three-dimensional space to perform a rotation process for the projection image of the three-dimensional image projected on the screen. ### Claim 20 A non-transitory computer recording medium storing a three-dimensional image projection control program that causes a computer to function as: a processor configured to recognize an inclination of a screen in a three-dimensional space, wherein the screen is a flat plate and configured to be movable three-dimensionally without changing a shape of the flat plate by a user; a projection unit control unit that controls a projection unit such that a projection image of an organ of a three-dimensional image of a subject is projected on the screen on the basis of the position of the screen; and the processor is configured to change parameters of three-dimensional image processing performed for the three-dimensional image, on the basis of a change in the inclination of the screen in the three-dimensional space to perform a rotation process for the projection image of the three-dimensional image projected on the screen. ### Claim 21 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the processor is configured to recognize the missing of a portion of the projection image and output information indicating the missing of the portion of the projection image; and the change in the projection image is indicated by the information indicating the missing of the portion of the projection image. ### Claim 22 (depends on 1) The three-dimensional image projection apparatus according to claim 1, wherein the missing of a portion of the projection image is caused by a hand of the user.