# US 11,442,254 B2 · Augmented reality projection device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,254 B2 - Document ID: US11442254B2 - Status: Granted - Kind: B2 - Application number: 16/377,060 - Filed: 2019-04-05 - Published: 2020-10-08 - Granted: 2022-09-13 - Earliest claimed: 2019-04-05 - Assignees: Inner Ray, Inc. - Applicants: Inner Ray, Inc. - Inventors: Dong Wook Nam; Joo Hyung Hong - Web page: https://patentgrep.com/patent/US11442254B2 ## CPC - G02B 13/0095: {Relay lenses or rod lenses (in instruments for viewing the inside of hollow bodies G02B23/2446)} - G02B 13/16: for use in conjunction with image converters or intensifiers {, or for use with projectors, e.g. objectives for projection TV} - G02B 26/0833: {the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD (G02B26/0825 takes precedence; micromechanical devices in general B81B)} - G02B 27/1006: {for splitting or combining different wavelengths (G02B27/1086, G02B27/141 take precedence)} - G02B 27/141: {using dichroic mirrors} - G02B 27/20: for imaging minute objects, e.g. light-pointer - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - A61B 2090/366: {using projection of images directly onto the body} - G02B 13/22: Telecentric objectives or lens systems - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract An augmented reality projection device is provided. The augmented reality projection device includes an excitation light source that generates excitation light and a fluorescent light detector that detects fluorescent light generated in the fluorescent light generation area and generates a fluorescent image. The device also includes a projector that converts an image signal for displaying a visual indicator in the fluorescent light generation area into visual indicator light. The device further includes a processor that controls operations of the excitation light source, the fluorescent light detector, and the projection. The device further includes a coaxial optics that delivers the excitation light and the visual indicator light to the fluorescent light generation area and delivers the fluorescent light to the fluorescent light detector. ## Claims ### Claim 1 An augmented reality projection device comprising: an excitation light source configured to generate excitation light which is applied to a fluorescent light generation area to excite a fluorescent material; a fluorescent light detector configured to detect fluorescent light generated in the fluorescent light generation area and generate a fluorescent image, the fluorescent light including a fluorescent light detection area corresponding to the fluorescent light generation area; a projector configured to convert an image signal for displaying a visual indicator in the fluorescent light generation area into visual indicator light, the visual indicator being generated to correspond to the identified fluorescent light detection area; a processor configured to control operations of the excitation light source, the fluorescent light detector, and the projector; and a coaxial optics configured to deliver the excitation light and the visual indicator light to the fluorescent light generation area and deliver the fluorescent light to the fluorescent light detector, wherein the excitation light, the fluorescent light, and the visual indicator light pass through optical paths which are partially common by the coaxial optics, wherein the coaxial optics includes: an optical path branching/merging unit configured to cause an optical path of the excitation light and an optical path of the visual indicator light to be substantially identical to each other and substantially branch an optical path of the fluorescent light from the optical path of the excitation light; a relay optics configured to deliver the excitation light, the fluorescent light, and the visual indicator light in processing directions thereof; a half pentaprism horizontally coupled to the relay optics and configured to incline the optical path of the excitation light and the optical path of the visual indicator light, and cause the optical path of the fluorescent light to be horizontal; and an objective optics obliquely coupled to the half pentaprism and configured to deliver the excitation light and the visual indicator light emitted from the half pentaprism to the fluorescent light generation area and to deliver the fluorescent light emitted from the fluorescent light generation area to the half pentaprism. ### Claim 2 (depends on 1) The augmented reality projection device according to claim 1, wherein the optical path branching/merging unit includes: a first dichroic mirror that is disposed oblique at a position at which the excitation light emitted from the excitation light source and progressing horizontally and the fluorescent light progressing vertically to the fluorescent light detector intersect each other and is configured to reflect the excitation light and to transmit the fluorescent light; and a second dichroic mirror that is disposed oblique at a position at which the visual indicator light emitted from the projector and progressing horizontally and the excitation light reflected by the first dichroic mirror and progressing vertically intersect each other and is configured to reflect the excitation light and the fluorescent light and to transmit the visual indicator light. ### Claim 3 (depends on 2) The augmented reality projection device according to claim 2, further comprising: an image sensor that generates a color signal using visible light which is reflected from the fluorescent light generation area to which a visual indicator has been projected using the visual indicator light; and a third dichroic mirror that is disposed oblique between the second dichroic mirror and the projector and is configured to transmit the visual indicator light and to reflect the visible light, wherein the visible light reaches the image sensor through the coaxial optics. ### Claim 4 (depends on 3) The augmented reality projection device according to claim 3, wherein the first dichroic mirror and the third dichroic mirror are disposed to be substantially parallel to each other and the first dichroic mirror and the second dichroic mirror are disposed to be perpendicular to teach other. ### Claim 5 (depends on 3) The augmented reality projection device according to claim 3, wherein a resolution of the fluorescent light detector is equal to or less than a resolution of the image sensor. ### Claim 6 (depends on 1) The augmented reality projection device according to claim 1, wherein the optical path branching/merging unit includes: a first dichroic mirror that is disposed oblique at a position at which the excitation light emitted from the excitation light source and progressing horizontally and the fluorescent light progressing vertically intersect each other and is configured to transmit the excitation light and to reflect the fluorescent light; and a second dichroic mirror that is disposed oblique at a position at which the visual indicator light emitted from the projector and progressing horizontally and the excitation light transmitted by the first dichroic mirror intersect each other and is configured to reflect the excitation light and the fluorescent light and to transmit the visual indicator light. ### Claim 7 (depends on 1) The augmented reality projection device according to claim 1, wherein brightness of the visual indicator reflects brightness of the fluorescent light. ### Claim 8 (depends on 1) The augmented reality projection device according to claim 1, wherein the visual indicator light is monochromatic light. ### Claim 9 (depends on 1) The augmented reality projection device according to claim 1, further comprising an I/O interface that communicates with the outside, wherein the fluorescent image is output to the outside via the I/O interface and the image signal is received from the outside via the I/O interface. ### Claim 10 (depends on 1) The augmented reality projection device according to claim 1, wherein the processor identifies the fluorescent light detection area in the fluorescent image and generates the image signal for displaying the visual indicator corresponding to the identified fluorescent light detection area. ### Claim 11 (depends on 1) The augmented reality projection device according to claim 1, wherein the fluorescent image is a still image or a moving image. ### Claim 12 (depends on 1) The augmented reality projection device according to claim 1, wherein the visual indicator changes when a difference between fluorescent images which are generated at different times is equal to or greater than a threshold value. ### Claim 13 An augmented reality projection device comprising: an excitation light source configured to generate excitation light which is applied to a fluorescent light generation area to excite a fluorescent material; a fluorescent light detector configured to detect fluorescent light generated in the fluorescent light generation area and generate a fluorescent image, the fluorescent light including a fluorescent light detection area corresponding to the fluorescent light generation area; a projector configured to convert an image signal for displaying a visual indicator in the fluorescent light generation area into visual indicator light, the visual indicator being generated to correspond to the identified fluorescent light detection area; an image sensor configured to generate a color signal using visible light which is reflected from the fluorescent light generation area to which a visual indicator has been projected using the visual indicator light; and a coaxial optics configured to deliver the excitation light and the visual indicator light to the fluorescent light generation area and deliver the fluorescent light to the fluorescent light detector, wherein the excitation light, the fluorescent light, and the visual indicator light pass through optical paths which are partially common by the coaxial optics, wherein the coaxial optics includes: an optical path branching/merging unit configured to cause an optical path of the excitation light and an optical path of the visual indicator light to be substantially identical to each other and substantially branch an optical path of the fluorescent light and an optical path of the visible light from the optical path of the excitation light; a relay optics configured to deliver the excitation light, the fluorescent light, the visual indicator light, and the visible light in processing directions thereof; a half pentaprism horizontally coupled to the relay optics and configured to incline the optical path of the excitation light and the optical path of the visual indicator light, and cause the optical path of the fluorescent light and the optical path of the visible light to be horizontal; and an objective optics obliquely coupled to the half pentaprism and configured to deliver the excitation light and the visual indicator light emitted from the half pentaprism to the fluorescent light generation area and to deliver the fluorescent light and the visible light emitted from the fluorescent light generation area to the half pentaprism. ### Claim 14 (depends on 13) The augmented reality projection device according to claim 13, further comprising an I/O interface that communicates with the outside, wherein the fluorescent image is output to the outside via the I/O interface and the image signal is received from the outside via the I/O interface. ### Claim 15 (depends on 13) The augmented reality projection device according to claim 13, wherein brightness of the visual indicator reflects brightness of the fluorescent light. ### Claim 16 An augmented reality projection device comprising: an excitation light source configured to generate excitation light which is applied to a fluorescent light generation area to excite a fluorescent material; a fluorescent light detector configured to detect fluorescent light generated in the fluorescent light generation area and generate a fluorescent image, the fluorescent light including a fluorescent light detection area corresponding to the fluorescent light generation area; a projector configured to convert an image signal for displaying a visual indicator in the fluorescent light generation area into visual indicator light, the visual indicator being generated to correspond to the identified fluorescent light detection area; an image sensor configured to generate a color signal using visible light which is reflected from the fluorescent light generation area to which a visual indicator has been projected using the visual indicator light; and a coaxial optics configured to deliver the excitation light and the visual indicator light to the fluorescent light generation area and deliver the fluorescent light to the fluorescent light detector, wherein the excitation light, the fluorescent light, and the visual indicator light pass through optical paths which are partially common by the coaxial optics, wherein the coaxial optics includes an optical path branching/merging unit configured to cause an optical path of the excitation light and an optical path of the visual indicator light to be substantially identical to each other and substantially branch an optical path of the fluorescent light and an optical path of the visible light from the optical path of the excitation light, and wherein the optical path branching/merging unit includes: a first dichroic mirror that is disposed oblique at a position at which the excitation light emitted from the excitation light source and progressing horizontally and the fluorescent light progressing vertically to the fluorescent light detector intersect each other and is configured to reflect the excitation light and to transmit the fluorescent light; a second dichroic mirror that is disposed oblique at a position at which the visual indicator light emitted from the projector and progressing horizontally and the excitation light reflected by the first dichroic mirror and progressing vertically intersect each other and is configured to reflect the excitation light and the fluorescent light and to transmit the visual indicator light; and a third dichroic mirror that is disposed oblique between the second dichroic mirror and the projector and is configured to transmit the visual indicator light and to reflect the visible light.