# US 12,764,429 B2 · Multiple band pass Bayer pattern filter > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,764,429 B2 - Document ID: US12764429B2 - Status: Granted - Kind: B2 - Application number: 18/830,209 - Filed: 2024-09-10 - Published: 2024-12-26 - Granted: 2026-10-13 - Earliest claimed: 2018-07-12 - Assignees: Intuitive Surgical Operations, Inc. - Applicants: Intuitive Surgical Operations, Inc. - Inventors: William Jason Culman; Jeffrey M. DiCarlo; Ian E. McDowall - Web page: https://patentgrep.com/patent/US12764429B2 ## CPC - H04N 25/131: including elements passing infrared wavelengths - A61B 1/00009: {of image signals during a use of endoscope} - A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens} - A61B 5/4887: {Locating particular structures in or on the body} - A61B 5/742: {using visual displays (displays for heart-related electrical signals, e.g. ECG, A61B5/339)} - A61B 34/25: {User interfaces for surgical systems} - A61B 90/30: Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure - G02B 5/201: {in the form of arrays} - A61B 34/35: for telesurgery - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract An apparatus including a pixel array comprising two or more photo-sensor elements; a first optical filter disposed on a first photo-sensor element of the pixel array, the first optical filter configured such that a spectral response of the first optical filter includes: a first passband in a first wavelength range, and a second passband in a second wavelength range, the first passband and the second passband being separated by a first stop band; and a second optical filter disposed on a second photo-sensor element of the pixel array, the second optical filter configured such that a spectral response of the second optical filter includes: a third passband in the first wavelength range, and a fourth passband in the second wavelength range, the third passband and the fourth passband being separated by a second stop band; wherein the second passband and the fourth passband are discrete passbands in an infrared range. ## Claims ### Claim 1 An image sensing apparatus comprising: a pixel array comprising two or more photo-sensor elements; a first optical filter disposed on a first photo-sensor element of the pixel array, the first optical filter configured such that a spectral response of the first optical filter includes: a first passband in a first wavelength range, and a second passband in a second wavelength range, the first passband and the second passband being separated by a first stop band; and a second optical filter disposed on a second photo-sensor element of the pixel array, the second optical filter configured such that a spectral response of the second optical filter includes: a third passband in the first wavelength range, and a fourth passband in the second wavelength range, the third passband and the fourth passband being separated by a second stop band; wherein the second passband and the fourth passband are discrete passbands in an infrared range. ### Claim 2 (depends on 1) The image sensing apparatus of claim 1, wherein: the second passband has a wavelength between 1200 and 1250 nm; and the fourth passband has a wavelength between 1300 and 1350 nm. ### Claim 3 (depends on 1) The image sensing apparatus of claim 1, wherein the first optical filter is configured such that an absorption characteristic of a first tissue type of a surgical site in the second passband is higher than an absorption characteristic of a second tissue type of a surgical site in the second passband, and the second optical filter is configured such that an absorption characteristic of the second tissue type in the fourth passband is higher than an absorption characteristic of the first tissue type in the fourth passband. ### Claim 4 (depends on 1) The image sensing apparatus of claim 1, further comprising a third optical filter disposed on a third photo-sensor element of the pixel array, the third optical filter configured such that a spectral response of the third optical filter includes: a fifth passband in the first wavelength range, and a sixth passband in the second wavelength range, the fifth passband and the sixth passband being separated by a third stop band. ### Claim 5 (depends on 4) The image sensing apparatus of claim 4, wherein the first, second, and third optical filters are arranged in a Bayer pattern. ### Claim 6 (depends on 4) The image sensing apparatus of claim 4, wherein a portion of the first stop band overlaps with a portion of the second stop band and a portion of the third stop band. ### Claim 7 (depends on 1) The image sensing apparatus of claim 1, further comprising: one or more processing devices configured to generate a representation of an image of using output signals from the pixel array; and circuitry configured to provide the output signals from the pixel array to the one or more processing devices. ### Claim 8 (depends on 7) The image sensing apparatus of claim 7, wherein the one or more processing devices are configured to generate the representation of the image by combining electromagnetic radiation received by the pixel array in the first wavelength range, and electromagnetic radiation received by the pixel array in the second wavelength range. ### Claim 9 (depends on 8) The image sensing apparatus of claim 8, wherein the one or more processing devices are configured to: present the representation of the image on one or more displays. ### Claim 10 (depends on 1) The image sensing apparatus of claim 1, wherein a portion of the first stop band overlaps with a portion of the second stop band. ### Claim 11 (depends on 1) The image sensing apparatus of claim 1, wherein the first wavelength range is within the visible wavelength range, and wherein the second wavelength range is outside the visible wavelength range. ### Claim 12 (depends on 1) The image sensing apparatus of claim 1, wherein the first and second wavelength ranges are in the range 400-2000 nm. ### Claim 13 A surgical system comprising: one or more displays; an image sensing apparatus configured to receive electromagnetic radiation reflected or transmitted from a surgical scene, the image sensing apparatus comprising: a pixel array comprising two or more photo-sensor elements, a first optical filter disposed on a first photo-sensor element of the pixel array, the first optical filter configured such that a spectral response of the first optical filter includes: a first passband in a first wavelength range, and a second passband in a second wavelength range, the first passband and the second passband being separated by a first stop band, and a second optical filter disposed on a second photo-sensor element of the pixel array, the second optical filter configured such that a spectral response of the second optical filter includes: a third passband in the first wavelength range, and a fourth passband in the second wavelength range, the third passband and the fourth passband being separated by a second stop band, wherein the second passband and the fourth passband are discrete passbands in an infrared range; one or more processing devices configured to: obtain a representation of a first image of a surgical scene using electromagnetic radiation in the first wavelength range, obtain a representation of a second image of the surgical scene using electromagnetic radiation in the second wavelength range, and present a visual representation of the surgical scene on the one or more displays, wherein the visual representation of the surgical scene is rendered using the representation of the first image and the representation of the second image; and an input device configured to receive a user-input for controlling at least a portion of a surgical device. ### Claim 14 (depends on 13) The surgical system of claim 13, wherein: the second passband has a wavelength between 1200 and 1250 nm; and the fourth passband has a wavelength between 1300 and 1350 nm. ### Claim 15 (depends on 13) The surgical system of claim 13, wherein the first optical filter is configured such that an absorption characteristic of a first tissue type of a surgical site in the second passband is higher than an absorption characteristic of a second tissue type of a surgical site in the second passband, and the second optical filter is configured such that an absorption characteristic of the second tissue type in the fourth passband is higher than an absorption characteristic of the first tissue type in the fourth passband. ### Claim 16 (depends on 13) The surgical system of claim 13, further comprising: an illumination apparatus configured to illuminate the surgical scene using electromatic radiation in a first wavelength range and a second wavelength range; wherein the representation of the first image is obtained responsive to illuminating the surgical scene using electromagnetic radiation in the first wavelength range; and wherein the representation of the second image is obtained responsive to illuminating the surgical scene using electromagnetic radiation in the second wavelength range. ### Claim 17 (depends on 16) The surgical system of claim 16, wherein illuminating the surgical scene using electromagnetic radiation in the first wavelength range and the second wavelength range comprises: illuminating the surgical scene using electromagnetic radiation in the first wavelength range during a first time period; and illuminating the surgical scene using electromagnetic radiation in the second wavelength range during a second time period that is at least partially non-overlapping with the first time period. ### Claim 18 (depends on 13) The surgical system of claim 13, wherein the representation of the second image is obtained at substantially the same time as the representation of the first image. ### Claim 19 A method of providing visual feedback during a surgical process using a visual representation of a surgical scene rendered on one or more displays associated with a surgical device, the method comprising: illuminating a surgical scene using electromagnetic radiation in a first illumination-wavelength range; obtaining, responsive to illuminating the surgical scene using electromagnetic radiation in the first illumination-wavelength range, a representation of a first image of the surgical scene using a pixel array including two or more photo-sensor elements, wherein a first photo-sensor element of the two or more photo-sensor elements is configured to receive electromagnetic radiation through a first filter that includes a first passband in a first wavelength range and a second passband in a second wavelength range, the first passband and the second passband being separated by a first stop band, and wherein a second photo-sensor element of the two or more photo-sensor elements is configured to receive electromagnetic radiation through a second filter that includes a third passband in the first wavelength range and a fourth passband in the second wavelength range, the third passband and the fourth passband being separated by a second stop band, wherein the second passband and the fourth passband are discrete passbands in an infrared range; illuminating the surgical scene using electromagnetic radiation in a second illumination-wavelength range; obtaining, responsive to illuminating the surgical scene using electromagnetic radiation in the second illumination-wavelength range, a representation of a second image of the surgical scene using the pixel array; and presenting the visual representation of the surgical scene on the one or more displays, wherein the visual representation of the surgical scene is rendered using the representation of the first image and the representation of the second image. ### Claim 20 (depends on 19) The method of claim 19, wherein: the second passband has a wavelength between 1200 and 1250 nm; and the fourth passband has a wavelength between 1300 and 1350 nm.