# US 10,484,623 B2 · Sensor with alternating visible and infrared sensitive pixels > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,623 B2 - Document ID: US10484623B2 - Status: Granted - Kind: B2 - Application number: 15/385,713 - Filed: 2016-12-20 - Published: 2018-06-21 - Granted: 2019-11-19 - Earliest claimed: 2016-12-20 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Raymond Kirk Price; Denis Demandolx; Michael Bleyer; Jian Zhao - Web page: https://patentgrep.com/patent/US10484623B2 ## CPC - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N 25/534: depending on the spectral component - H10F 39/103: {the at least one element covered by H10F30/00 having potential barriers, e.g. integrated devices comprising photodiodes or phototransistors} - H10F 39/8033: {Photosensitive area} - H10F 39/8053: {Colour filters} - H10F 39/182: {Colour image sensors} - H10F 39/184: {Infrared image sensors} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H10F: INORGANIC SEMICONDUCTOR DEVICES SENSITIVE TO INFRARED RADIATION, LIGHT, ELECTROMAGNETIC RADIATION OF SHORTER WAVELENGTH OR CORPUSCULAR RADIATION ## Abstract A sensor device is provided that includes an array of light sensitive pixels configured in a predetermined pattern of light sensitive pixels that alternates between broadband visible light sensitive pixels and infrared light sensitive pixels, each broadband visible light sensitive pixel being configured to have increased sensitivity to light of the visible light spectrum and to have reduced sensitivity to light of the infrared light spectrum compared to each infrared light sensitive pixel, and each infrared light sensitive pixel being configured to have increased sensitivity to light of the infrared light spectrum and to have reduced sensitivity to light of the visible light spectrum compared to each broadband visible light sensitive pixel. ## Claims ### Claim 1 A sensor device, comprising: an array of light sensitive pixels arranged in a predetermined pattern of light sensitive pixels that alternates between broadband visible light sensitive pixels and infrared light sensitive pixels; a color filter array of visible light filters and infrared light filters; each broadband visible light sensitive pixel including a respective infrared light filter of the color filter array that causes that broadband visible light sensitive pixel to have increased sensitivity to light of the visible light spectrum and to have reduced sensitivity to light of the infrared light spectrum compared to each infrared light sensitive pixel; and each infrared light sensitive pixel including a respective visible light filter of the color filter array that causes that infrared light sensitive pixel to have increased sensitivity to light of the infrared light spectrum and to have reduced sensitivity to light of the visible light spectrum compared to each broadband visible light sensitive pixel. ### Claim 2 (depends on 1) The sensor device of claim 1, wherein the broadband visible light sensitive pixels are configured to have increased sensitivity to light in the 400 nm to 700 nm range, and the infrared light sensitive pixels are configured to have increased sensitivity to light in the 700 nm to 1000 nm range. ### Claim 3 (depends on 1) The sensor device of claim 1, wherein relative sensitivities to light of the light sensitive pixels includes a band gap in the 650 nm to 750 nm range, wherein both the broadband visible light sensitive pixels and the infrared light sensitive pixels are configured to have reduced sensitivity to light in the band gap range. ### Claim 4 (depends on 1) The sensor device of claim 1, further comprising a lens that includes an infrared bandpass filter configured to isolate ambient infrared light from the infrared light sensitive pixels. ### Claim 5 (depends on 1) The sensor device of claim 1, wherein the predetermined pattern of light sensitive pixels is a checkerboard pattern. ### Claim 6 (depends on 1) The sensor device of claim 1, wherein the predetermined pattern of light sensitive pixels alternates between columns of broadband visible light sensitive pixels and columns of infrared light sensitive pixels. ### Claim 7 (depends on 1) The sensor device of claim 1, wherein the predetermined pattern of light sensitive pixels alternates between rows of broadband visible light sensitive pixels and rows of infrared light sensitive pixels. ### Claim 8 (depends on 1) The sensor device of claim 1, wherein each broadband visible light sensitive pixel includes two or more differential visible light sensitive pixels, and each infrared light sensitive pixel includes two or more differential infrared light sensitive pixels. ### Claim 9 (depends on 1) The sensor device of claim 1, wherein the broadband visible light sensitive pixels are configured to output grayscale values. ### Claim 10 (depends on 1) The sensor device of claim 1, wherein the infrared light sensitive pixels are configured to be globally shuttered. ### Claim 11 (depends on 1) The sensor device of claim 1, wherein the infrared light sensitive pixels and the broadband visible light sensitive pixels are configured to be rolling shuttered. ### Claim 12 (depends on 1) The sensor device of claim 1, further configured to control a first shutter speed for the broadband visible light sensitive pixels independently of a second shutter speed for the infrared light sensitive pixels. ### Claim 13 (depends on 1) The sensor device of claim 1, further configured to concurrently expose the broadband visible light sensitive pixels and the infrared light sensitive pixels. ### Claim 14 (depends on 1) The sensor device of claim 1, wherein each light sensitive pixel is trenched such that each light sensitive pixel is optically isolated from each other light sensitive pixel of the array of light sensitive pixels. ### Claim 15 A method comprising, providing a sensor device including an array of light sensitive pixels arranged in a predetermined pattern of light sensitive pixels that alternates between broadband visible light sensitive pixels and infrared light sensitive pixels, a color filter array of visible light filters and infrared light filters, each broadband visible light sensitive pixel including a respective infrared light filter of the color filter array that causes that broadband visible light sensitive pixel to light of the visible light spectrum and to have reduced sensitivity to light of the infrared light spectrum compared to each infrared light sensitive pixel, and each infrared light sensitive pixel including a respective visible light filter of the color filter array that causes that infrared light sensitive pixel to have increased sensitivity to light of the infrared light spectrum and to have reduced sensitivity to light of the visible light spectrum compared to each broadband visible light sensitive pixel; at a processor, concurrently performing infrared sensing and tracking and simultaneous localization and mapping based on output of the sensor device. ### Claim 16 A head mounted display device comprising, a sensor device including an array of light sensitive pixels arranged in a predetermined pattern of light sensitive pixels that alternates between broadband visible light sensitive pixels and infrared light sensitive pixels, a color filter array of visible light filters and infrared light filters, each broadband visible light sensitive pixel including a respective infrared light filter of the color filter array that causes that broadband visible light sensitive pixel to have increased sensitivity to light of the visible light spectrum and to have reduced sensitivity to light of the infrared light spectrum compared to each infrared light sensitive pixel, and each infrared light sensitive pixel including a respective visible light filter of the color filter array that causes that infrared light sensitive pixel to have increased sensitivity to light of the infrared light spectrum and to have reduced sensitivity to light of the visible light spectrum compared to each broadband visible light sensitive pixel; and a processor configured to concurrently perform infrared sensing and tracking and simultaneous localization and mapping based on output of the sensor device. ### Claim 17 (depends on 16) The head mounted display device of claim 16, further comprising an active illuminator. ### Claim 18 (depends on 16) The head mounted display device of claim 16, wherein the sensor device is one of a plurality of sensor devices having the same configuration, and the plurality of sensor devices are located on the head mounted display device with overlapping field of views. ### Claim 19 (depends on 16) The head mounted display device of claim 16, wherein the sensor device is one of a plurality of sensor devices having the same configuration, and the plurality of sensor devices are located on the head mounted display device with non-overlapping field of views.