# US 10,771,766 B2 · Method and apparatus for active stereo vision > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,766 B2 - Document ID: US10771766B2 - Status: Granted - Kind: B2 - Application number: 16/359,721 - Filed: 2019-03-20 - Published: 2019-10-03 - Granted: 2020-09-08 - Earliest claimed: 2018-03-30 - Assignees: Mediatek Inc. - Applicants: MediaTek Inc. - Inventors: Han-Yang Wang; Yu-Chun Chen; Po-Hao Huang; Chao-Chung Cheng; Ying-Jui Chen; Te-Hao Chang - Web page: https://patentgrep.com/patent/US10771766B2 ## CPC - H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) - H04N 13/25: using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor - H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects - H04N 13/257: Colour aspects - H04N 23/20: for generating image signals from infrared radiation only - H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Various examples with respect to method and apparatus for active stereo vision are described. An apparatus may include an electromagnetic (EM) wave emitter, a first sensor and a second sensor. During operation, the EM wave emitter emits EM waves toward a scene, the first sensor captures a first image of the scene in an infrared (IR) spectrum, and the second sensor captures a second image of the scene in a light spectrum. The first image and second image, when processed, may enable active stereo vision. ## Claims ### Claim 1 An apparatus, comprising: an electromagnetic (EM) wave emitter which, during operation, emits EM waves toward a scene; a first sensor which, during operation, captures a first image of the scene in an infrared (IR) spectrum; and a second sensor which, during operation, captures a second image of the scene in a light spectrum, wherein the first image and second image, when processed, enable active stereo vision, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, and wherein each of the first sensor and the second sensor respectively comprises an IR camera, a red-green-blue (RGB) camera containing one or more pixels capable of receiving light in the IR spectrum, a monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through. ### Claim 2 (depends on 1) The apparatus of claim 1, further comprising: a third sensor which, during operation, captures a third image of the scene in the IR spectrum, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the first sensor comprises a first IR camera, wherein the second sensor comprises a RGB camera, wherein the third sensor comprises a second IR camera, and wherein the first sensor and the second sensor are physically adjacent to one another. ### Claim 3 (depends on 1) The apparatus of claim 1, further comprising: a third sensor which, during operation, captures a third image of the scene in the IR spectrum, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the first sensor comprises a first IR camera, a first RGB camera containing one or more pixels capable of receiving light in the IR spectrum, a monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, wherein the second sensor comprises a second RGB camera, wherein the third sensor comprises a second IR camera, and wherein the first sensor and the second sensor are physically adjacent to one another. ### Claim 4 (depends on 1) The apparatus of claim 1, further comprising: a third sensor which, during operation, captures a third image of the scene in the IR spectrum, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the first sensor comprises a first IR camera, wherein the second sensor comprises a first RGB camera, wherein the third sensor comprises a second IR camera, a second RGB camera containing one or more pixels capable of receiving light in the IR spectrum, a monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, and wherein the first sensor and the second sensor are physically adjacent to one another. ### Claim 5 (depends on 1) The apparatus of claim 1, further comprising: a third sensor which, during operation, captures a third image of the scene in the IR spectrum, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the first sensor comprises a first IR camera, a first RGB camera containing one or more pixels capable of receiving light in the IR spectrum, a first monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a first RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a first monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, wherein the second sensor comprises a second RGB camera, wherein the third sensor comprises a second IR camera, a second RGB camera containing one or more pixels capable of receiving light in the IR spectrum, a second monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a second RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a second monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, and wherein the first sensor and the second sensor are physically adjacent to one another. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the EM wave emitter further comprises a light-emitting diode (LED), and wherein the LED emits a floodlight during operation. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the EM wave emitter further comprises a ring-shaped IR light, wherein the ring-shaped IR light surrounds the IR projector, the first sensor and the second sensor, and wherein the ring-shaped IR light emits a floodlight during operation. ### Claim 8 (depends on 1) The apparatus of claim 1, further comprising: a display panel which emits a floodlight during operation, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the display panel comprises a liquid crystal display (LCD), wherein the EM wave emitter, the first sensor and the second sensor are disposed under a display surface of the LCD, and wherein, during operation, the EM wave emitter emits the structured IR light and the first sensor and the second sensor capture the first image and the second image, respectively, when the display surface of the LCD is transparent. ### Claim 10 A method, comprising: controlling an electromagnetic (EM) wave emitter to emit EM waves toward a scene; receiving from a first sensor a first image of the scene in an infrared (IR) spectrum; receiving from a second sensor a second image of the scene in a light spectrum; and performing depth estimation using active stereo vision based on the first image and the second image, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, and wherein the second sensor comprises a red-green-blue (RGB) camera containing one or more pixels capable of receiving light in the IR spectrum, a monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through. ### Claim 11 (depends on 10) The method of claim 10, further comprising: receiving from a third sensor a third image of the scene in the IR spectrum, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the first sensor comprises a first IR camera, wherein the second sensor comprises a RGB camera, wherein the third sensor comprises a second IR camera, and wherein the first sensor and the second sensor are physically adjacent to one another. ### Claim 12 (depends on 10) The method of claim 10, further comprising: receiving from a third sensor a third image of the scene in the IR spectrum, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the first sensor comprises a first IR camera, a first RGB camera containing one or more pixels capable of receiving light in the IR spectrum, a monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, wherein the second sensor comprises a second RGB camera, wherein the third sensor comprises a second IR camera, and wherein the first sensor and the second sensor are physically adjacent to one another. ### Claim 13 (depends on 10) The method of claim 10, further comprising: receiving from a third sensor a third image of the scene in the IR spectrum, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the first sensor comprises a first IR camera, wherein the second sensor comprises a first RGB camera, wherein the third sensor comprises a second IR camera, a second RGB camera containing one or more pixels capable of receiving light in the IR spectrum, a monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, and wherein the first sensor and the second sensor are physically adjacent to one another. ### Claim 14 (depends on 10) The method of claim 10, further comprising: receiving from a third sensor a third image of the scene in the IR spectrum, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the first sensor comprises a first IR camera, a first RGB camera containing one or more pixels capable of receiving light in the IR spectrum, a first monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a first RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a first monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, wherein the second sensor comprises a second RGB camera, wherein the third sensor comprises a second IR camera, a second RGB camera containing one or more pixels capable of receiving light in the IR spectrum, a second monochrome camera containing one or more pixels capable of receiving light in the IR spectrum, a second RGB camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, or a second monochrome camera with dual-band bandpass filtering to allow light in the visible spectrum and the IR spectrum to pass through, and wherein the first sensor and the second sensor are physically adjacent to one another. ### Claim 15 (depends on 10) The method of claim 10, wherein the EM wave emitter comprises an IR projector and a light-emitting diode (LED), wherein the IR projector projects a structured IR light during operation, and wherein the LED emits a floodlight during operation. ### Claim 16 (depends on 10) The method of claim 10, wherein the EM wave emitter comprises an IR projector and a ring-shaped IR light, wherein the IR projector projects a structured IR light during operation, wherein the ring-shaped IR light surrounds the IR projector, the first sensor and the second sensor, and wherein the ring-shaped IR light emits a floodlight during operation. ### Claim 17 (depends on 10) The method of claim 10, further comprising: controlling a display panel to emit a floodlight, wherein the EM wave emitter comprises an IR projector which projects a structured IR light during operation, wherein the display panel comprises a liquid crystal display (LCD), wherein the EM wave emitter, the first sensor and the second sensor are disposed under a display surface of the LCD, and wherein, during operation, the EM wave emitter emits the structured IR light and the first sensor and the second sensor capture the first image and the second image, respectively, when the display surface of the LCD is transparent.