# US 10,371,943 B2 · Method and system to control the focus depth of projected images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,371,943 B2 - Document ID: US10371943B2 - Status: Granted - Kind: B2 - Application number: 13/831,080 - Filed: 2013-03-14 - Published: 2014-09-18 - Granted: 2019-08-06 - Earliest claimed: 2013-03-14 - Assignees: Valve Corporation - Applicants: Valve Corporation - Inventors: Jeri Janet Ellsworth - Web page: https://patentgrep.com/patent/US10371943B2 ## CPC - G02B 27/0172: {characterised by optical features} - G02B 27/017: {Head mounted} - G02B 2027/0127: {comprising devices increasing the depth of field} - G02B 2027/0147: {comprising a device modifying the resolution of the displayed image} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract Methods and systems are disclosed for controlling the focus depth of a 2D projected image on a pixel by pixel basis or on a region by region basis to create a 3D projected image that may be used for a heads up display (HUD) for augmented reality applications. The 3D projected image may be overlaid or combined with a 3D real world view using multiple reflective LCD arrays. The multiple reflective LCD arrays may receive a 2D projected image and may generate different length optical paths that may add depth to the 2D projected image to create a 3D projected image. The 3D projected image may be combined with the real world 3D image to create a 3D image encompassing a real world image and a 3D projection image that looks real and contains depth within the image. ## Claims ### Claim 1 An image combiner system, comprising: a 2D image projector; a 2D to 3D image converter that receives a 2D projected image from the 2D image projector and converts the 2D projected image to a 3D projected image; and an image combiner that receives the 3D projected image from the 2D to 3D image converter and combines the 3D projected image with a 3D real world image; wherein the 2D to 3D image converter comprises a reflective LCD pixel array and a reflective screen, the reflective LCD pixel array positioned in front of the reflective screen relative to the 2D image projector to initially receive the 2D projected image from the 2D image projector, and the reflective LCD pixel array comprises a plurality of LCD pixel shutters, each of the LCD pixel shutters is programmable to either a closed position where the LCD pixel shutter is reflective and reflects a portion of the 2D projected image toward the image combiner, or an open position where the LCD pixel shutter is transparent and passes a portion of the 2D projected image toward the reflective screen to be reflected thereby toward the image combiner, such that portions of the 2D projected image reflected by the LCD pixel shutters have a shorter optical path length than portions of the 2D projected image reflected by the reflective screen to convert the 2D projected image into the 3D projected image provided to the image combiner. ### Claim 2 (depends on 1) The image combiner system of claim 1, wherein the reflective LCD pixel array is a first reflective LCD pixel array, and the 2D to 3D image converter comprises: a second reflective LCD pixel array positioned in front of the first reflective LCD pixel array, the second reflective LCD pixel array comprising a plurality of LCD pixel shutters, each of the LCD pixel shutters is programmable to either a closed position where the LCD pixel shutter is reflective and reflects a portion of the 2D projected image toward the image combiner, or an open position where the LCD pixel shutter is transparent and passes a portion of the 2D projected image toward the first reflective LCD pixel array. ### Claim 3 (depends on 1) The image combiner system of claim 1, wherein the reflective LCD pixel array comprises M rows and N columns of LCD pixel shutters. ### Claim 4 (depends on 2) The image combiner system of claim 2, wherein the 2D to 3D image converter comprises: a third reflective LCD pixel array positioned in front of the second reflective LCD pixel array, the third reflective LCD pixel array comprising a plurality of LCD pixel shutters, each of the LCD pixel shutters is programmable to either a closed position where the LCD pixel shutter is reflective and reflects a portion of the 2D projected image toward the image combiner, or an open position where the LCD pixel shutter is transparent and passes a portion of the 2D projected image toward the second reflective LCD pixel array. ### Claim 5 (depends on 1) The image combiner system of claim 1, wherein each of the plurality of LCD pixel shutters is programmable to reflect at least one group of pixels from the 2D projected image. ### Claim 6 (depends on 1) The image combiner system of claim 1, wherein each of the plurality of LCD pixel shutters is programmable to permit at least one pixel from the 2D projected image to pass transparently through the reflective LCD pixel array. ### Claim 7 (depends on 1) The image combiner system of claim 1, wherein each of the plurality of LCD pixel shutters is programmable to permit at least one group of pixels from the 2D projected image to pass transparently through the reflective LCD pixel array. ### Claim 8 (depends on 1) The image combiner system of claim 1, wherein the image combiner comprises two triangularly shaped prisms coupled together. ### Claim 9 (depends on 1) The image combiner system of claim 1, wherein the 2D to 3D image converter includes a focus optics element positioned in front of the reflective LCD pixel array. ### Claim 10 (depends on 1) The image combiner system of claim 1, wherein each LCD pixel shutter of the plurality of LCD pixel shutters selectively reflects or passes a single pixel of the 2D projected image. ### Claim 11 (depends on 1) The image combiner system of claim 1, wherein each LCD pixel shutter of the plurality of LCD pixel shutters selectively reflects or passes a group of pixels of the 2D projected image. ### Claim 12 A method of combining images, comprising: projecting a 2D image; converting the 2D image to a 3D projected image; and combining, via an image combiner, the 3D projected image with a 3D real world image; wherein the converting is performed at least in part by a reflective LCD pixel array and a reflective screen, the reflective LCD pixel array positioned in front of the reflective screen and comprises a plurality of LCD pixel shutters, each of the LCD pixel shutters is programmable to either a closed position where the LCD pixel shutter is reflective and reflects a portion of the 2D projected image toward the image combiner, or an open position where the LCD pixel shutter is transparent and passes a portion of the 2D projected image toward the reflective screen to be reflected thereby toward the image combiner, such that portions of the 2D projected image reflected by the LCD pixel shutters have a shorter optical path length than portions of the 2D projected image reflected by the reflective screen to convert the 2D projected image into the 3D projected image provided to the image combiner. ### Claim 13 (depends on 12) The method of claim 12, wherein the reflective LCD pixel array comprises a first reflective LCD pixel array, and the converting is performed at least in part by: a second reflective LCD pixel array positioned in front of the first reflective LCD pixel array, the second reflective LCD pixel array comprising a plurality of LCD pixel shutters, each of the LCD pixel shutters is programmable to either a closed position where the LCD pixel shutter is reflective and reflects a portion of the 2D projected image toward the image combiner, or an open position where the LCD pixel shutter is transparent and passes a portion of the 2D projected image toward the first reflective LCD pixel array. ### Claim 14 (depends on 13) The method of claim 13, wherein the converting is performed at least in part by: a third reflective LCD pixel array positioned in front of the second reflective LCD pixel array, the third reflective LCD pixel array comprising a plurality of LCD pixel shutters, each of the LCD pixel shutters is programmable to either a closed position where the LCD pixel shutter is reflective and reflects a portion of the 2D projected image toward the image combiner, or an open position where the LCD pixel shutter is transparent and passes a portion of the 2D projected image toward the second reflective LCD pixel array. ### Claim 15 (depends on 12) The method of claim 12, wherein the plurality of LCD pixel shutters comprises M rows and N columns of LCD pixel shutters. ### Claim 16 (depends on 12) The method of claim 12, wherein the converting programming at least one of the plurality of LCD pixel shutters to reflect at least one pixel from the 2D image away from the reflective LCD pixel array. ### Claim 17 (depends on 12) The method of claim 12, wherein the converting programming at least one of the plurality of LCD pixel shutters to reflect at least one group of pixels from the 2D image away from the reflective LCD pixel array. ### Claim 18 (depends on 12) The method of claim 12, wherein the converting programming at least one of the plurality of LCD pixel shutters to permit at least one pixel from the 2D image to pass transparently through the reflective LCD pixel array.