# US 9,436,980 B2 · Reducing ghosting and other image artifacts in a wedge-based imaging system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,980 B2 - Document ID: US9436980B2 - Status: Granted - Kind: B2 - Application number: 13/648,281 - Filed: 2012-10-10 - Published: 2014-04-10 - Granted: 2016-09-06 - Earliest claimed: 2012-10-10 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Corporation - Inventors: Karlton Powell - Web page: https://patentgrep.com/patent/US9436980B2 ## CPC - G06T 5/73: Deblurring; Sharpening - G06T 5/80: Geometric correction - H04N 25/60: Noise processing, e.g. detecting, correcting, reducing or removing noise - H04N 25/626: Reduction of noise due to residual charges remaining after image readout, e.g. to remove ghost images or afterimages - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A computational image processing filter processes an image from a wedge-based imaging system so as to remove artifacts such as blurring and ghost images. By removing the artifacts computationally instead of optically, manufacturing costs and complexity are reduced over prior solutions. In one implementation, the computational image processing filter performs a two-dimensional transform to align a ghost image with a pixel grid defined by the wedge. The transformed image is then stretched using a nonlinear transform to make the ghost pitch versus position a constant. Next, an anti-ghost point spread filter is created and deconvolved. Finally, an inverse of the nonlinear mapping is applied. Artifacts introduced by other optical layers can be reduced by deconvolving the artifact from the image according to a point-spread function representing an effect of the optical layers on the image. ## Claims ### Claim 1 A process comprising: receiving an image from a sensor into memory of the computer, wherein the sensor is configured to receive output light from a window of an optical wedge and output the image representative of the output light, wherein the optical wedge comprises a face configured to receive light from objects and the window configured to output light representative of the objects; processing the image with a processor of the computer so as to substantially reduce ghost image data by applying a filter that reverses optical effects of the optical wedge which generate the ghost image data in a wedge-based imaging system by moving ghost pixels from points in the image to points from which the ghost pixels originated; and outputting the processed image to the memory of the computer. ### Claim 2 (depends on 1) The process of claim 1, wherein outputting comprises outputting the processed image for access by an object processing system to identify one or more objects in the image. ### Claim 3 (depends on 1) The process of claim 1, wherein the wedge based imaging system includes one or more optical layers above the optical wedge which introduces one or more artifacts and wherein processing of the image includes deconvolving the artifact from the image according to a point-spread function representing an effect of the optical layers on the image. ### Claim 4 (depends on 1) The process of claim 1, wherein processing the image comprises: applying a two-dimensional transform to the image to align the ghost image data with a pixel grid defined by the optical wedge. ### Claim 5 (depends on 4) The process of claim 4, wherein processing the image further comprises: applying a point-spread filter to the transformed image to substantially remove the ghost image data from the transformed image. ### Claim 6 (depends on 5) The process of claim 5, wherein processing the image further comprises: applying a transform to the transformed image with the ghost image data substantially removed so as to transform the image back to an original orientation. ### Claim 7 (depends on 6) The process of claim 6, wherein the wedge based imaging system includes one or more optical layers above the optical wedge which introduces one or more artifacts and wherein processing the image includes deconvolving the artifact from the image according to a point-spread function representing an effect of the optical layers on the image. ### Claim 8 An article of manufacture comprising: a computer storage medium comprising a memory device or a storage device; computer program instructions stored on the computer storage medium which, when processed by a processor of a computer, instruct the computer to perform a process comprising: controlling a wedge-based imaging system connected to the computer to capture an image, the wedge-based imaging system comprising an optical wedge and a sensor, wherein the optical wedge comprises a face configured to receive light from objects and a window configured to output light representative of the objects, wherein the sensor is configured to receive the output light from the window of the wedge and output the image to the computer, the image being representative of the output light from the window of the wedge; receiving the image from the sensor into memory of the computer; processing the image with a processor of the computer so as to substantially reduce ghost image data by applying a filter that reverses optical effects of the optical wedge which generate the ghost image data in the wedge-based imaging system by moving ghost pixels from points in the image to points from which the ghost pixels originated; and outputting the processed image to the memory of the computer. ### Claim 9 (depends on 8) The article of manufacture of claim 8, wherein outputting comprises outputting the processed image for access by an object processing system to identify one or more objects in the image. ### Claim 10 (depends on 8) The article of manufacture of claim 8, wherein the wedge based imaging system includes one or more optical layers above the optical wedge which introduces one or more artifacts and wherein processing the image includes deconvolving the artifact from the image according to a point-spread function representing an effect of the optical layers on the image. ### Claim 11 (depends on 8) The article of manufacture of claim 8, wherein processing the image comprises: applying a two-dimensional transform to the image to align the ghost image data with a pixel grid defined by the optical wedge. ### Claim 12 (depends on 11) The article of manufacture of claim 11, wherein processing the image further comprises: applying a point-spread filter to the transformed image to substantially remove the ghost image data from the transformed image. ### Claim 13 (depends on 12) The article of manufacture of claim 12, wherein processing the image further comprises: applying a transform to the transformed image with the ghost image data substantially removed so as to transform the image back to an original orientation. ### Claim 14 (depends on 8) The article of manufacture of claim 8, wherein computationally reversing the optical effects of the wedge-based imaging system further comprises filtering the image based on presumed locations of ghost data generated by the optical wedge. ### Claim 15 A wedge-based imaging system, comprising: an optical wedge having a face configured to receive light from objects and a window configured to output light representative of the objects; a sensor configured to receive the output light from the window of the wedge and output image data representative of the output light; and a computational image processing system having an input for receiving the image data from the sensor and an output providing the image data with artifacts substantially reduced by applying a filter that reverses optical effects of the optical wedge which generate the ghost image data in the wedge-based imaging system by moving ghost pixels from points in the image to points from which the ghost pixels originated. ### Claim 16 (depends on 15) The wedge-based imaging system of claim 15, wherein outputting comprises outputting the image for access by an object processing system to identify one or more objects in the image. ### Claim 17 (depends on 15) The wedge-based imaging system of claim 15, wherein outputting comprises outputting the image for display on a display. ### Claim 18 (depends on 15) The wedge-based imaging system of claim 15, wherein processing the image comprises: applying a two-dimensional transform to the image data to align ghost image data by position. ### Claim 19 (depends on 18) The wedge-based imaging system of claim 18, wherein processing the image further comprises: applying a point-spread filter to the transformed image to substantially remove ghost data from the transformed image. ### Claim 20 (depends on 19) The wedge-based imaging system of claim 19, wherein processing the image further comprises applying a transform to the transformed image with the ghost data substantially removed so as to transform the image back to an original orientation. ### Claim 21 (depends on 19) The wedge-based imaging system of claim 19, wherein processing the image further comprises applying a transform to the transformed image with the ghost data substantially removed so as to transform the image back to alignment with the original input image. ### Claim 22 (depends on 21) The wedge-based imaging system of claim 21, further comprising one or more optical layers above the optical wedge and that introduces one or more artifacts, and wherein the processing includes deconvolving the one or more artifacts from the image according to a point-spread function representing an effect of the optical layers on the image. ### Claim 23 (depends on 19) The wedge-based imaging system of claim 19, wherein processing the image further comprises applying a transform to the transformed image with the ghost data substantially removed so as to transform the image back in its original grid. ### Claim 24 (depends on 15) The wedge-based imaging system of claim 15, wherein processing the image further comprises filtering the image based on presumed locations of ghost data. ### Claim 25 (depends on 15) The wedge-based imaging system of claim 15, further comprising one or more optical layers above the optical wedge and that introduces one or more artifacts, and wherein the processing includes deconvolving the one or more artifacts from the image according to a point-spread function representing an effect of the optical layers on the image.