# US 2014/0192048 A1 · Device, Method and Computer Program for Generating a Synthesized Image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0192048 A1 - Document ID: US20140192048A1 - Status: Published application - Kind: A1 - Application number: 14/237,044 - Filed: 2011-10-31 - Published: 2014-07-10 - Earliest claimed: 2011-08-09 - Assignees: Telefonaktiebolaget L M Ericsson (publ) - Inventors: Björn Johansson; Martin Pettersson; Thomas Rusert - Web page: https://patentgrep.com/patent/US20140192048A1 ## CPC - G06T 15/00: Three-dimensional [3D] image rendering - G06T 15/205: {Image-based rendering} - G06T 15/503: {Blending, e.g. for anti-aliasing} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The invention relates to a method of generating a synthesized image representing a view of a scene from a first input image representing the view and a second input image representing the view, the synthesized image comprising synthesized image positions, by assigning a synthesized image data value to a synthesized image position. The method comprises determining whether input images have at the synthesised image position data values associated with them. If either one has a data value associated, that data value is assigned to the synthesized image position. If both, an average of both values is assigned. The average is a weighed average, with weighing factors being a function of a distance or distances to a closest image position with no image data value or valid image data value assigned. ## Claims ### Claim at position 1 1-14. (canceled) ### Claim 15 A method of generating a synthesized image representing a view of a scene from a first input image representing the view and a second input image representing the view, the synthesized image comprising synthesized image positions, by assigning a synthesized image data value to a synthesized image position, comprising: determining whether the first input image has a first image data value assigned to a first input image position of the first input image corresponding to the synthesized image position; determining whether the second input image has a second image data value assigned to a second input image position of the second input image corresponding to the synthesized image position; assigning the first image data value to the synthesized image position if the first image position has the first image data value assigned to it and the second image position has no image data value assigned to it; assigning the second image data value to the synthesized image position if the second image position has the second image data value assigned to it and the first image position has no image data value assigned to it; assigning a synthesized image data value to the synthesized image position if the first input image position has a first image data value assigned to it and the second input image position has a second image data value assigned to it, determining of the synthesized image data value comprising: determining a distance between the synthesized image position and a further image position where a corresponding further first input image position or a corresponding further second input image position does not have an image data value assigned to it; and calculating the synthesized image data value by calculating a weighted average of the first image data value and the second image data value, the weighting being a weighting function of the determined distance. ### Claim 16 (depends on 15) The method of claim 15, comprising: if the first image position has the first image data value assigned to it, determining for the first input image position a first validity distance between the first image position and a first non-value image position of the first input image, which first non-value image position is an image position of the first input image that is closest to the first image position and has no image value assigned to it; and, if the second image position has the second image data value assigned to it, determining for the second input image position a second validity distance between the second image position and a second non-value image position of the second input image, which second non-value image position is an image position of the second input image that is closest to the second image position and has no image value assigned to it; wherein the weighting function is a function of the first validity distance and the second validity distance. ### Claim 17 (depends on 16) The method of claim 16, wherein the weighting function is a continuous function. ### Claim 18 (depends on 16) The method of claim 16, wherein the weighting function is a linear function of the first validity distance and the second validity distance. ### Claim 19 (depends on 16) The method of claim 16, wherein the weighting function has continuous derivatives of all orders. ### Claim 20 (depends on 16) The method of claim 16, wherein the weighting function is constant for a part of, but not all, values of the first validity distance and the second validity distance. ### Claim 21 (depends on 20) The method of claim 20, wherein the weighting function is constant if the first validity distance and the second validity distance are above a pre-determined distance threshold. ### Claim 22 (depends on 15) The method of claim 15, wherein the first input image has a first input depth map associated with it and the second input image a second input depth map associated with it, the method comprising, if the first input image position has a first image data value assigned to it and the second input image position has a second image data value assigned to it: retrieving a first depth value from the input first depth map corresponding to the first input image position; retrieving a second depth value from the second input depth map corresponding to the second input image position; determining a depth difference between the first depth value and the second depth value; determining which of the first input image position and the second input image position corresponds to a foreground object if the depth difference is above a pre-determined depth threshold; and assigning an image data value to the synthesized image position that corresponds to an input image data value of the input image position corresponding to a foreground object. ### Claim 23 (depends on 15) The method of claim 15, wherein the first input image has a first input depth map associated with it and the second input image a second input depth map associated with it, the first input depth map comprising depth data values corresponding to first input image positions and the second input depth map comprising depth data values corresponding to second input image positions, the method further comprising: determining whether the first input depth map has a first depth data value assigned to the first input image position; determining whether the second input depth map has a second depth data value assigned to the second input image position; assigning the first depth data value to a synthesized depth map position of a synthesized depth map if the first image position has the first depth data value assigned to it and the second image position has no depth data value assigned to it, the synthesized depth map being associated with the synthesized image and the synthesized depth map position corresponding to the synthesized image position; assigning the second depth data value to the synthesized depth map position if the second image position has the second depth data value assigned to it and the first image position has no depth data value assigned to it; assigning a synthesized depth data value to the synthesized depth map position if the first input image position has a first depth data value assigned to it and the second input image position has a second depth data value assigned to it, determining of the synthesized depth data value comprising: determining a distance between the synthesized image position and a further image position where a corresponding further first input image position or a corresponding further second input image position does not have an depth data value assigned to it; calculating the synthesized depth data value by calculating a weighted average of the first depth data value and the second depth data value, the weighting being a weighting function of the determined distance. ### Claim 24 (depends on 15) The method of claim 15, wherein the view is a three-dimensional view and the scene is a three-dimensional scene, the method comprising: receiving a first raw input image representing a first raw view of the three-dimensional scene and a first raw depth map associated with the first raw input image; receiving a second raw input image representing a second raw view of the three-dimensional scene and a second raw depth map associated with the second raw input image; receiving a virtual camera position corresponding to a virtual view of the three-dimensional scene; generating the first input image from the first raw input image by calculation of shifts of first image data positions associated with a change of view point from the first raw view to the virtual view; and generating the second input image from the second raw input image by calculation of shifts of first image data positions associated with a change of view point from the second raw view to the virtual view. ### Claim 25 (depends on 15) The method of claim 15, wherein image data positions are single pixels. ### Claim 26 (depends on 15) The method of claim 15, wherein image data positions are groups of pixels. ### Claim 27 A device for generating a synthesized image representing a view of a scene from a first input image representing the view and a second input image representing the view, the synthesized image comprising synthesized image positions by assigning a synthesized image data value to a synthesized image position, the device comprising a receiving unit arranged to receive the first input image and the second input image and a processing unit arranged to: determine whether the first input image has a first image data value assigned to a first input image position of the first input image corresponding to the synthesized image position; determine whether the second input image has a second image data value assigned to a second input image position of the second input image corresponding to the synthesized image position; assign the first image data value to the synthesized image position if the first image position has the first image data value assigned to it and the second image position has no image data value assigned to it; assign the second image data value to the synthesized image position if the second image position has the second image data value assigned to it and the first image position has no image data value assigned to it; assign a synthesized image data value to the synthesized image position if the first input image position has a first image data value assigned to it and the second input image position has a second image data value assigned to it, determining of the synthesized image data value comprising: determine a distance between the synthesized image position and a further image position where a corresponding further first input image position or a corresponding further second input image position does not have an image data value assigned to it; and calculate the synthesized image data value by calculating a weighted average of the first image data value and the second image data value, the weighting being a weighting function of the determined distance. ### Claim 28 A non-transitory computer-readable medium comprising, stored thereupon, a computer program for generating a synthesized image representing a view of a scene from a first input image representing the view and a second input image representing the view, the synthesized image comprising synthesized image positions, by assigning a synthesized image data value to a synthesized image position, the computer program comprising software instructions that, when executed by a computer, cause the computer to: determine whether the first input image has a first image data value assigned to a first input image position of the first input image corresponding to the synthesized image position; determine whether the second input image has a second image data value assigned to a second input image position of the second input image corresponding to the synthesized image position; assign the first image data value to the synthesized image position if the first image position has the first image data value assigned to it and the second image position has no image data value assigned to it; assign the second image data value to the synthesized image position if the second image position has the second image data value assigned to it and the first image position has no image data value assigned to it; assign a synthesized image data value to the synthesized image position if the first input image position has a first image data value assigned to it and the second input image position has a second image data value assigned to it, determining of the synthesized image data value comprising: determine a distance between the synthesized image position and a further image position where a corresponding further first input image position or a corresponding further second input image position does not have an image data value assigned to it; and calculate the synthesized image data value by calculating a weighted average of the first image data value and the second image data value, the weighting being a weighting function of the determined distance.