# US 10,311,633 B2 · Method and apparatus for visualization of geo-located media contents in 3D rendering applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,633 B2 - Document ID: US10311633B2 - Status: Granted - Kind: B2 - Application number: 15/111,212 - Filed: 2014-09-30 - Published: 2016-11-17 - Granted: 2019-06-04 - Earliest claimed: 2014-01-17 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Kimmo Roimela - Web page: https://patentgrep.com/patent/US10311633B2 ## CPC - G06T 17/05: Geographic models - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06T 15/00: Three-dimensional [3D] image rendering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An approach is provided for accurate processing and registering of media content for rendering in 3D maps and other applications. The approach includes determining at least one first pixel of at least one image that geometrically corresponds to at least one second pixel of at least one rendered three-dimensional map. Further, the approach includes processing and/or facilitating a processing of (a) the at least one first pixel; (b) the at least one second pixel; (c) metadata associated with at least one of the at least one first pixel and the second pixel; or (d) a combination thereof to determine at least one confidence value, wherein the at least one confidence value is indicative of an estimated level of geometric distortion resulting from projecting the at least one first pixel onto the at least one second pixel. Furthermore, the approach includes determining whether to cause, at least in part, a rendering of the at least one first pixel onto the at least one rendered three-dimensional map based, at least in part, on the confidence value. ## Claims ### Claim 1 An apparatus comprising: at least one processor; and at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following, determine each of a plurality of first pixels of an image that geometrically corresponds to a second pixel of a rendered three-dimensional map, wherein the three-dimensional map is a three-dimensional geographic map; process the each first pixel and the second pixel to determine a confidence value that increases as a transparency level for the rendering of the each first pixel onto the three-dimensional map decreases, wherein the confidence value indicates an estimated level of geometric distortion of the each first pixel as projected onto the three-dimensional map resulting from a difference between an image capturing perspective of the image with respect to the each first pixel and a virtual viewing perspective of the three-dimensional map; initiate a rendering of the first pixels onto the three-dimensional map from the virtual viewing perspective based on the confidence value; determine at least one user interaction with the three-dimensional map; and initiate an updating of the confidence value, the rendering of the first pixels onto the three-dimensional map, or a combination thereof in real time or substantially in real time based on the at least one user interaction, wherein the confidence value of a respective first pixel decreases as the virtual viewing perspective moves further away from the image capturing perspective. ### Claim 2 (depends on 1) An apparatus of claim 1, wherein the at least one user interaction results in a change in (a) the virtual viewing perspective for the three-dimensional map, (b) at least one rendering characteristic for at least one of the first pixels, the image, the second pixel, and the three-dimensional map; or (c) a combination thereof; and wherein the at least one rendering characteristic includes a fade parameter, a toggle parameter, a size parameter, or a combination thereof. ### Claim 3 (depends on 1) An apparatus of claim 1, wherein the apparatus is further caused to: initiate a comparison of the confidence value against at least one threshold confidence criterion to determine whether to initiate the rendering of the first pixels onto the three-dimensional map. ### Claim 4 (depends on 1) An apparatus of claim 1, wherein the apparatus is further caused to: process metadata associated with the each first pixel and the second pixel; determine a first camera location associated with the image and a second camera location associated with the three-dimensional map based on the metadata; and determine an offset between the first camera location and the second camera location, wherein the confidence value is further based on the offset. ### Claim 5 (depends on 4) An apparatus of claim 4, wherein the apparatus is further caused to: determine at least one first ray extending from the first camera location to at least one pixel location of one of the first pixels and the second pixel; determine at least one second ray extending from the second camera location to the at least one pixel location; and determine at least one angle formed between the at least one first ray and the at least one second ray, wherein the confidence value is determined based on the at least one angle. ### Claim 6 (depends on 5) An apparatus of claim 5, wherein the apparatus is further caused to: determine the transparency level for the rendering of the each first pixel onto the three-dimensional map based on an accumulated confidence value added from a plurality of confidence values determined per pixel for a plurality of images including the image. ### Claim 7 (depends on 6) An apparatus of claim 6, wherein the transparency level decreases with an increase in the number of the images. ### Claim 8 (depends on 4) An apparatus of claim 4, wherein the apparatus is further caused to: initiate an estimation of at least one depth offset value for the image with reference to the second camera location; and initiate an iterative updating of a depth offset value buffer with at least one estimated depth offset value. ### Claim 9 (depends on 8) An apparatus of claim 8, wherein the rendering of the first pixels onto the three-dimensional map is based on the at least one depth offset value. ### Claim 10 (depends on 8) An apparatus of claim 8, wherein the rendering of the first pixels onto the three-dimensional map is based on at least one user interaction and the at least one depth offset value. ### Claim 11 (depends on 8) An apparatus of claim 8, wherein the apparatus is further caused to: generate at least one new candidate depth offset value for the image with reference to the second camera location; initiate a first rendering of at least one test image onto the three-dimensional map based on the at least one depth offset value; initiate a second rendering of the at least one test image onto the three-dimensional map based on the at least one new candidate depth offset value; and initiate an analysis of the first rendering and the second rendering to determine one or more rendering characteristics associated with the first rendering and the second rendering. ### Claim 12 (depends on 8) An apparatus of claim 8, wherein the apparatus is further caused to: initiate an updating of the depth offset value buffer with the at least one depth offset value or the at least one new candidate depth offset value based on the analysis. ### Claim 13 (depends on 1) An apparatus of claim 1, wherein the apparatus is further caused to: determine that the image includes a plurality of overlapping images; and initiate a sorting, a blending, or a combination thereof of the plurality of overlapping images based on the confidence value. ### Claim 14 (depends on 1) An apparatus of claim 1, wherein the image includes at least one video, the apparatus is further caused to: determine at least one three-dimensional motion track for the at least one video, wherein the confidence value, the rendering of the first pixels onto the three-dimensional map is based on the at least one three-dimensional motion track. ### Claim 15 A method comprising: determining, by an apparatus, each of a plurality of first pixels of an image that geometrically corresponds to a second pixel of a rendered three-dimensional map, wherein the three-dimensional map is a three-dimensional geographic map; processing, by the apparatus, the each first pixel and the second pixel to determine a confidence value that increases as a transparency level for the rendering of the each first pixel onto the three-dimensional map decreases, wherein the confidence value indicates an estimated level of geometric distortion of the each first pixel as projected onto the three-dimensional map resulting from a difference between an image capturing perspective of the image with respect to the each first pixel and a virtual viewing perspective of the three-dimensional map; initiating, by the apparatus, a rendering of the first pixels onto the three-dimensional map from the virtual viewing perspective based on the confidence value; determining, by the apparatus, at least one user interaction with the three-dimensional map; and initiating, by the apparatus, a dynamic updating of the confidence value, the rendering of the first pixels onto the three-dimensional map, or a combination thereof based on the at least one user interaction, wherein the confidence value of a respective first pixel decreases as the virtual viewing perspective moves further away from the image capturing perspective. ### Claim 16 (depends on 15) A method of claim 15, wherein the at least one user interaction results in a change in (a) the virtual viewing perspective for the three-dimensional map, (b) at least one rendering characteristic for at least one of the first pixels, the image, the second pixel, and the three-dimensional map; or (c) a combination thereof, and wherein the at least one rendering characteristic includes a fade parameter, a toggle parameter, a size parameter, or a combination thereof. ### Claim 17 (depends on 15) A method of claim 15, further comprising: initiating a comparison of the confidence value against at least one threshold confidence criterion to determine whether to cause the rendering of the first pixels onto the three-dimensional map. ### Claim 18 (depends on 15) A method of claim 15, further comprising: processing metadata associated with the each first pixel and the second pixel; determining a first camera location associated with the image and a second camera location associated with the three-dimensional map based on the metadata; and determining an offset between the first camera location and the second camera location, wherein the confidence value is further based on the offset.