# US 2021/0241523 A1 · Electronic device for providing visual localization based on outdoor three-dimension map information and operating method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241523 A1 - Document ID: US20210241523A1 - Status: Published application - Kind: A1 - Application number: 17/166,489 - Filed: 2021-02-03 - Published: 2021-08-05 - Earliest claimed: 2020-02-04 - Assignees: Naver Corporation; Naver Labs Corporation - Applicants: Naver Corporation; Naver Labs Corporation - Inventors: Deokhwa KIM; Donghwan LEE - Web page: https://patentgrep.com/patent/US20210241523A1 ## CPC - G06T 17/05: Geographic models - G06F 18/29: {Graphical models, e.g. Bayesian networks} - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06V 10/764: using classification, e.g. of video objects - G06V 10/84: using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks - G06V 20/182: {Network patterns, e.g. roads or rivers} - G06V 20/38: {Outdoor scenes} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Electronic devices and/or operating methods of the electronic device to provide visual localization based on outdoor three-dimensional (3D) map information may be provided. Such electronic devices may be configured to acquire two-dimensional (2D) image information about an outdoor environment, generate 3D map information about the outdoor environment based on the 2D image information, and determine a position of a point in the 3D map information corresponding to a query image. ## Claims ### Claim 1 An operating method of an electronic device, the method comprising: acquiring, using a processor included in the electronic device, two-dimensional (2D) image information about an outdoor environment; generating, using the processor, three-dimensional (3D) map information about the outdoor environment based on the 2D image information; and determining, using the processor, a position of a point in the 3D map information corresponding to a query image. ### Claim 2 (depends on 1) The method of claim 1, wherein the generating 3D map information comprises: segmenting the outdoor environment into a plurality of areas; and generating the 3D map information for each of the areas. ### Claim 3 (depends on 2) The method of claim 2, wherein the generating the 3D map information for each of the areas comprises: classifying at least one node into a boundary node or a non-boundary node in each of the areas; applying a confidence to each of the boundary node and the non-boundary node, the confidence applied to the boundary node being different from the confidence applied to the non-boundary node; and generating the 3D map information based on the node. ### Claim 4 (depends on 3) The method of claim 3, wherein the generating the 3D map information based on the node comprises generating the 3D map information based on pose information of each node. ### Claim 5 (depends on 4) The method of claim 4, wherein the generating the 3D map information based on the pose information comprises: detecting an error of at least one edge connected to the node based on the confidence; estimating the pose information of the node based on the error; estimating 3D point information of the node based on the error; and generating the 3D map information based on the pose information and the 3D point information. ### Claim 6 (depends on 5) The method of claim 5, wherein the generating the 3D map information based on the pose information further comprises verifying first pose information of the node based on the 2D image information. ### Claim 7 (depends on 6) The method of claim 6, wherein the detecting an error of the at least one edge comprises: detecting the error of the at least one edge connected to the node based on the first pose information; and summing the error using the confidence. ### Claim 8 (depends on 7) The method of claim 7, wherein the estimating the pose information of the node based on the error comprises estimating second pose information of which the error decreases, and the generating the 3D map information based on the pose information comprises generating the 3D map information based on the second pose information. ### Claim 9 An electronic device comprising: a memory; and a processor configured to connect to the memory and to execute at least one instruction stored in the memory, the processor further configured to, acquire two-dimensional (2D) image information about an outdoor environment, generate three-dimensional (3D) map information about the outdoor environment based on the 2D image information, and determine a position of a point in the 3D map information corresponding to a query image. ### Claim 10 (depends on 9) The electronic device of claim 9, wherein the processor is further configured to segment the outdoor environment into a plurality of areas, and to generate the 3D map information for each of the areas. ### Claim 11 (depends on 10) The electronic device of claim 10, wherein the processor is further configured to classify at least one node into a boundary node or a non-boundary node in each of the areas, apply a confidence to each of the boundary node and the non-boundary node, the confidence applied to the boundary node being different from the confidence applied to the non-boundary node, and generate the 3D map information based on the node. ### Claim 12 (depends on 11) The electronic device of claim 11, wherein the processor is further configured to generate the 3D map information using a structure-from-motion (SfM) algorithm. ### Claim 13 (depends on 11) The electronic device of claim 11, wherein the processor is further configured to generate the 3D map information based on pose information of each node. ### Claim 14 (depends on 13) The electronic device of claim 13, wherein the processor is further configured to detect an error of at least one edge connected to the node based on the confidence, estimate the pose information and 3D point information of the node based on the error, and generate the 3D map information based on the pose information and the 3D point information. ### Claim 15 (depends on 14) The electronic device of claim 14, wherein the processor is further configured to verify first pose information of the node based on the 2D image information, detect the error of the at least one edge connected to the node based on the first pose information, sum the error using the confidence, estimate second pose information of which the error decreases, and generate the 3D map information based on the second pose information. ### Claim 16 (depends on 10) The electronic device of claim 10, wherein the 2D image information comprises a plurality of street view images for each of the areas. ### Claim 17 A non-transitory computer-readable record medium storing instructions that, when executed by a processor, cause a computer to implement a visual localization providing method, the visual localization providing method comprising: acquiring, using the processor, two-dimensional (2D) image information about an outdoor environment; generating, using the processor, three-dimensional (3D) map information about the outdoor environment based on the 2D image information; and determining, using the processor, a position of a point in the 3D map information corresponding to a query image.