# US 12,008,819 B2 · Navigation system with mono-camera based traffic sign tracking and positioning mechanism and method of operation thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,819 B2 - Document ID: US12008819B2 - Status: Granted - Kind: B2 - Application number: 17/394,112 - Filed: 2021-08-04 - Published: 2023-02-09 - Granted: 2024-06-11 - Earliest claimed: 2021-08-04 - Assignees: Telenav, Inc. - Applicants: TeleNav, Inc. - Inventors: Hui Fang; Congmin Bai; Lina Hu - Web page: https://patentgrep.com/patent/US12008819B2 ## CPC - G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera} - G06V 20/582: {of traffic signs} - G01C 21/3623: {using a camera or code reader, e.g. for optical or magnetic codes} - G01C 21/3632: {Guidance using simplified or iconic instructions, e.g. using arrows (G01C21/365 takes precedence)} - G01C 21/3673: {Labelling using text of road map data items, e.g. road names, POI names} - G06V 10/82: using neural networks - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions - G08G 1/09623: {Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle (G08G1/0967 takes precedence)} - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract A method of operation of a navigation system comprising: receiving multiple frames, including a first frame and a second frame, of images; detecting a traffic sign from the images between the first frame and the second frame based on a sign recognition model; extracting a first image from the first frame and a second image from the second frame; matching the traffic sign is the same in the first image and the second image based on a similarity model; generating a sign location of the traffic sign with an inertial measurement unit reading based on the first image and the second image; and generating a global coordinate for the sign location for displaying on a navigation map. ## Claims ### Claim 1 A method of operation of a navigation system comprising: receiving multiple frames, including a first frame and a second frame, of images; detecting a traffic sign from the images between the first frame and the second frame based on a sign recognition model; extracting a first image from the first frame and a second image from the second frame; matching the traffic sign is the same in the first image and the second image based on a similarity model; generating a sign location of the traffic sign with an inertial measurement unit reading based on the first image and the second image; generating a vehicle relative position associated with a mono-camera for receiving the multiple frames; generating a camera compensation for the mono-camera; and generating a global coordinate for the sign location, including triangulating the sign location based on a vehicle relative position, a camera compensation, and a matching, for displaying on a navigation map. ### Claim 2 (depends on 1) The method as claimed in claim 1 wherein generating the sign location include generating a three-dimensional coordinate with the inertial measurement unit reading. ### Claim 3 (depends on 1) The method as claimed in claim 1 wherein receiving multiple frames of images includes receiving a video from a mono-camera. ### Claim 4 (depends on 1) The method as claimed in claim 1 wherein generating the global coordinate for the sign location including generating the global coordinate based on the inertial measurement unit reading and a global navigation satellite system reading. ### Claim 5 (depends on 1) The method as claimed in claim 1 further comprising: tracking the traffic sign from the images in the first frame and the second frame; and wherein: extracting the first image and the second image includes extracting the traffic sign being tracked in the first image and the second image. ### Claim 6 (depends on 1) The method as claimed in claim 1 further comprising storing the global coordinate for the sign location with the navigation map. ### Claim 7 A navigation system comprising: a communication unit configured to receive multiple frames, including a first frame and a second frame, of images; and a control unit, coupled to the communication unit, configured to: detect a traffic sign from the images between the first frame and the second frame based on a sign recognition model; extract a first image from the first frame and a second image from the second frame; match the traffic sign is the same in the first image and the second image based on a similarity model; generate a sign location of the traffic sign with an inertial measurement unit reading based on the first image and the second image; generate a vehicle relative position associated with a mono-camera for receiving the multiple frames, generate camera compensation for the mono-camera; and generate a global coordinate for the sign location, including triangulating the sign location based on a vehicle relative position, a camera compensation, and a matching for displaying on a navigation map. ### Claim 8 (depends on 7) The system as claimed in claim 7 wherein the control unit is configured to generate a three-dimensional coordinate with the inertial measurement unit reading. ### Claim 9 (depends on 7) The system as claimed in claim 7 wherein the communication unit is configured to receive a video from a mono-camera. ### Claim 10 (depends on 7) The system as claimed in claim 7 wherein the control unit is configured to generate the global coordinate based on the inertial measurement unit reading and a global navigation satellite system reading. ### Claim 11 (depends on 7) The system as claimed in claim 7 wherein the control unit is configured to: track the traffic sign from the images in the first frame and the second frame; and extract the first image and the second image includes extracting the traffic sign being tracked in the first image and the second image. ### Claim 12 (depends on 7) The system as claimed in claim 7 wherein the control unit is configured to store the global coordinate for the sign location with the navigation map. ### Claim 13 A non-transitory computer readable medium including instructions executable by a control circuit for a navigation system, the instructions comprising: receiving multiple frames, including a first frame and a second frame, of images; detecting a traffic sign from the images between the first frame and the second frame based on a sign recognition model; extracting a first image from the first frame and a second image from the second frame; matching the traffic sign is the same in the first image and the second image based on a similarity model; generating a sign location of the traffic sign with an inertial measurement unit reading based on the first image and the second image; generating a vehicle relative position associated with a mono-camera for receiving the multiple frames: generating a camera compensation for the mono-camera; and generating a global coordinate for the sign location, including triangulating the sign location based on a vehicle relative position, a camera compensation and a matching for displaying on a navigation map. ### Claim 14 (depends on 13) The non-transitory computer readable medium as claimed in claim 13, wherein the instructions include generating a three-dimensional coordinate with the inertial measurement unit reading. ### Claim 15 (depends on 13) The non-transitory computer readable medium as claimed in claim 13, wherein receiving multiple frames of images includes receiving a video from a mono-camera. ### Claim 16 (depends on 13) The non-transitory computer readable medium as claimed in claim 13, wherein the instructions include generating the global coordinate for the sign location including generating the global coordinate based on the inertial measurement unit reading and a global navigation satellite system reading. ### Claim 17 (depends on 13) The non-transitory computer readable medium as claimed in claim 13, wherein the instructions include: tracking the traffic sign from the images in the first frame and the second frame; and extracting the first image and the second image includes extracting the traffic sign being tracked in the first image and the second image.