Using ocr to detect shelf labels and associating them with 3d data to use them as landmarks

A method for autonomous mobile robot localization in industrial environments, the method comprising:

• during autonomous operation of a mobile robot through an environment comprising textual labels on environmental features:
• acquiring image data and spatial data of said environment using onboard sensors;
• extracting textual content from said image data, wherein said textual content represents existing labels, signs, or markings not installed specifically for robotic navigation;
• determining spatial positions associated with said extracted textual content;
• maintaining a database of semantic landmarks, each semantic landmark associating textual content with a spatial position;
• upon detecting textual content:
• matching said detected textual content against semantic landmarks in said database,
• when a match is found, computing robot pose using known position of matched landmark and observed relative position,
• when no match is found, adding a new semantic landmark to said database;
• providing computed robot pose to navigation control;
• and
• performing said acquiring, extracting, determining, maintaining, matching, and providing continuously and autonomously during robot operation,
• wherein said method enables localization using pre-existing environmental features without requiring human intervention to specify robot starting position or install localization infrastructure.

A method for autonomous robot recovery from position loss, the method comprising:

• detecting that a mobile robot has lost knowledge of its position in an environment due to at least one of:
• power cycle, manual relocation, tracking failure, or extended loss of sensor data;
• upon detecting position loss:
• capturing images of said environment using onboard camera,
• extracting textual content from environmental features comprising labels, signs, or markings not installed specifically for robot navigation,
• determining spatial positions associated with detected textual content using depth sensing,
• comparing detected textual content against landmark database storing known text-position associations;
• upon finding matching landmarks:
• computing robot pose from geometric relationship between robot and observed landmarks with known positions,
• validating computed pose through consistency checks,
• reinitializing robot localization system with computed pose;
• resuming autonomous operation without requiring:
• human intervention to specify position,
• manual movement to known location,
• system restart or recalibration,
• wherein autonomous recovery from position loss events enables resilient operation and reduces operational downtime.

A multi-robot localization system comprising:

• a plurality of autonomous mobile robots each comprising sensors for image acquisition and spatial sensing;
• a shared semantic landmark repository accessible by all robots in said plurality, said repository storing text-annotated landmarks observed by any robot;
• a collaborative mapping module configured to:
• receive landmark observations from multiple robots,
• aggregate observations of identical textual content from different viewpoints,
• improve landmark position estimates through multi-robot statistical fusion,
• detect and resolve inconsistencies when different robots report conflicting positions for same textual identifier,
• identify newly added or relocated labels based on change patterns across robot observations,
• distribute updated landmark data to said plurality of robots;
• and
• a bootstrapping module configured to enable a newly deployed robot to:
• download pre-existing landmark map built by other robots,
• localize immediately upon startup using said pre-existing landmark map without manual initialization,
• contribute additional landmark observations to refine map accuracy,
• wherein collaborative observation from multiple robots improves landmark position accuracy beyond what any single robot could achieve, and wherein new robots benefit from prior mapping effort without duplicating exploration.

A non-transitory computer-readable storage medium storing instructions that, when executed by processing hardware of an autonomous mobile robot system, cause said system to:

• process sensor data to identify textual information in an environment;
• associate identified textual information with spatial positions;
• build and maintain a semantic map comprising text-position associations;
• determine robot pose by matching observed textual information against said semantic map;
• adapt localization behavior based on environmental conditions including at least one of:
• degrading position accuracy estimates when observing distant or degraded labels,
• increasing confidence in position estimates when multiple independent landmarks are simultaneously observed,
• triggering re-localization when accumulated pose uncertainty exceeds a threshold,
• adjusting processing parameters based on ambient lighting conditions;
• and
• interface with task management systems by:
• receiving task specifications referencing textual identifiers,
• translating textual identifiers to spatial coordinates using said semantic map,
• executing navigation to spatial coordinates corresponding to specified textual identifiers,
• wherein said instructions enable localization and task execution using human-readable environmental labels without requiring dedicated localization infrastructure.