Autonomous robotic system for automatically monitoring the state of shelves in shops

A method for automatically monitoring the state of shelves in stores through a robotic system that includes,

• a mobile robot having a robot body, the robot body comprising:
• a mobile base comprising a drive system connected to a movement and direction means;
• an upper structure arranged to accommodate sensors, at least one processing unit and a communications means, the sensors comprising:
• at least one laser sensor;
• at least one distance or depth or proximity sensor;
• and
• at least one image sensor;
• wherein at least one processing unit comprises at least one storage means and at least one processor;
• a navigation system communicating with the at least one laser sensor, at least one image sensor, at least one distance or proximity sensor and at least one processor, wherein:
• the laser sensor is configured to measure the robot's environment by capturing information of the store planimetry, wherein said information on the planimetry is processed by the navigation system which constructs a map of the store's spatial configuration and operates the movement and direction means by guiding the robot to navigate the store aisles;
• and
• the image sensor and the proximity sensor are configured to display and measure the robot's environment, and capture display and distance information, wherein the display and distance information is processed by the navigation system which generates dynamic navigation routes that adapt to obstacles present in the robot's environment;
• a recognition system communicating with the at least one image sensor, at least one distance or proximity sensor, at least one processing unit and the communications means, wherein:
• the recognition system comprises deep-learning detection and recognition algorithms, the deep-learning detection and recognition algorithms are configured to detect and recognize relevant information present in different areas of the store shelves
• the image sensor is-configured to capture images of the different areas of the store's shelves, and
• the distance or proximity sensor is configured to determine the distance between the robot and a main plane of the shelf and/or a main plane of each relevant piece of information present on the shelves,
• wherein the relevant information is contained in the captured images and stored in the storage means;
• and
• the detection and recognition algorithms are trained with examples of typical store scenarios comprising specific architectures optimized for the detection and recognition of relevant information present in the different areas of the shelves, the relevant information comprising:
• letters, numbers and characters commonly used in stores and products;
• and
• a multi-target planning system in communication with at least one processing unit and with the navigation system, wherein:
• the multi-target planning system comprises a dynamic route planning routine that evaluates coverage of all areas of the store's shelves to be monitored, thereby ensuring full coverage of the store;
• and
• the multi-target planning system communicates with the navigation system to guide the robot according to the dynamic route planning in real time
• wherein the method comprises:
• starting a travel of a mobile robot in a store for monitoring the state of the shelves;
• autonomously navigating by the mobile robot the aisles of the store, said navigation being configured to ensure a desired coverage of the store using the multi-target planning and navigation systems wherein the multi-target planning and navigation systems incorporate obstacle-avoiding capability and route replanning;
• capturing images, by the robot, while navigating the store, to get relevant information on the product shelves requiring supervision or monitoring by using the robot's capture sensor(s) and processing unit(s);
• detecting and recognizing the relevant information contained in the captured images using deep-learning detection and recognition algorithms;
• processing the detected and recognized relevant information and communicating the processed information through the robot's communications means to an end user using generated reports and/or alarms.

A method or automatically monitoring the state of shelvesin stores through a robotic system, the method comprising:

• starting a travel of a mobile robot in a store to monitor the state of the shelves;
• autonomously navigating by the mobile robot the aisles of the store, said navigation being configured to ensure a desired coverage of the store using multi-target planning and navigation systems to direct the robot in real time, while incorporating obstacle-avoiding capability and route replanning;
• capturing images, by the robot, while navigating the store, to get relevant information on the product shelves that require supervision or monitoring using the robot's capture sensors and the processing unit(s);
• detecting and recognizing the relevant information contained in the captured images using deep-learning detection and recognition algorithms;
• processing the detected and recognized relevant information and communicating the processed information through the robot's communications means to an end user using generated reports and/or alarms.