Autonomous robotic system for collecting products

An actuator for collecting products from a real store by means of a mobile robot, said actuator being arranged on a mobile base comprising:

• at least one drive unit configured to drive and direct the mobile robot, said actuator being configured to manipulate and move real products from shelves of the real store and towards at least one temporary storage region arranged on the mobile base, configured to receive real products from the at least one actuator;
• a navigation system in communication with the at least one drive unit, through the at least one communications unit and the at least one processing unit, wherein the navigation system accesses planimetry information of the real store;
• a product recognition system in communication with the at least one vision sensor assembly, through the at least one communications unit and the at least one processing unit, wherein the product recognition system accesses the vision information;
• and
• a multi-objective planning system in communication with the navigation system;
• wherein, when a mobile robot, which travels through the real store following at least one optimal route, identifies a real product matching a product to be collected from at least one selection of products to be collected, a product recognition system of the mobile robot is configured to drive the at least one drive unit of the at least one mobile robot to position itself in the vicinity of the shelf where said matching real product is located and to drive the actuator of the at least one mobile robot based on the location and distance information of the matching real product to collect the matching real product from the shelf and place it in the at least one temporary storage region, continuing with the at least one optimal route;
• wherein driving the actuator to collect a matching real product based on the location and distance information of said matching real product comprises executing a set of movements of the actuator according to a position of the matching real product and a distance between the actuator and the matching real product;
• wherein the actuator has at least two degrees of freedom comprising the following movements:
• in height, along a first vertical axis covering a height of the shelves in the real store;
• and
• in depth, along a first horizontal axis, covering the distance between the mobile robot and the products to be collected;
• wherein the actuator further comprises a third degree of freedom comprising rotational movement around a second vertical axis, covering at least two positions, one position for collecting products from shelves of the real store and one position for temporary storage of products in the at least one temporary storage region;
• and
• wherein the actuator comprises an end effector comprising at least two suction cups, wherein one suction cup is smaller than the other, wherein said end effector, in turn, has at least two degrees of freedom comprising the following movements:
• rotation about a second horizontal axis, rotating the end effector to correctly position it in relation to the matching item or product to be collected;
• and
• tilting, tilting the end effector once the matching product has been collected, compensating for the strength to weight ratio of said product.

A method of operating an autonomous robotic system for collecting products remotely from a virtual store, wherein the autonomous robotic system comprises:

• at least one real store comprising shelves with real products;
• at least one mobile robot which is formed by a body comprising:
• at least one processing unit;
• at least one communications unit;
• at least one vision sensor assembly configured to obtain image information and distance information, called vision information;
• a mobile base comprising at least one drive unit configured to drive and direct the mobile robot;
• at least one actuator arranged on the mobile base configured to manipulate and move real products from the shelves of the real store;
• and
• at least one temporary storage region arranged on the movable base configured to receive real products from the at least one actuator;
• a navigation system in communication with the at least one drive unit, through the at least one communications unit and the at least one processing unit, wherein the navigation system accesses planimetry information of the real store;
• a product recognition system in communication with the at least one vision sensor assembly, through the at least one communications unit and the at least one processing unit, wherein the product recognition system accesses the vision information;
• a multi-objective planning system in communication with the navigation system;
• a computer system in communication with the multi-objective planning system and the product recognition system;
• and
• at least one user device in communication with the computer system;
• the method comprising:
• a) showing, by means of the at least one user device, a virtual store to a user through a user interface, wherein said virtual store is a graphic representation of the real store and comprises shelves with a graphic representation of the real products called virtual products;
• b) receiving, through said user interface, at least one selection of virtual products by the user, communicating said at least one selection of virtual products to the computer system, wherein the computer system communicates said at least one selection of virtual products to the multi-objective planning and product recognition system;
• c) calculating, through the multi-objective planning system and based on the planimetry information of the real store, one or more optimal routes for the collection of real products matching the at least one selection of virtual products communicating said one or more optimal routes to the navigation system;
• d) driving, through the navigation system the at least one drive unit of at least one mobile robot to travel the real store following at least one received optimal route;
• e) identifying, by means of the recognition system comprising identification algorithms, real products that match the virtual products of the at least one selection of the virtual products received, comprising:
• e.1) obtaining, by means of the at least one vision sensor assembly, vision information of the real store shelves,
• e.2) recognizing objects shapes corresponding to an exterior shape based on the visual appearance of real products in said vision information, and
• e.3) reading logos or texts of the real products in said vision information, identifying the matching real products and obtaining location and distance information of the identified matching real products;
• f) collecting the identified matching real products, wherein the product recognition system drives the at least one drive unit of the at least one mobile robot to position itself in the vicinity of the shelf where an identified matching real product is located and drives the at least one actuator of the at least one mobile robot based on location and distance information of the identified matching real product, in order to collect it from the shelf of the real store;
• g) placing the collected real products in the at least one temporary storage region located in the body of the at least one mobile robot;
• and
• h) to continue with at least one optimal route.

A mobile robot for collecting products remotely formed by a body that includes:

• at least one processing unit;
• at least one communications unit;
• at least one vision sensor assembly configured to obtain image information and distance information, called vision information;
• a mobile base comprising at least one drive unit configured to drive and direct the mobile robot;
• at least one actuator arranged on the mobile base configured to manipulate and move real products from the shelves of the real store;
• and
• at least one temporary storage region arranged on the mobile base configured to receive real products from the at least one actuator;
• wherein the robot also includes:
• a navigation system in communication with the at least one drive unit, through the at least one communications unit and the at least one processing unit, wherein the navigation system accesses planimetry information of a real store;
• a product recognition system in communication with the at least one vision sensor assembly, through the at least one communications unit and the at least one processing unit, wherein the product recognition system accesses the vision information;
• a multi-objective planning system in communication with the navigation system;
• and
• a computer system in communication with the multi-objective planning system and the product recognition system;
• wherein the computer system is configured to receive at least one products selection and to communicate it to the multi-objective planning system and the product recognition system, wherein based on the planimetry information of the real store the multi-objective planning system is configured to calculate an optimal route for the collection of matching real products with the at least one selection of products, and wherein said optimal route is communicated to the navigation system;
• wherein the navigation system is configured to receive said optimal route and to drive the at least one drive unit of the mobile robot to travel the real store following the received optimal route;
• wherein the product recognition system comprises identification algorithms configured to identify each real product on the shelves of the real store, wherein the at least one vision sensor assembly in communication with said product recognition system is arranged to obtain vision information from the real store shelves, and wherein the identification algorithms are configured to recognize objects shapes corresponding to an exterior shape of the real products in said vision information, and to read logos or texts of the real products in said vision information identifying the real products and obtaining location and distance information of the identified real products;
• and
• wherein, when the mobile robot, which travels through the real store following the optimal route, identifies a real product matching one of the products of the at least one received selection of products, the product recognition system is configured to activate the at least one drive unit of the mobile robot to position itself in the vicinity of the shelf where said matching real product is located and to operate the at least one actuator of the mobile robot based on the location and distance information of the matching real product, to collect the matching real product from the shelf and placing it in the at least one temporary storage region continuing with the optimal route.