Object enrollment in a robotic cart coordination system

One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:

• determining one or more simulated capture paths for a virtual robot through a physical environment based on an environment navigation model of the physical environment determined based on sensor data collected via a handheld computing device, the environment navigation model including a plurality of semantic labels corresponding to elements of the physical environment;
• receiving as input a three-dimensional model of an object type;
• determining a plurality of simulated object parameter values for the object type, the simulated object parameter values identifying a plurality of possible variations of real-world conditions in which an object of the object type may appear, the simulated object parameter values including a parameter value selected from the group consisting of:
• object grouping, object rotation, and object stacking;
• determining simulated sensor data for a plurality of simulated instances of the object type based on the one or more simulated capture paths, the environment navigation model, the three-dimensional model, and the simulated object parameter values, the simulated sensor data including image data collected from a simulated camera at the virtual robot;
• training an environment-specific object recognition model based on the simulated sensor data, the environment-specific object recognition model being trained to recognize an object corresponding with the object type based on variations in appearance of objects corresponding to the object type that are specific to the physical environment;
• transmitting the environment-specific object recognition model to a plurality of physical robots;
• and
• instructing the plurality of physical robots to autonomously navigate the physical environment and recognize objects based on the environment-specific object recognition model and sensor data collected via the plurality of physical robots.

A system comprising a processor, a storage device, and memory, the memory including instructions to perform a method comprising:

• determining one or more simulated capture paths for a virtual robot through a physical environment based on an environment navigation model of the physical environment determined based on sensor data collected via a handheld computing device, the environment navigation model including a plurality of semantic labels corresponding to elements of the physical environment;
• receiving as input a three-dimensional model of an object type;
• determining a plurality of simulated object parameter values for the object type, the simulated object parameter values identifying a plurality of possible variations of real-world conditions in which an object of the object type may appear, the simulated object parameter values including a parameter value selected from the group consisting of:
• object grouping, object rotation, and object stacking;
• determining simulated sensor data for a plurality of simulated instances of the object type based on the one or more simulated capture paths, the environment navigation model, the three-dimensional model, and the simulated object parameter values, the simulated sensor data including image data collected from a simulated camera at the virtual robot;
• training an environment-specific object recognition model based on the simulated sensor data, the environment-specific object recognition model being trained to recognize an object corresponding with the object type based on variations in appearance of objects corresponding to the object type that are specific to the physical environment;
• transmitting the environment-specific object recognition model to a plurality of physical robots;
• and
• instructing the plurality of physical robots to autonomously navigate the physical environment and recognize objects based on the environment-specific object recognition model and sensor data collected via the plurality of physical robots.