High speed automated capture of 3D models of packaged items

A computer-implemented method, comprising:

• detecting a first physical object has reached a designated location, based on sensor data collected from one or more sensor devices;
• generating a three-dimensional (3D) mesh based on depth values calculated from a first subset of a plurality of near-infrared digital images of the first physical object, wherein the plurality of near-infrared digital images are synchronously captured by a plurality of stereo near-infrared camera devices while a predefined near-infrared pattern is projected onto the first physical object;
• generating a 3D model of the first physical object by applying a texture to the 3D mesh, wherein applying the texture includes mapping points from each of a first subset of a plurality of color digital images of the first physical object onto the 3D mesh, wherein the plurality of color digital images are synchronously captured using a plurality of color camera devices that capture light in a visible light spectrum;
• performing, by operation of one or more computer processors, (i) a mesh verification of the 3D model using a remainder of the plurality of near-infrared digital images and (ii) a texture verification of the 3D model using a remainder of the plurality of color digital images;
• upon success of both of the mesh verification and the texture verification, determining that the 3D model is properly generated, wherein the computer-implemented method is performed by a program configured to, upon failure of at least one of the mesh verification or the texture verification, discard the 3D model and reinitiate the computer-implemented method on the first physical object to regenerate the 3D model;
• regenerating the 3D model upon determining that based on the 3D model, a robotic arm controller has performed a predefined number of failed picking operations within a designated time window on one or more physical objects identified as being of a same type as the first physical object.

A non-transitory computer-readable medium containing computer program code executable to perform an operation comprising:

• detecting a first physical object has reached a designated location;
• upon identifying the first physical object as a predefined training object, retrieving object profile information specifying a physical size, shape, and appearance of the predefined training object;
• capturing near-infrared digital images of the first physical object using a plurality of stereo near-infrared camera devices, in a synchronous manner and at least substantially at a first moment in time;
• capturing color digital images of the first physical object using a plurality of color camera devices, in a synchronous manner and at least substantially at a second moment in time, wherein the plurality of color camera devices capture light in a visible light spectrum;
• determining relative three-dimensional (3D) positional information for each of the plurality of stereo near-infrared camera devices and the plurality of color camera devices, by construing the near-infrared digital images and the color digital images in view of the retrieved object profile information for the predefined training object;
• storing the relative 3D positional information as calibration information for the plurality of stereo near-infrared camera devices and the plurality of color camera devices;
• generating a 3D model for a second physical object, by generating a 3D mesh and applying a texture to the 3D mesh, based on the determined relative 3D positional information for each of a first subset of the plurality of stereo near-infrared camera devices and a first subset of the plurality of color camera devices, and using images of the second physical object, the images including (i) a first subset of a plurality of near-infrared digital images synchronously captured by the plurality of stereo near-infrared camera devices and (ii) a first subset of a plurality of color digital images synchronously captured by the plurality of color camera devices;
• performing, by one or more computer processors when executing the computer program code, (i) a mesh verification of the 3D model using a remainder of the plurality of near-infrared digital images and (ii) a texture verification of the 3D model using a remainder of the plurality of color digital images;
• upon success of both of the mesh verification and the texture verification, determining that the 3D model is properly generated, wherein the computer program code is configured to, upon failure of at least one of the mesh verification or the texture verification, discard the 3D model and reinitiate the operation on the second physical object to regenerate the 3D model;
• and
• outputting the 3D model, wherein the 3D model is regenerated upon determining that based on the 3D model, a robotic arm controller has performed a predefined number of failed picking operations within a designated time window on one or more physical objects identified as being of a same type as the second physical object.