Crane-mounted system for automated object detection and identification

A method for automatically identifying objects from image data generated by an image sensor mounted on a crane, the method comprising:

• receiving a first set of 3D image data, generated by the image sensor, representing sensed data within a predetermined area, wherein the image sensor is mounted on a beam that is coupled to a track and movable along the track;
• receiving position data identifying a position of the crane associated with the first set of 3D image data;
• determining a target object to be identified within the first set of 3D image data;
• determining a point cloud of data points representing the target object based on a 3D model for the target object or a primitive 3D geometric shape for the target object;
• applying a filtering algorithm to the set of 3D image data, so as to generate a second set of 3D image data that contains less data points than the first set of 3D image data;
• identifying one or more object positions based on a comparison of the second set of 3D image data to the point cloud of data points representing the target object;
• generating a set of position data based on the identified one or more object positions;
• determining, for each position data within the generated set of position data, whether or not the position data represents the location of a pickable object;
• and
• transmitting, to a device configured to control one or more components of the crane, position data determined to represent the location of a pickable object.

A non-transitory computer-readable storage medium comprising instructions stored therein, which when executed by one or more processors, cause the one or more processors to perform operations comprising:

• receiving a first set of 3D image data representing sensed data within a predetermined area, the first set of 3D image being generated by an image sensor mounted on a crane, wherein the image sensor is mounted on a beam that is coupled to a track and movable along the track;
• receiving position data identifying a position of the crane associated with the first set of 3D image data;
• identifying a target object to be identified within the first set of 3D image data;
• determining a point cloud of data points representing the target object based on a 3D model for the target object or a primitive 3D geometric shape for the target object;
• applying a filtering algorithm to the set of 3D image data, so as to generate a second set of 3D image data;
• identifying one or more object positions based on a comparison of the second set of 3D image data to the point cloud of data points representing the target object;
• generating a set of position data based on the identified one or more object positions;
• and
• transmitting, to a device configured to control one or more components of the crane, position data, of the set of position data, representing an identified object position.

An object picking system comprising:

• a crane comprising a gripper capable of picking up an object;
• a control module having an actuator configured to control movement of the gripper;
• an image sensor mounted on the crane;
• and
• a computing system comprising at least one processor configured to:
• receive a first set of 3D image data, generated by the image sensor, representing sensed data within a predetermined area;
• receive position data identifying a position of the crane associated with the first set of 3D image data;
• determine a target object to be identified within the first set of 3D image data;
• determine a point cloud of data points representing the target object based on a 3D model for the target object or a primitive 3D geometric shape for the target object;
• apply a filtering algorithm to the set of 3D image data, so as to generate a second set of 3D image data that contains less data points than the first set of 3D image data;
• identify one or more object locations based on a comparison of the second set of 3D image data to the point cloud of data points representing the target object;
• generate a set of position data based on the identified one or more object locations;
• determine, for each position data within the generated set of position data, whether or not the position data represents the location of a pickable object;
• and
• transmit, to the control module, position data determined to represent the location of a pickable object,
• wherein the at least one processor in applying the filtering algorithm is configured to:
• identify a portion of the first set of 3D image data representing an object at a location that is inaccessible to the gripper;
• and
• filter out the portion of the first set of 3D image data in response to a determination that the portion represents the object that is inaccessible to the gripper.

An object picking system comprising:

• an overhead crane comprising a gripper capable of picking up objects;
• a control module having an actuator configured to control movement of the gripper;
• an image sensor mounted on the overhead crane;
• and
• a computing system comprising at least one processor configured to:
• receive a first set of 3D image data, generated by the image sensor, representing sensed data within an area;
• receive position data identifying a position of the crane associated with the first set of 3D image data;
• receive a job identifier identifying a job associated with moving at least one target object within the area;
• based on the job identifier, select a 3D model for the at least one target object for use in performance of the job;
• retrieve from memory the selected 3D model for the at least one target object;
• determine a point cloud of data points representing the at least one target object based on the 3D model;
• identify a portion of the 3D image data associated with an area that is inaccessible to the gripper;
• filter the first set of 3D image data to remove the identified portion, so as to generate a second set of 3D image data that contains less data points than the first set of 3D image data;
• compare the point cloud of data points representing the at least one target object to data points of the second set of 3D image data to identify the at least one target object in the second set of 3D image data;
• based on the position data and identification of the at least one target object in the second set of 3D image data, determine at least one object location for the at least one target object;
• and
• transmit, to the control module, position data indicating the at least one object location for the at least one target object,
• wherein the control module is configured to control the gripper with the actuator to cause the gripper to pick up and move the at least one target object based on the position data indicating the object location for the at least one target object.

An object picking method comprising:

• controlling movement of a gripper of a crane with an actuator, the gripper capable of picking up an object;
• receiving, with at least one processor, a first set of 3D image data generated by an image sensor mounted on the crane, the first set of 3D image data representing sensed data within a predetermined area;
• receiving, with the at least one processor, position data identifying a position of the crane associated with the first set of 3D image data;
• determining, with the at least one processor, a target object to be identified within the first set of 3D image data;
• determining, with the at least one processor, a point cloud of data points representing the target object based on a 3D model for the target object or a primitive 3D geometric shape for the target object;
• applying, with the at least one processor, a filtering algorithm to the set of 3D image data, so as to generate a second set of 3D image data that contains less data points than the first set of 3D image data;
• identifying, with the at least one processor, one or more object locations based on a comparison of the second set of 3D image data to the point cloud of data points representing the target object;
• generating, with the at least one processor, a set of position data based on the identified one or more object locations;
• determining, with the at least one processor for each position data within the generated set of position data, whether or not the position data represents the location of a pickable object;
• and
• transmitting, with the at least one processor to the control module, position data determined to represent the location of a pickable object,
• wherein the applying comprises:
• identifying a portion of the first set of 3D image data representing an object at a location that is inaccessible to the gripper;
• and
• filtering out the portion of the first set of 3D image data in response to a determination that the portion represents the object that is inaccessible to the gripper.

An object picking method comprising:

• controlling movement of a gripper of an overhead crane with an actuator, the gripper capable of picking up objects;
• receiving, with at least one processor, a first set of 3D image data generated by an image sensor mounted on the overhead crane, the first set of 3D image data representing sensed data within an area;
• receiving, with the at least one processor, position data identifying a position of the crane associated with the first set of 3D image data;
• receiving, with the at least one processor, a job identifier identifying a job associated with moving at least one target object within the area;
• based on the job identifier, selecting with the at least one processor a 3D model for the at least one target object for use in performance of the job;
• retrieving, with the at least one processor from memory, the selected 3D model for the at least one target object;
• determining, with the at least one processor, a point cloud of data points representing the at least one target object based on the 3D model;
• identifying, with the at least one processor, a portion of the 3D image data associated with an area that is inaccessible to the gripper;
• filtering, with the at least one processor, the first set of 3D image data to remove the identified portion, so as to generate a second set of 3D image data that contains less data points than the first set of 3D image data;
• comparing, with the at least one processor, the point cloud of data points representing the at least one target object to data points of the second set of 3D image data to identify the at least one target object in the second set of 3D image data;
• based on the position data and identification of the at least one target object in the second set of 3D image data, determining with the at least one processor at least one object location for the at least one target object;
• transmitting, with the at least one processor to the control module, position data indicating the at least one object location for the at least one target object;
• and
• controlling the gripper with the actuator to cause the gripper to pick up and move the at least one target object based on the position data indicating the object location for the at least one target object.