Systems and methods for adaptive robotic behavior based on surface feature recognition

A computer-implemented method for robotic picking with surface feature protection, the method comprising:

• capturing, by at least one imaging sensor, image data of a pick area containing a plurality of objects;
• processing, by an image processing module, the captured image data to generate:
• a first pixel map identifying object surface regions suitable for robotic contact, and
• a second pixel map identifying surface features to be protected on the objects;
• executing, by a pick planning processor, a unified optimization algorithm that:
• assigns variable penalty costs to pixels identified as surface features to be protected,
• simultaneously computes a target object selection, a three-dimensional end effector pose, and an activation configuration for a plurality of independently controllable end effector portions, and
• maximizes an objective function combining pick success probability with reduction of contact with surface features to be protected;
• transmitting control signals to a robotic picking unit wherein the control signals are operable to:
• position a multi-axis robotic arm according to the computed three-dimensional pose, and
• selectively control portions of the end effector according to the computed activation configuration, wherein end effector portions aligned with surface features to be protected are deactivated or operated at a reduced activation level.

A non-transitory computer-readable storage medium storing instructions that, when executed by specialized image processing and optimization hardware, cause the hardware to perform operations comprising:

• receiving image data from at least one imaging sensor monitoring a pick area;
• generating, using neural network circuitry, a first pixel map identifying graspable surface regions and a second pixel map identifying surface features to be protected;
• executing a unified optimization that:
• treats surface features to be protected as fuzzy obstacles with tunable penalty costs,
• jointly optimizes object selection, end effector positioning, and selective end effector portion activation,
• computes an activation pattern that reduces contact or interaction over surface features to be protected;
• and
• outputting time-synchronized control signals for robotic arm motion and end effector control to execute the optimized pick operation.