Computer-implemented operating method for handling work-pieces by an inpainting model reconstruction of occluded parts
AssigneeSiemens Aktiengesellschaft
InventorsAlexander Köbler, Harald Funk, Tobias Pfeifer, Robert Schmeißer, Yucheng Liao, Ralf Gross, Ingo Thon, Michael Fiedler
Life of this patent
Estimating the term…
Abstract
A system includes a perception system and a processing function including a camera. A vision system processes data from the camera. The system takes the camera data and reconstructs occluded parts to reestablish a violated prior assumption needed for a downstream system. The system is to be trained to reconstruct incomplete information in a sequence of images. The training may be based on historic knowledge or on simulated data.
Claims14
A method for operating a work-station in an industrial facility for processing workpieces, the workpieces including a first workpiece and a second workpiece, the method being a computer-implemented operating method and comprising:
- transporting the workpieces in the industrial facility in direction to the work-station, or moving the work-station towards the workpieces, wherein the work-station is configured to pick up the workpieces;
- during the transporting, scanning, by a perception system, the workpieces, such that scanning data is produced from the workpieces, the scanning, by the perception system, of the workpieces, such that the scanning data is produced from the workpieces comprising scanning the workpieces with a camera or a laser scanner, such that a three-dimensional (3D) point cloud is created, wherein an evaluation of the scanning data shows that the second workpiece is positioned such that the second workpiece is occluded from being completely scanned by the perception system;
- reconstructing an occluded part of the second workpiece in the scanning data using an inpainting process on the received scanning data, based on a trained model using training data;
- recombining the reconstructed 3D point cloud of a lower object and the reconstructed 3D point cloud of an upper object to a single point cloud without missing values due to occlusion;
- and
- removing, by the work-station, the workpieces, the removing comprising first picking the first workpiece, and then picking the second workpiece for further processing in the industrial facility.
(canceled)
A data processing system comprising:
- a processor configured to operate a work-station in an industrial facility for processing workpieces, the workpieces including a first workpiece and a second workpiece, the processor being configured to operate the work-station comprising the processor being configured to:
- transport the workpieces in the industrial facility in direction to the work-station, or move the work-station towards the workpieces, wherein the work-station is configured to pick up the workpieces;
- during the transport, scan, by a perception system, the workpieces, such that scanning data is produced from the workpieces, the scan, by the perception system, of the workpieces, such that the scanning data is produced from the workpieces comprising scan of the workpieces with a camera or a laser scanner, such that a three-dimensional (3D) point cloud is created, wherein an evaluation of the scanning data shows that the second workpiece is positioned such that the second workpiece is occluded from being completely scanned by the perception system;
- reconstruct an occluded part of the second workpiece in the scanning data using an inpainting process on the scanning data, based on a trained model using training data;
- recombine the reconstructed 3D point cloud of a lower object and the reconstructed 3D point cloud of an upper object to a single point cloud without missing values due to occlusion;
- and
- remove, by the work-station, the workpieces, the removal comprising a first pick of the first workpiece, and then a pick of the second workpiece for further processing in the industrial facility.
(canceled)
(canceled)
In a non-transitory computer-readable storage medium that stores instructions executable by one or more processors to operate a work-station in an industrial facility for processing workpieces, the workpieces including a first workpiece and a second workpiece, the instructions comprising:
- transporting the workpieces in the industrial facility in direction to the work-station, or move the work-station towards the workpieces, wherein the work-station is configured to pick up the workpieces;
- during the transporting, scanning, by a perception system, the workpieces, such that scanning data is produced from the workpieces, the scanning, by the perception system, of the workpieces, such that the scanning data is produced from the workpieces comprising scanning of the workpieces with a camera or a laser scanner, such that a three-dimensional (3D) point cloud is created, wherein an evaluation of the scanning data shows that the second workpiece is positioned such that the second workpiece is occluded from being completely scanned by the perception system;
- reconstructing an occluded part of the second workpiece in the scanning data using an inpainting process on the scanning data, based on a trained model using training data;
- recombining the reconstructed 3D point cloud of a lower object and the reconstructed 3D point cloud of an upper object to a single point cloud without missing values due to occlusion;
- and
- removing, by the work-station, the workpieces, the removing comprising first picking the first workpiece, and then picking the second workpiece for further processing in the industrial facility.
Dependencies are inferred from the claim text.