Method and system for predicting a collision free posture of a kinematic system

A computer-implemented method for creating a training dataset for training a collision free detection (CFD) function, which comprises the steps of:

• a) receiving a set of 3D virtual environments and for each 3D virtual environment, a prescribed location defined within the 3D virtual environment;
• b) receiving a 3D virtual kinematic system and a set of 3D postures defined for the 3D virtual kinematic system;
• c) receiving, for each of the 3D virtual environments, a target task to be performed by the 3D virtual kinematic system with respect to the 3D virtual environment;
• d) automatically selecting one of the 3D virtual environments and automatically performing the following steps for a selected 3D virtual environment:
• d1) automatically selecting a posture within the set of postures;
• d2) automatically performing the following steps for a selected posture:
• i) creating a set of data containing an identification of the 3D virtual environment that has been selected, an identification of the prescribed location, an identification of the target task, and an identification of the posture that has been selected;
• ii) placing the 3D virtual kinematic system characterized by the selected posture at the prescribed location defined within the selected 3D virtual environment to create a 3D system containing the 3D virtual environment and the 3D virtual kinematic system placed at the prescribed location;
• iii) determining whether the 3D virtual kinematic system characterized by the selected posture is able to perform the target task when placed at the prescribed location, and if yes, proceeding with a next step iv), otherwise labeling a created set of data with a label characterized by a first value, storing in a database the label associated to the set of data, then selecting another posture of the set of postures that has not yet been selected if any, and repeating step d2) for a newly selected posture if such a posture exists, otherwise proceeds to step e);
• iv) determining whether a collision between the 3D virtual kinematic system characterized by the selected posture and the 3D virtual environment occurs;
• v) labeling the created set of data with the label, wherein the label is characterized by the first value if a collision occurs and a second value otherwise, if there is no collision;
• vi) storing in the database the label for the created set of data;
• vii) selecting another posture of the set of postures that has not yet been selected if any, and repeating step d2 for the newly selected posture, otherwise, if all postures have been selected, proceeding to step e);
• e) repeating step d) until all 3D virtual environments of the set of 3D virtual environments have been selected in order to create a training dataset wherein:
• as training input data:
• the 3D virtual environments and their respective said prescribed location and the target task, and the set of 3D postures;
• and
• as training output data:
• a set of labels, wherein each said label is characterized by either the second value or the first value, and wherein each said label is associated or assigned to a set of data containing an identification of the 3D virtual environment, of the prescribed location, of the target task, and of the posture for which the label value has been assigned.

A computer-implemented method for providing a trained collision free detection (CFD) function, which comprises the steps of:

• a) receiving a training dataset with a first interface, wherein the training dataset having training input data and training output data, wherein the training input data comprises:
• a set of 3D virtual environments, and for each 3D virtual environment of the set of 3D virtual environments, a prescribed location and a target task,
• a set of postures defined for a 3D virtual kinematic system;
• and wherein the training output data comprises:
• a set of labels, wherein each label is characterized by either a second value or a first value, and wherein each said label is associated or assigned to a set of data containing an identification of the 3D virtual environment, of the prescribed location, of the target task, and of the posture for which the label value has been assigned;
• b) automatically selecting one of the 3D virtual environments and automatically performing following steps for a selected 3D virtual environment:
• b1) acquiring a depth image of the 3D virtual environment from the prescribed location;
• b2) using the depth image, the target task, and the prescribed location as inputs to a dimensionality reduction algorithm configured for providing as output a 2D image;
• b3) repeating the following steps (i) and (ii) for training a convolutional neural network (CNN) of the CFD function until each posture of the set of postures has been selected once by:
• (i) automatically selecting a posture of the set of postures that has not yet been selected;
• and
• (ii) training the CNN, wherein the 2D image and the selected posture are used as input training data for the CNN and the label value associated to the set of data containing the identification of the 3D virtual environment, of the prescribed location, of the target task, and of the posture that has been selected is used as output training data;
• c) repeating step b) until all 3D virtual environments of the set of 3D virtual environments have been processed;
• and
• d) providing the trained CFD function with a second interface.

A data processing system, comprising:

• a processor;
• and
• an accessible memory, the data processing system configured to:
• a) receive a 3D virtual environment being a representation of a surrounding environment;
• b) receive a 3D representation of a kinematic system, namely a 3D virtual kinematic system, and a set of 3D postures defined for the 3D virtual kinematic system;
• c) receive a target task to be performed by the kinematic system with respect to the surrounding environment;
• d) receive a prescribed location within the 3D virtual environment, the prescribed location defining a position at which the 3D virtual kinematic system has to be placed within the 3D virtual environment;
• e) apply a collision free detection (CFD) function trained by a machine learning algorithm to a set of input data containing the 3D virtual environment, the target task, the prescribed location and the set of postures, wherein the CFD function is configured for generating a set of output data that is a subset of the set of 3D postures containing all 3D postures enabling the kinematic system to perform the target task when located at the prescribed location;
• f) select at least one collision free posture from the set of output data;
• and
• g) display the at least one collision free posture, which was selected in step f), of the 3D virtual kinematic system within the 3D virtual environment.

A non-transitory computer-readable medium encoded with executable instructions that, when executed, cause at least one processing system to:

• a) receive a 3D representation of a surrounding environment being a 3D virtual environment;
• b) receive a 3D representation of the kinematic system being a 3D virtual kinematic system and a set of 3D postures defined for the 3D virtual kinematic system;
• c) receive a target task to be performed by the kinematic system with respect to the surrounding environment;
• d) receive a prescribed location within the 3D virtual environment, the prescribed location defining a position at which the 3D virtual kinematic system has to be placed within the 3D virtual environment;
• e) apply a collision free detection (CFD) function trained by a machine learning algorithm to a set of input data containing the 3D virtual environment, the target task, the prescribed location and the set of postures, wherein the CFD function is configured for generating a set of output data that is a set of collision free postures enabling the kinematic system to perform the target task when located at said prescribed location;
• f) select at least one collision free posture from the set of collision free postures;
• and
• g) display the at least collision free posture, which was selected in step f), of the 3D virtual kinematic system within the 3D virtual environment.