Noise-based 3d layout prediction

A computer-implemented method of applying a function trained by machine-learning, the method comprising:

• machine-learning including:
• obtaining a dataset of ground truth 3D layouts, each ground truth 3D layout representing a respective scene and including:
• a respective floorplan,
• a respective 3D arrangement of one or more 3D bounding boxes, each bounding box being defined by a value of a predetermined set of one or more parameters, and
• for each 3D bounding box, a respective semantic category;
• obtaining a probability distribution of noise levels;
• for each ground truth 3D layout, obtaining a respective perturbed 3D layout computable by perturbing at least one parameter of at least one 3D bounding box of the ground truth 3D layout, the perturbing including:
• sampling a respective noise level based on the probability distribution;
• and
• for each respective parameter of the at least one parameter:
• sampling a respective noise value based on the respective noise level, and applying the respective noise value to the respective parameter;
• and
• training a function configured to:
• take an input 3D layout and a given noise level, the input 3D layout including:
• a given floorplan,
• a first 3D arrangement of one or more given 3D bounding boxes, each bounding box being defined in the first 3D arrangement by a first value of the predetermined set of one or more parameters, and
• for each given 3D bounding box, a given semantic category, and
• predict an output 3D layout, the output predicted 3D layout including:
• the given floorplan,
• a second 3D arrangement of the one or more given 3D bounding boxes, each bounding box being defined in the second 3D arrangement by a second value of the predetermined set of one or more parameters, and
• for each given 3D bounding box, the given semantic category,
• wherein the function is further configured to predict a second value of the predetermined set of one or more parameters different from the first value of the predetermined set of one or more parameters, and
• wherein the training is performed over the dataset based on a loss which penalizes a dissimilarity metric between each ground truth 3D layout and a respective predicted 3D layout obtainable by applying the function to the respective perturbed 3D layout;
• obtaining the input 3D layout and the given noise level, the input 3D layout including:
• the given floorplan,
• the first 3D arrangement of one or more given 3D bounding boxes, each bounding box being defined in the first 3D arrangement by a first value of the predetermined set of one or more parameters, and
• for each given 3D bounding box, the given semantic category;
• and
• applying the trained function one or more times starting from the input 3D layout to predict an output 3D layout, the output 3D layout including:
• the given floorplan,
• the second 3D arrangement of the one or more given 3D bounding boxes, each bounding box being defined in the second 3D arrangement by the second value of the predetermined set of one or more parameters, and for each given 3D bounding box, the given semantic category.

A device comprising:

• a processor;
• and
• a non-transitory computer-readable data storage medium having recorded thereon:
• a first computer program having instructions for performing machine-learning that when executed by the processor causes the processor to be configured to:
• obtain a dataset of ground truth 3D layouts, each ground truth 3D layout representing a respective scene and including:
• a respective floorplan,
• a respective 3D arrangement of one or more 3D bounding boxes, each bounding box being defined by a value of a predetermined set of one or more parameters, and
• for each 3D bounding box, a respective semantic category;
• obtain a probability distribution of noise levels;
• for each ground truth 3D layout, obtain a respective perturbed 3D layout computable by perturbing at least one parameter of at least one 3D bounding box of the ground truth 3D layout, the perturbing including:
• sampling a respective noise level based on the probability distribution;
• and
• for each respective parameter of the at least one parameter:
• sampling a respective noise value based on the respective noise level, and applying the respective noise value to the respective parameter;
• and
• train a function configured to:
• take an input 3D layout and a given noise level, the input 3D layout including:
• a given floorplan,
• a first 3D arrangement of one or more given 3D bounding boxes, each bounding box being defined in the first 3D arrangement by a first value of the predetermined set of one or more parameters, and
• for each given 3D bounding box, a given semantic category, and
• predict an output 3D layout, the output predicted 3D layout including:
• the given floorplan,
• a second 3D arrangement of the one or more given 3D bounding boxes, each bounding box being defined in the second 3D arrangement by a second value of the predetermined set of one or more parameters, and
• for each given 3D bounding box, the given semantic category,
• wherein the function is further configured to predict a second value of the predetermined set of one or more parameters different from the first value of the predetermined set of one or more parameters, and
• wherein the training is performed over the dataset based on a loss which penalizes a dissimilarity metric between each ground truth 3D layout and a respective predicted 3D layout obtainable by applying the function to the respective perturbed 3D layout,
• a second computer program having instructions for applying a function trained according to the machine-learning that when executed by the processor causes the processor to be configured to:
• obtain the input 3D layout and the given noise level, and
• apply the trained function one or more times starting from the input 3D layout to predict the output 3D layout;
• and
• a third computer program having instructions for training the function by machine-learning that when executed by the processor causes the processor to be configured to:
• train the function, the training being performed over the dataset based on the loss which penalizes the dissimilarity metric between each ground truth 3D layout and the respective predicted 3D layout obtainable by applying the function to the respective perturbed 3D layout.