Method and apparatus for generating compact multiplane images

A device for generating a multiplane image, comprising:

• an image acquisition system configured for acquiring a multi-view image of a 3D scene comprising n views indexed from 1 to n;
• and
• a processor associated with a memory and configured to perform:
• generating a first multiplane image by inputting views of the multi-view image indexed by 1 to n−1 and a set of at least one parameter in a convolutional neural network trained to generate a multiplane image representative of a multi-view image, the set of at least one parameter being used in a penalization term of the convolutional neural network to determine a redundancy of information between layers of the multiplane image;
• synthetizing the n-th view based on the first multiplane image;
• updating the set of at least one parameter responsive to a comparison between the synthetized n-th view and the n-th view of the multi-view image;
• and
• generating a second multiplane image by inputting the n views of the multi-view image and the updated set of at least one parameter in the convolutional neural network.

A method for training a convolutional neural network to generate a multiplane image, the method comprising:

• obtaining first data representative of a 3D scene;
• inputting the first data in the convolutional neural network to generate the multiplane image;
• and
• adapting the convolutional neural network according to the multiplane image and to a penalization function determined as a normalized sum over pixels of an accumulated alpha image of the multiplane image of a maximum value between zero and a difference, the difference being between a value of the accumulated alpha image at the pixel and a parameter of a set of at least one parameter, the set of at least one parameter being used to determine a redundancy of information between layers of the multiplane image.

A device for training a convolutional neural network to generate a multiplane image, the device comprising a memory associated with a processor configured for applying a training process comprising:

• obtaining first data representative of a 3D scene;
• inputting the first data in the convolutional neural network to generate the multiplane image;
• and
• adapting the convolutional neural network according to the multiplane image and to a penalization function determined as a normalized sum over pixels of an accumulated alpha image of the multiplane image of a maximum value between zero and a difference, the difference being between a value of the accumulated alpha image at the pixel and a parameter of a set of at least one parameter, the set of at least one parameter being used to determine a redundancy of information between layers of the multiplane image.