Method and apparatus for training a neural network
AssigneeCondense Reality Ltd.
InventorsNicholas Fellingham, Oliver Moolan-Feroze
Life of this patent
Estimating the term…
Abstract
A first aspect of the invention provides a method of training a neural network for capturing volumetric video, comprising: generating a 3D model of a scene; using the 3D model to generate a high fidelity depth map; capturing a perceived depth map of the scene, having a field of view that is aligned with a field of view of the high fidelity depth map; and training the neural network based on the high fidelity depth map and the perceived depth map, wherein the high fidelity depth map has a higher fidelity to the scene than the perceived depth map has.
Claims20
A method of training a neural network for capturing volumetric video, comprising:
- generating a 3D model of a scene;
- using the 3D model to generate a high fidelity depth map;
- capturing a perceived depth map of the scene, having a field of view that is aligned with a field of view of the high fidelity depth map;
- and
- training the neural network based on the high fidelity depth map and the perceived depth map,
- wherein the high fidelity depth map has a higher fidelity to the scene than the perceived depth map has.
Apparatus for training a neural network for capturing volumetric video, comprising:
- a neural network;
- a plurality of cameras configured to generate a 3D model of a scene;
- and
- a processor configured to use the 3D model of the scene to generate a high fidelity depth map;
- wherein:
- a first subset of the plurality of cameras is further configured to capture a perceived depth map of the scene, the perceived depth map having a field of view that is aligned with a field of view of the high fidelity depth map;
- the neural network is configured to receive, as training inputs, the high fidelity depth map and the perceived depth map;
- and
- the high fidelity depth map has a higher fidelity to the scene than the perceived depth map has.
Apparatus for training a neural network for capturing volumetric video, comprising:
- a neural network;
- and
- a computer program configured to:
- generate a 3D model of a scene;
- use the 3D model to generate a high fidelity depth map;
- and
- generate a perceived depth map of the scene, the perceived depth map having a field of view that is aligned with a field of view of the high fidelity depth map, wherein:
- the neural network is configured to receive, as training inputs, the high fidelity depth map and the perceived depth map;
- and
- the high fidelity depth map has a higher fidelity to the scene than the perceived depth map has.
Apparatus for training a neural network for capturing volumetric video, comprising:
- a neural network;
- a first plurality of cameras configured to generate a 3D model of a scene;
- a processor configured to use the 3D model of the scene to generate a high fidelity depth map;
- and
- a second camera, not included in the first plurality of cameras, configured to capture a perceived depth map of the scene, the perceived depth map having a field of view that is aligned with a field of view of the high fidelity depth map;
- wherein:
- the neural network is configured to receive, as training inputs, the high fidelity depth map and the perceived depth map;
- and
- the high fidelity depth map has a higher fidelity to the scene than the perceived depth map has.
Dependencies are inferred from the claim text.