Neural radiance field for vehicle
AssigneesFord Global Technologies, LLC, The Regents of the University of Michigan
InventorsRadhika Ravi, Manikandasriram Srinivasan Ramanagopal, Ramanarayan Vasudevan, Katherine Skinner
Life of this patent
Estimating the term…
Abstract
A computer includes a processor and a memory, and the memory stores instructions executable by the processor to train a NeRF network to model a dynamic scene and, during the training, supervise the NeRF network with data from an event camera. The NeRF network is a neural radiance field modeling a geometry of the scene and a light intensity of the scene. The NeRF network includes a baseline network modeling the scene at an initial time and a deformation network modeling change to the scene since the initial time.
Claims18
A computer comprising a processor and a memory, the memory storing instructions executable by the processor to:
- train a NeRF network to model a dynamic scene, the NeRF network being a neural radiance field modeling a geometry of the scene and a light intensity of the scene, the NeRF network including a baseline network modeling the scene at an initial time and a deformation network modeling change to the scene since the initial time;
- during the training, supervise the NeRF network with data from an event camera, the data from the event camera including a plurality of events, each event being a change in the light intensity;
- and
- update the NeRF network based on a loss function, the loss function including a nonevent loss based on a period between consecutive events at a pixel location;
- wherein, in response to the event occurring over the period at a neighboring pixel location, the nonevent loss includes a difference between a predicted change in the light intensity at the pixel location over the period according to the NeRF network and a contrast threshold.
A method comprising:
- training a NeRF network to model a dynamic scene, the NeRF network being a neural radiance field modeling a geometry of the scene and a light intensity of the scene, the NeRF network including a baseline network modeling the scene at an initial time and a deformation network modeling change to the scene since the initial time;
- during the training, supervising the NeRF network with data from an event camera, the data from the event camera including a plurality of events, each event being a change in the light intensity;
- and
- updating the NeRF network based on a loss function, the loss function including a nonevent loss based on a period between consecutive events at a pixel location;
- wherein, in response to the event occurring over the period at a neighboring pixel location, the nonevent loss includes a difference between a predicted change in the light intensity at the pixel location over the period according to the NeRF network and a contrast threshold.
Dependencies are inferred from the claim text.