Mechanism for improved light estimation

A method for light estimation, the method comprising:

• receiving a plurality of frames associated with a scene, the plurality of frames including a first frame and a second frame occurring after the first frame;
• determining, based on image data of the first frame, a first light estimate associated with the scene, the first light estimate including at least one of a first estimated intensity or a first estimated color temperature of light emitted by at least one light source illuminating the scene;
• determining, based on image data of the second frame, a second light estimate associated with the scene, the second light estimate including at least one of a second estimated intensity or a second estimated color temperature of the light emitted by the at least one light source;
• assigning a first weight to one or more first estimated feature vectors associated with the first light estimate to generate one or more first weighted feature vectors and a second weight to one or more second estimated feature vectors associated with the second light estimate to generate one or more second weighted feature vectors;
• and
• generating an aggregate light estimate for modeling temporal lighting parameters associated with the scene based on the one or more first weighted feature vectors and the one or more second weighted feature vectors, the one or more first estimated feature vectors and the one or more second estimated feature vectors being generated using a neural network having an encoder-decoder architecture.

A non-transitory computer-readable storage medium for light estimation, the non-transitory computer-readable storage medium comprising instructions stored therein which, when executed by one or more processors, cause the one or more processors to:

• receive a plurality of frames associated with a scene, the plurality of frames including a first frame and a second frame occurring after the first frame;
• determine, based on image data of the first frame, a first light estimate associated with the scene, the first light estimate including at least one of a first estimated intensity or a first estimated color temperature of light emitted by at least one light source illuminating the scene;
• determine, based on image data of the second frame, a second light estimate associated with the scene, the second light estimate including at least one of a second estimated intensity or a second estimated color temperature of the light emitted by the at least one light source;
• assign a first weight to one or more first estimated feature vectors associated with the first light estimate to generate one or more first weighted feature vectors and a second weight to one or more second estimated feature vectors associated with the second light estimate to generate one or more second weighted feature vectors;
• and
• generate an aggregate light estimate for modeling temporal lighting parameters associated with the scene based on the one or more first weighted feature vectors and the one or more second weighted feature vectors, the one or more first estimated feature vectors and the one or more second estimated feature vectors being generated using a neural network having an encoder-decoder architecture.