Image decomposition and path-space motion estimation

One or more computer-readable non-transitory storage media embodying software comprising instructions operable when executed to:

• perform a light path analysis of an image comprising a scene, wherein the scene comprises at least one refractive or reflective object;
• for each of a plurality of pixels in the image:
• decompose a color of the pixel based on the light path analysis into a plurality of components, each of the components representing a contribution to lighting in the scene by a different type of light interaction, wherein the decomposing the color of the pixel comprises, for each type of light interaction:
• extracting a color component representing a contribution by the respective type of light interaction to color for the scene;
• extracting a reflectance component representing a contribution by the respective type of light interaction to texture for the scene;
• computing an irradiance component representing a contribution by the respective type of light interaction to lighting for the scene;
• and
• assessing a residual component representing a contribution by all unmatched paths to lighting for the scene;
• for each of the components, extract one or more motion vectors for each of the components in order to capture motion in the scene;
• and
• compute, based on the motion vectors, a final contribution of each of the components to the color of the pixel.

A system comprising one or more processors and a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:

• perform a light path analysis of an image comprising a scene, wherein the scene comprises at least one refractive or reflective object;
• for each of a plurality of pixels in the image:
• decompose a color of the pixel based on the light path analysis into a plurality of components, each of the components representing a contribution to lighting in the scene by a different type of light interaction, wherein the decomposing the color of the pixel comprises, for each type of light interaction:
• extracting a color component representing a contribution by the respective type of light interaction to color for the scene;
• extracting a reflectance component representing a contribution by the respective type of light interaction to texture for the scene;
• computing an irradiance component representing a contribution by the respective type of light interaction to lighting for the scene;
• and
• assessing a residual component representing a contribution by all unmatched paths to lighting for the scene;
• for each of the components, extract one or more motion vectors for each of the components in order to capture motion in the scene;
• and
• compute, based on the motion vectors, a final contribution of each of the components to the color of the pixel.