Systems and methods for surface modeling using polarization cues

A system for surface modeling, the system comprising:

• a polarization camera comprising a polarizing filter, the polarization camera being configured to capture polarization raw frames at different polarizations;
• and
• a processing system comprising a processor and memory storing instructions that, when executed by the processor, cause the processor to:
• receive one or more polarization raw frames of a surface of a physical object, the polarization raw frames corresponding to different polarizations of light;
• extract one or more first tensors in one or more polarization representation spaces from the polarization raw frames;
• and
• detect a surface characteristic of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces, the instructions further comprising instructions that, when executed by the processor, cause the processor to detect the stored characteristic by:
• loading a stored model corresponding to a location of the surface of the physical object, the stored model comprising one or more reference tensors in the one or more polarization representation spaces;
• and
• computing the surface characteristic in accordance with the stored model and the one or more first tensors in the one or more polarization representation spaces by computing a difference between the one or more reference tensors and the one or more first tensors in the one or more polarization representation spaces.

One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• receiving one or more polarization raw frames of a surface of a physical object, the polarization raw frames being captured at different polarizations by a polarization camera comprising a polarizing filter;
• extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames;
• and
• detecting a surface characteristic of the surface of the physical object based on the one or more first tensors in the one or more polarization representation spaces, comprising:
• loading a stored model corresponding to a location of the surface of the physical object, the stored model comprising one or more reference tensors in the one or more polarization representation spaces;
• and
• computing the surface characteristic in accordance with the stored model and the one or more first tensors in the one or more polarization representation spaces comprising computing a difference between the one or more reference tensors and the one or more first tensors in the one or more polarization representation spaces.