Content-based passthrough image processing

A head-mounted-device (HMD) comprising:

• a non-transitory computer-readable storage medium;
• a display;
• and
• one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause performance of operations comprising:
• determining a context of a user viewing views of an extended reality (XR) environment on the display, the views comprising images of a physical environment captured via the outward-facing camera;
• generating information to provide to the ISP based on the context, wherein an exposure parameter or a white balance parameter of the camera is adjusted via the ISP based on the information, wherein the information is based on prioritizes one or more portions of the XR environment for parameter adjustment or is based on an identified user activity;
• capturing additional images of the physical environment via the camera based on the adjusted exposure parameter or the adjusted white balance parameter of the camera;
• and
• presenting additional views of the XR environment comprising the additional images.

A non-transitory computer-readable storage medium storing program instructions executable via one or more processors, of a head-mounted-device having a display, to perform operations comprising:

• determining a context of a user viewing views of an extended reality (XR) environment on the display, the views comprising images of a physical environment captured via the outward-facing camera;
• generating information to provide to the ISP based on the context, wherein an exposure parameter or a white balance parameter of the camera is adjusted via the ISP based on the information, wherein the information is based on prioritizes one or more portions of the XR environment for parameter adjustment or is based on an identified user activity;
• capturing additional images of the physical environment via the camera based on the adjusted exposure parameter or the adjusted white balance parameter of the camera;
• and
• presenting additional views of the XR environment comprising the additional images.