Lighting, color vector, and virtual background correction during a video conference session

A method of monitoring and adjusting the composition of a video frame during a video conference session, comprising:

• with a processor, executing a trained intelligent collaboration contextual session management system (ICCSMS) neural network to receive as input:
• indications of enabled AV processing instruction modules descriptive of how to visually transform a video frame during a video conference session executed by a multimedia multi-user collaboration application;
• sensor data from a plurality of sensors including an ambient light sensor to detect ambient light around a participant of the video conference session and a color senser to detect color vectors in the video frame;
• with the processor, applying AV processing instruction adjustments received as output from the trained ICCSMS neural network to adjust the lighting and color vectors of the video frame based on the sensor inputs and the AV processing instruction modules;
• with an AV processing instruction manager, directing processing resources associated with the AV processing instruction modules of the video frame during the video conference session;
• and
• offloading video frame processing tasks of at least one AV processing instruction module to an alternate processing unit of the information handling system.

An information handling system executing an intelligent collaboration contextual session management system (ICCSMS), comprising:

• a memory;
• a power management unit;
• a camera to capture video of a user participating in a video conference session;
• a processor configured to execute code instructions of a trained intelligent collaboration contextual session management system (ICCSMS) machine learning module to:
• determine that one or more AV processing instruction modules configured to visually transform a video frame during a video conference session executed by a multimedia multi-user collaboration application are enabled;
• receive as inputs to the trained ICCSMS neural network, sensor data from a plurality of sensors including an ambient light sensor to detect ambient light around a participant of the video conference session and a color senser to detect color vectors and shading in the video frame;
• an AV processing instruction manager configured to direct processing resources associated with at least one enabled AV processing instruction module of the video frame during a video conference session to a second processor based on an optimized offload instruction output from the ICCSMS neural network;
• the second processor configured to apply AV processing instruction adjustments received as output from the trained ICCSMS machine learning module to color blend and match the lighting and color vectors of a virtual background relative to an image of a user based on the sensor inputs via the enabled AV processing instructions.