Block level update rate control based on gaze sensing

A method for encoding video data based on gaze sensing, comprising

• receiving video data captured by at least one sensor array;
• receiving locations associated with a decoded video stream to designate skip block insertion points for encoding the received video data, wherein the locations are based on gaze points determined at a device;
• identifying, based upon the received locations, skip block insertion points prior to encoding the received video data, wherein the skip block insertion points designate blocks within inter-frames having reduced update rates;
• determining, for the identified skip block insertion points, a frequency for the reduced update rate;
• and
• encoding inter-frames having blocks associated with the identified skip block insertion points based on the determined frequency.

A device configured to decode video data based on gaze sensing, comprising:

• a display;
• a communication interface configured to exchange data over a network;
• a processor, coupled to the display and the communication interface;
• and
• a memory, coupled to the processor, which stores instructions causing the processor to:
• decode an encoded video stream received from an encoder associated with a camera,
• present the decoded video stream on the display,
• detect a gaze point of an operator viewing the display,
• designate locations associated with the decoded video stream, based upon the gaze point, as skip block insertion points, and
• send the locations to the encoder, wherein the encoder reduces an update rate of inter-frame coded blocks corresponding to the skip block insertion points when encoding video data produced by the camera.

A camera configured to encode video data based on gaze sensing, comprising:

• a sensor array;
• a communication interface configured to exchange data over a network;
• a controller, an image processor, and a video encoder, coupled to the sensor array and the communication interface;
• and
• a memory, coupled to the controller, the image processor, and the video encoder, which stores instructions causing at least one of the controller, the image processor, or the video encoder to:
• receive video data captured by the sensor array,
• receive locations associated with a decoded video stream to designate skip block insertion points for encoding the received video data, wherein the locations are based on gaze points determined at a client device,
• identify, based upon the received locations, skip block insertion points prior to encoding the received video data, wherein the skip block insertion points designate blocks within inter-frames having reduced update rates,
• determine, for the identified skip block insertion points, a frequency for the reduced update rate, and
• encode inter-frames having blocks associated with the identified skip block insertion points based on the determined frequency.