Enhancing A Region Of Interest In Video Frames Of A Video Stream

A computer, preferably a media device, comprising:

• a computer readable storage medium having computer readable program code embodied therewith, and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform executable operations comprising:
• receiving media data defining base video frames of at least one base stream, said base stream being associated with one or more enhancement tile streams, wherein an enhancement tile stream comprises media data defining tiled video frames, a tiled video frame comprising a tile associated with media data for enhancing visual content in a subregion of an image region of a base video frame, wherein media data of an enhancement tile stream are independently decodable from the media data of said base stream;
• requesting media data of one or more enhancement tile streams, preferably one or more HEVC enhancement tile streams, the one or more tiles of the tiled video frames of the requested media data being associated with at least one predetermined region of interest in the image region of the base video frames;
• generating base video frames on the basis of the media data of the at least one base stream and generating tiled video frames on the basis of the media data of the one or more enhancement tile streams, the generated tiled video frames comprising one or more tiles forming enhanced visual content in the at least one region of interest;
• and, forming video frames comprising an enhanced region of interest by modifying at least part of the visual content of the at least one region of interest in the video frames of said base stream on the basis of at least part of the enhanced visual content of the tiled video frames, said modifying comprising:
• replacing or blending at least part of the visual content of the at least one region of interest in the video frames of said base stream with at least part of the enhanced visual content of the tiled video frames.

Non-transitory computer-readable storage media comprising a data structure for enabling enhancement of a region of interest in video frames of a video stream, said data structure comprising computer readable program code, said code comprising:

• at least one base stream identifier and one or more enhancement tile stream identifiers associated with the at least one base stream identifier, preferably a stream identifier comprising (at least part of) an URL or comprising information for determining such URL, configured for enabling a client device to request a base stream and/or one or more enhancement tile streams;
• tile position information configured for signaling the client device on the spatial relation between visual content in the video frames of the base stream and visual content in the tiles of the tiled video frames of the one or more enhancement tile streams, preferably said position information comprising one or more spatial relationship descriptors (SRDs), a spatial relationship descriptor being associated with a base stream and/or an enhancement tile streams, the spatial relationship descriptor configured for signaling the client device about the position of the visual content in the video frames of the base stream and/or about the position of a tile in the tiled video frames of the tiled enhancement tile stream;
• and, optionally,
• enhancement information associated with at least one of the one or more enhancement tile streams, the enhancement information configured for signaling the client device about the type of enhancement provided by an enhancement tile stream, and, optionally, for providing the client device one or more enhancement parameters associated with at least one of the resolution, bitrate, color depth, frame rate of the one or more enhancement tile streams defined by said enhancement tile stream identifiers.