Method, device and system for producing a merged digital video sequence

A system for producing a frame of a merged digital video sequence, the system comprising:

• a digital video camera arranged to obtain a first digital video sequence;
• an analysis unit, of the digital video camera, the analysis unit arranged to analyze frames of the first digital video sequence to identify pixel blocks considered to be of relevance;
• a mask generator, of the analysis unit, the mask generator arranged to output a mask indicating pixel blocks of relevance in the frames of the first digital video sequence;
• an encoding unit, of the digital video camera, the encoding unit arranged to encode a frame of the first digital video sequence using the mask indicating pixel blocks of relevance into a frame in an encoded version of the first video sequence, wherein the pixel blocks being masked out as not being of relevance are encoded using skip blocks;
• the digital video camera is further arranged to transmit the encoded version of the first video sequence over a network to a client,
• the digital video camera is further arranged to obtain a second digital video sequence, wherein the first and second digital video sequences both include at least one block of pixels that depicts approximately the same area of relevance and are not encoded as skip blocks wherein the frames of the first digital video sequence have a higher number of pixels than the second digital video sequence,
• the digital video camera being further arranged to transmit the second digital video sequence over the network to the client,
• the system further comprises a scaler at the client arranged to scale a frame of the second digital video sequence such that an up-scaled frame of the second digital video sequence is obtained, the up-scaled frame of the second digital video sequence having the same number of pixels as the frame of the first video sequence;
• the system further comprises a decoder at the client arranged to decode the encoded frame of the first digital video sequence;
• the system further comprises a skip block identifier arranged to, at the client, while decoding, using the decoder, the encoded frame of the first digital video sequence, identify a position for at least one skip block and a position for at least one non-skip block in the frame of the first digital video sequence;
• the system further comprises a block extractor at the client arranged to extract based on the identified position for the at least one skip block in the frame of the first digital video sequence, at least one corresponding block of pixels from the up-scaled frame of the second digital video sequence and based on the identified position for the at least one non-skip block in the frame of the first digital video sequence, at least one corresponding block of pixels from the frame of the first digital video sequence;
• and
• a merging unit at the client arranged to merge the extracted block of pixels from the frame of the second digital video sequence and the extracted block of pixels from the frame of the first digital video sequence such that the frame of the merged video sequence is produced, wherein the pixels of the extracted block of pixels from the frame of the first digital video sequence are generated independently of the pixels of the extracted block of pixels from the frame of the second digital video sequence.

A method for producing a frame of a merged digital video sequence, comprising:

• obtaining a first digital video sequence using an image processor;
• obtaining a second digital video sequence using the image processor, wherein frames of the first digital video sequence have a higher number of pixels than the frames of the second digital video sequence;
• identifying relevant pixel blocks in the frames of the first digital video sequence using an analysis unit, wherein the second digital video sequence includes at least one pixel block that is not encoded as a skip block and that depicts approximately the same area of relevance as that depicted in at least one relevant pixel block identified in the frames of the first digital video sequence;
• generating a mask based on the identified relevant pixel blocks of the frames of the first digital video sequence;
• encoding a frame of the first digital video sequence using the mask into a frame in an encoded version of the first video sequence using an encoder, wherein non-relevant pixel blocks being masked out are encoded using skip blocks;
• transmitting the encoded version of the first video sequence and the second digital video sequence to a client;
• up-scaling a frame of the second digital video sequence using a scaler at the client, wherein an up-scaled frame has a same number of pixels as the frame of the first digital video sequence;
• identifying, at the client, a position for at least one skip block and a position for at least one non-skip block in the frame of the first digital video sequence while decoding the encoded frame of the first digital video sequence and using a skip block identifier;
• extracting at least one corresponding block of pixels from the up-scaled frame of the second digital video sequence using a block extractor at the client and based on the identified position for the at least one skip block in the frame of the first digital video sequence;
• extracting at least one corresponding block of pixels from the frame of the first digital video sequence using the block extractor and based on the identified position for the at least one non-skip block in the frame of the first digital video sequence;
• and
• merging the extracted block of pixels from the frame of the second digital video sequence and the extracted block of pixels from the frame of the first digital video sequence using a merging unit at the client to produce the frame of the merged video sequence, wherein the pixels of the extracted block of pixels from the frame of the first digital video sequence are generated independently of the pixels of the extracted block of pixels from the frame of the second digital video sequence.