Sparse geometry for super resolution video processing

A video processing device, comprising computing circuitry arranged to analyze an input video sequence to associate pixels of synthesized images of an output video sequence with respective directions of regularity belonging to a predefined set of directions, wherein the analysis of the input video sequence comprises:

• determining, from the predefined set of directions, a first subset of candidate directions for a region of a first image of the output video sequence;
• determining, from the predefined set of directions, a second subset of candidate directions for a corresponding region of a second synthesized image of the output video sequence following the first image, based on images of the input video sequence and the first subset of candidate directions;
• wherein determining the second subset of candidate directions includes,
• evaluating a first margin for a first direction of the first subset, the first margin equal to a contribution of the first direction to a reduction of a cost function associated with the first subset,
• evaluating a second margin for a second direction of the predefined set of directions, the second margin equal to a decrement of the cost function associated with the first subset resulting from addition of the second direction to the first subset, and
• substituting the second direction for the first direction responsive to the second margin exceeding the first margin;
• and
• detecting the respective directions of regularity for pixels of said region of the second synthesized image from the second subset of candidate directions.

A non-transitory computer-readable medium having a program stored therein, wherein said program comprises instructions to analyze an input video sequence when said program is run in a computer processing unit, to associate pixels of synthesized images of an output video sequence with respective directions of regularity belonging to a predefined set of directions, wherein the analysis of the input video sequence comprises:

• determining, from the predefined set of directions, a first subset of candidate directions for a region of a first image of the output video sequence;
• determining, from the predefined set of directions, a second subset of candidate directions for a corresponding region of a second synthesized image of the output video sequence following the first image, based on images of the input video sequence and the first subset of candidate directions;
• wherein determining the second subset of candidate directions includes,
• evaluating a first margin for a first direction of the first subset, the first margin equal to a contribution of the first direction to a reduction of a cost function associated with the first subset,
• evaluating a second margin for a second direction of the predefined set of directions, the second margin equal to a decrement of the cost function associated with the first subset resulting from addition of the second direction to the first subset, and
• substituting the second direction for the first direction responsive to the second margin exceeding the first margin;
• and
• detecting the respective directions of regularity for pixels of said region of the second synthesized image from the second subset of candidate directions.

A video processing method, comprising:

• receiving successive images of an input video sequence;
• analyzing the input video sequence to associate pixels of synthesized images of an output video sequence with respective directions of regularity;
• and
• generating the output video sequence from the input video sequence using the detected directions of regularity,
• wherein analyzing the input video sequence comprises:
• determining, from the predefined set of directions, a first subset of candidate directions for a region of a first image of the output video sequence;
• determining, from the predefined set of directions, a second subset of candidate directions for a corresponding region of a second synthesized image of the output video sequence following the first image, based on images of the input video sequence and the first subset of candidate directions;
• wherein determining the second subset of candidate directions includes,
• evaluating a first margin for a first direction of the first subset, the first margin equal to a contribution of the first direction to a reduction of a cost function associated with the first subset,
• evaluating a second margin for a second direction of the predefined set of directions, the second margin equal to a decrement of the cost function associated with the first subset resulting from addition of the second direction to the first subset, and
• substituting the second direction for the first direction responsive to the second margin exceeding the first margin;
• and
• detecting the respective directions of regularity for pixels of said region of the second synthesized image from the second subset of candidate directions.

A video processing apparatus, comprising computing circuitry for:

• receiving successive images of an input video sequence;
• analyzing the input video sequence to associate pixels of synthesized images of an output video sequence with respective directions of regularity;
• and
• generating the output video sequence from the input video sequence using the detected directions of regularity;
• wherein the analysis of the input video sequence comprises:
• determining, from the predefined set of directions, a first subset of candidate directions for a region of a first image of the output video sequence;
• determining, from the predefined set of directions, a second subset of candidate directions for a corresponding region of a second synthesized image of the output video sequence following the first image, based on images of the input video sequence and the first subset of candidate directions;
• wherein determining the second subset of candidate directions includes,
• evaluating a first margin for a first direction of the first subset, the first margin equal to a contribution of the first direction to a reduction of a cost function associated with the first subset,
• evaluating a second margin for a second direction of the predefined set of directions, the second margin equal to a decrement of the cost function associated with the first subset resulting from addition of the second direction to the first subset, and
• substituting the second direction for the first direction responsive to the second margin exceeding the first margin;
• and
• detecting the respective directions of regularity for pixels of said region of the second synthesized image from the second subset of candidate directions.