Frame rate converting apparatus, pan/tilt determining apparatus, and video apparatus

A frame rate converting apparatus which converts a frame rate of a moving image, comprising:

• motion detecting means which detects information related to motion of an image for each frame of a moving image to be converted;
• pan/tilt determining means which determines whether the image is in a pan/tilt state on the basis of the information related to the motion of the image detected by the motion detecting means for each frame of the moving image to be converted;
• moving distance setting means which sets 0 as a moving distance of an image with respect to a frame the image of which is not determined as an image in a pan/tilt state by the pan/tilt determining means, which calculates a moving distance of an image from the information related to the motion of the image detected by the motion detecting means with respect to a frame the image of which is determined as an image in a pan/tilt state, and which sets the obtained moving distance as a moving distance of the image;
• time-direction smoothing means which smoothes the moving distance of the image set for each frame by the moving distance setting means in a direction of time;
• and
• prediction image generating means which generates a prediction image necessary for frame rate conversion on the basis of the moving distance of the image obtained by the time-direction smoothing means.

A video apparatus having a frame rate converting apparatus which converts a frame rate of a moving image, wherein

• the frame rate converting apparatus includes:
• motion detecting means which detects information related to motion of an image for each frame of a moving image to be converted;
• pan/tilt determining means which determines whether the image is in a pan/tilt state on the basis of the information related to the motion of the image detected by the motion detecting means for each frame of the moving image to be converted;
• moving distance setting means which sets 0 as a moving distance of an image with respect to a frame the image of which is not determined as an image in a pan/tilt state by the pan/tilt determining means, which calculates a moving distance of an image from the information related to the motion of the image detected by the motion detecting means with respect to a frame the image of which is determined as an image in a pan/tilt state, and which sets the obtained moving distance as a moving distance of the image;
• and
• prediction image generating means which generates a prediction image necessary for frame rate conversion on the basis of the moving distance of the image set for each frame by the moving distance setting means.

A video apparatus having a frame rate converting apparatus which converts a frame rate of a moving image, wherein

• the frame rate converting apparatus includes:
• motion detecting means which detects information related to motion of an image for each frame of a moving image to be converted;
• pan/tilt determining means which determines whether the image is in a pan/tilt state on the basis of the information related to the motion of the image detected by the motion detecting means for each frame of the moving image to be converted;
• moving distance setting means which sets 0 as a moving distance of an image with respect to a frame the image of which is not determined as an image in a pan/tilt state by the pan/tilt determining means, which calculates a moving distance of an image from the information related to the motion of the image detected by the motion detecting means with respect to a frame the image of which is determined as an image in a pan/tilt state, and which sets the obtained moving distance as a moving distance of the image;
• time-direction smoothing means which smooths the moving distance of the image set for each frame by the moving distance setting means in a direction of time;
• and
• prediction image generating means which generates a prediction image necessary for frame rate conversion on the basis of the moving distance of the image obtained by the time-direction smoothing means.

A pan/tilt determining apparatus which determines whether an image is in a pan/tilt state for each frame of a moving image, comprising:

• means which calculates a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values for each of a plurality of motion vector detecting regions set in a video area of each frame of a moving image by a typical point matching method;
• means which specifies a region in which the detected motion vector has high reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have high reliability;
• means which determines whether the number of detection regions in which the detected motion vectors have high reliability is not less than a first threshold value;
• means which, when the number of detection regions in which the detected moving vectors have high reliability is less than the first threshold value, determines that the frame is not panned/tilted;
• means which, when the number of detection regions in which the detected motion vectors have high reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have high reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have high reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;
• means which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;
• means which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted;
• and
• means which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, determines that the frame is panned/tilted.

A pan/tilt determining apparatus which determines whether an image is in a pan/tilt state for each frame of a moving image, comprising:

• means which calculates a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values for each of a plurality of motion vector detecting regions set in a video area of each frame of a moving image by a typical point matching method;
• means which specifies a region in which the detected motion vector has high reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have high reliability;
• means which determines whether the number of detection regions in which the detected motion vectors have high reliability is not less than a first threshold value;
• means which, when the number of detection regions in which the detected moving vectors have high reliability is less than the first threshold value, determines that the frame is not panned/tilted;
• means which, when the number of detection regions in which the detected motion vectors have high reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have high reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have high reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;
• means which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;
• means which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted;
• means which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, specifies detection regions in which motion vectors have similarity in the detection regions in which the motion vectors have high reliability and the motions of the plurality of types are not present on the basis of a motion vector detected for the detection region in which the motion vector has high reliability and the motions of the plurality of types are not present, and which calculates the number of detection regions in which motion vectors have similarity;
• means which determines whether the number of detection regions in which motion vectors have similarity is not less than a third threshold value;
• means which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is less than the third threshold value, determines that the frame is not panned/tilted;
• and
• means which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is not less than the third threshold value, determines that the frame is panned/tilted.

A video apparatus having a pan/tilt determining apparatus which determines whether an image is in a pan/tilt state for each frame of a moving image, wherein

• the pan/tilt determining apparatus includes:
• means which calculates a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values for each of a plurality of motion vector detecting regions set in a video area of each frame of a moving image by a typical point matching method;
• means which specifies a region in which the detected motion vector has high reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have high reliability;
• means which determines whether the number of detection regions in which the detected motion vectors have high reliability is not less than a first threshold value;
• means which, when the number of detection regions in which the detected moving vectors have high reliability is less than the first threshold value, determines that the frame is not panned/tilted;
• means which, when the number of detection regions in which the detected motion vectors have high reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have high reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have high reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;
• means which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;
• means which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted;
• and
• means which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, determines that the frame is panned/tilted.

A video apparatus having a pan/tilt determining apparatus which determines whether an image is in a pan/tilt state for each frame of a moving image, wherein

• the pan/tilt determining apparatus includes:
• means which calculates a motion vector, a minimum value of correlative accumulated values, and an average value of the correlative accumulated values for each of a plurality of motion vector detecting regions set in a video area of each frame of a moving image by a typical point matching method;
• means which specifies a region in which the detected motion vector has high reliability of the motion vector detecting regions on the basis of the average value of the correlative accumulated values detected for each of the motion vector detecting regions and which calculates the number of detection regions in which the detected motion vectors have high reliability;
• means which determines whether the number of detection regions in which the detected motion vectors have high reliability is not less than a first threshold value;
• means which, when the number of detection regions in which the detected moving vectors have high reliability is less than the first threshold value, determines that the frame is not panned/tilted;
• means which, when the number of detection regions in which the detected motion vectors have high reliability is not less than the first threshold value, specifies a detection region in which motions of a plurality of types are present and a detection region in which motions of a plurality of types are not present in the detection regions in which the motion vectors have high reliability on the basis of an average value of correlative accumulated values detected for the detection regions in which the motion vectors have high reliability and a minimum value of the correlative accumulated values, and which calculates the number of detection regions in which the motions of the plurality of types are present;
• means which determines whether the number of detection regions in which the motions of the plurality of types are present is less than a second threshold value;
• means which, when the number of detection regions in which the motions of the plurality of types are present is not less than the second threshold value, determines that the frame is not panned/tilted;
• means which, when the number of detection regions in which the motions of the plurality of types are present is less than the second threshold value, specifies detection regions in which motion vectors have similarity in the detection regions in which the motion vectors have high reliability and the motions of the plurality of types are not present on the basis of a motion vector detected for the detection region in which the motion vector has high reliability and the motions of the plurality of types are not present, and which calculates the number of detection regions in which motion vectors have similarity;
• means which determines whether the number of detection regions in which motion vectors have similarity is not less than a third threshold value;
• means which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is less than the third threshold value, determines that the frame is not panned/tilted;
• and
• means which, when the number of detection regions which are determined as detection regions in which the motion vectors have similarity is not less than the third threshold value, determines that the frame is panned/tilted.

A frame rate converting apparatus which converts a frame rate of a moving image, comprising:

• means which defines one of two continuous frames in a moving image as a current frame, defines the other as a previous frame, and defines a frame generated between the current frame and the previous frame as a prediction frame, which calculates a shift distance of the prediction image with respect to a previous frame image and a shift distance of the prediction image with respect to a current frame image, on the basis of a moving distance of an image between a previous frame image and a current frame image and a frame rate;
• means which shifts the previous frame image by using the shift distance of the prediction image with respect to the previous frame image to generate a first virtual prediction image constituted by a first blank portion in which the previous frame image is not present and a first original image portion in which the previous frame image is present;
• means which shifts the current frame image by using the shift distance of the prediction image with respect to the current frame image to generate a second virtual prediction image constituted by a second blank portion in which the current frame image is not present and a second original image portion in which the current frame image is present;
• and
• prediction image generating means which generates the prediction image on the basis of the previous frame image, the current frame image, the first virtual prediction image, and the second virtual prediction image,
• wherein the prediction image generating means, if the first virtual prediction image and the second virtual prediction image are overlapped, generates a prediction image from the second virtual prediction image in a portion where the first blank portion in the first virtual prediction image is overlapped on the second original image portion in the second virtual prediction image, generates a prediction image from the first virtual prediction image in a portion where the second blank portion in the second virtual prediction image is overlapped on the first original image portion in the first virtual prediction image, generates a prediction image by an image obtained by weighted-summing the current frame image and the previous frame image in a portion where the first blank portion and the second blank portion are overlapped, and generates a prediction image from one of the first virtual prediction image and the second virtual prediction image in a portion where the first original image portion in the first virtual prediction image is overlapped on the second original image portion in the second virtual prediction image.

A video apparatus having a frame rate converting apparatus which converts a frame rate of a moving image, wherein

• the frame rate converting apparatus includes:
• means which defines one of two continuous frames in a moving image as a current frame, defines the other as a previous frame, and defines a frame generated between the current frame and the previous frames as a prediction frame, which calculates a shift distance of the prediction image with respect to a previous frame image and a shift distance of the prediction image with respect to a current frame image, on the basis of a moving distance of an image between a previous frame image and a current frame image and a frame rate;
• means which shifts the previous frame image by using the shift distance of the prediction image with respect to the previous frame image to generate a first virtual prediction image constituted by a first blank portion in which the previous frame image is not present and a first original image portion in which the previous frame image is present;
• means which shifts the current frame image by using the shift distance of the prediction image with respect to the current frame image to generate a second virtual prediction image constituted by a second blank portion in which the current frame image is not present and a second original image portion in which the current frame image is present;
• and
• prediction image generating means which generates the prediction image on the basis of the previous frame image, the current frame image, the first virtual prediction image, and the second virtual prediction image,
• wherein the prediction image generating means, if the first virtual prediction image and the second virtual prediction image are overlapped, generates a prediction image from the second virtual prediction image in a portion where the first blank portion in the first virtual prediction image is overlapped on the second original image portion in the second virtual prediction image, generates a prediction image from the first virtual prediction image in a portion where the second blank portion in the second virtual prediction image is overlapped on the first original image portion in the first virtual prediction image, generates a prediction image by an image obtained by weighted-summing the current frame image and the previous frame image in a portion where the first blank portion and the second blank portion are overlapped, and generates a prediction image from one of the first virtual prediction image and the second virtual prediction image in a portion where the first original image portion in the first virtual prediction image is overlapped on the second original image portion in the second virtual prediction image.