# US 2016/0366366 A1 · Frame rate conversion system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2016/0366366 A1 - Document ID: US20160366366A1 - Status: Published application - Kind: A1 - Application number: 14/737,781 - Filed: 2015-06-12 - Published: 2016-12-15 - Earliest claimed: 2015-06-12 - Assignees: Sharp Laboratories of America, Inc. - Applicants: Sharp Laboratories of America, Inc. - Inventors: Xu CHEN; Petrus J.L. VAN BEEK - Web page: https://patentgrep.com/patent/US20160366366A1 ## CPC - H04N 7/0127: {by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter} - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 7/40: Analysis of texture (depth or shape recovery from texture G06T7/529) - G06T 7/60: Analysis of geometric attributes - G06V 20/62: Text, e.g. of license plates, overlay texts or captions on TV images - G06T 2207/20024: Filtering details - H04N 5/144: {Movement detection (for video coding H04N19/503; analysis of motion in general G06T7/20)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A system for the conversion of video from one frame rate to a different frame rate. The system includes the application of graphical element and text detection and processing in frame rate up conversion. ## Claims ### Claim 1 A system for detecting a graphical element in an image of a series of images comprising: (a) a motion compensated error map process that identifies regions of said image that have a value greater than a threshold of the difference between said image and a motion compensated said image based upon said series of images to provide a likelihood map; (b) a first filter process based upon said image and a plurality of said series of images other than said image to temporally smooth said likelihood map; (c) modifying said likelihood map based upon said first filter process. ### Claim 2 (depends on 1) The system of claim 1 further comprising a second filter process to filter out spatial locations of said likelihood map at spatial locations that are generally central in said image and modifying said likelihood map based upon said second filter process. ### Claim 3 (depends on 1) The system of claim 1 further comprising a third filter process to filter out spatial locations of said likelihood map where a difference between said image and a non-motion compensated said one of said series of images other than said image is greater than a threshold and modifying said likelihood map based upon said third filter process. ### Claim 4 (depends on 1) The system of claim 1 further comprising a fourth filter process to spatially smooth said likelihood map and modifying said likelihood map based upon said fourth filter process. ### Claim 5 (depends on 1) The system of claim 1 further comprising a fifth filter process to filter out regions having a size at least less than a less than threshold and greater than a greater than threshold and modifying said likelihood map based upon said first fifth process. ### Claim 6 (depends on 1) The system of claim 1 further comprising a second filter process to filter out spatial locations of said likelihood map at spatial locations that are generally central in said image, a third filter process to filter out spatial locations of said likelihood map where a difference between said image and a non-motion compensated said one of said series of images other than said image is greater than a threshold, a fourth filter process to spatially smooth said likelihood map, a fifth filter process to filter out regions having a size at least less than a less than threshold and greater than a greater than threshold and modifying said likelihood map based upon said first second, third, fourth, and fifth processes. ### Claim 7 (depends on 1) The system of claim 1 wherein said threshold is based upon a maximum pixel value within said image less than a maximum potential pixel value of said image. ### Claim 8 (depends on 1) The system of claim 1 wherein said first filter process includes a time averaged mask. ### Claim 9 (depends on 8) The system of claim 8 wherein said first filter process includes a candidate mask. ### Claim 10 (depends on 9) The system of claim 9 wherein said first filter process includes recursive filtering. ### Claim 11 (depends on 5) The system of claim 5 wherein said fifth filter process includes a width threshold. ### Claim 12 (depends on 5) The system of claim 5 wherein said fifth filter process includes a height threshold. ### Claim 13 (depends on 5) The system of claim 5 wherein said fifth filter process includes an area threshold. ### Claim 14 (depends on 6) The system of claim 6 wherein said likelihood map is provided to said first filter process which provides a first modified likelihood map. ### Claim 15 (depends on 14) The system of claim 14 wherein said first filter process provides said first modified likelihood map to said second filter process which provides a second modified likelihood map. ### Claim 16 The system 15 wherein said second filter process provides said second modified likelihood map to said third filter process which provides a third modified likelihood map. ### Claim 17 (depends on 16) The system of claim 16 wherein said third filter process provides said third modified likelihood map to said fourth filter process which provides a fourth modified likelihood map. ### Claim 18 (depends on 17) The system of claim 17 wherein said fourth filter process provides said fourth modified likelihood map to said fifth filter process which provides a fifth modified likelihood map. ### Claim 19 A system for detecting a graphical element in an image of a series of images comprising: (a) a motion compensated error map process that identifies regions of said image that have a value greater than a threshold of the difference between said image and a motion compensated said image based upon said series of images to provide a likelihood map; (b) at least one filter to filter said likelihood map based upon anticipated size of said graphical element and temporal characteristics of said graphical element. ### Claim 20 A method for a frame rate conversion for a series of images comprising: (a) identifying regions of one of said images including textual content with a first mask; (b) identifying regions of another one of said images including textual content with a second mask; (c) estimating at least one motion vector for said regions of said one of said images corresponding to said first mask; (d) modifying said first mask based upon said at least one motion vector and said second mask; (e) interpolating an image based upon said modified first mask. ### Claim 21 (depends on 20) The method of claim 20 wherein said modifying is based upon a union of said first mask and a shifted said second mask based upon said at least one motion vector. ### Claim 22 (depends on 21) The method of claim 21 wherein said one of said images and another of said images are sequential images of said series of images. ### Claim 23 (depends on 22) The method of claim 22 wherein said at least one motion vector is estimated using a robust key point matching technique. ### Claim 24 (depends on 20) The method of claim 20 wherein said identifying includes corner detection. ### Claim 25 A method for a frame rate conversion for a series of images comprising: (a) receiving a first frame of said series of images together with a first mask identifying whether a particular pixel of said first frame belongs to a textual region or a background region; (b) receiving a second frame of said series of images together with a second mask identifying whether a corresponding particular pixel of said second frame belongs to a textual region or a background region; (c) calculating a pixel of an interpolated frame in a first manner for said series of images when said particular pixel of said first frame belongs to said textual region and said corresponding particular pixel belong to said background region, wherein said first manner uses only the information from either said first frame or said second frame. ### Claim 26 (depends on 25) The method of claim 25 further comprising calculating said pixel of said pixel of said interpolated frame in a second manner for said series of images when either said particular pixel of said first frame belongs to said textual region or said corresponding particular pixel belong to said background region, wherein said second manner uses information from both said first frame or said second frame.