# US 2019/0172494 A1 · Reference and non-reference video quality evaluation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0172494 A1 - Document ID: US20190172494A1 - Status: Published application - Kind: A1 - Application number: 16/252,425 - Filed: 2019-01-18 - Published: 2019-06-06 - Earliest claimed: 2013-05-28 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Krasimir D. Kolarov; Steven E. Saunders - Web page: https://patentgrep.com/patent/US20190172494A1 ## CPC - H04N 17/002: {for television cameras} - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - H04N 17/004: {for digital television systems} - H04N 19/426: using memory downsizing methods - G06T 2207/30168: Image quality inspection - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract Devices and methods for determining image quality using full-reference and non-reference techniques. Full reference image quality may be determined prior to output of an image or video frame from an image sensor processor by temporarily retaining image data from the image sensor and comparing processed image data of the image to the retained, non-processed image data of the same image. Full reference image quality determination may be assisted by a heuristic-based fault indicator. Image quality may also be determined by a non-reference technique of matching the image to one of various scenarios that are associated with sets of heuristics and applying the heuristics of the particular scenario to the image. Instead of relying on a nominal frame rate, video timing quality may be determined by comparing the capture time interval between successive video frames to the presentation time interval of the same video frames. ## Claims ### Claim at position 1 -20. (canceled) ### Claim 21 A system, comprising: one or more computer processors; and memory storing program instructions that are executed by the one or more computer processors to implement a heuristic-based analyzer configured to: evaluate captured video to detect one or more faces in the video; and in response to detection of one or more faces in the video, analyze at least a portion of the video including the detected one or more faces with one or more face-scenario-specific heuristics; and output a result of said analyze in regard to the at least a portion of the video including the detected one or more faces. ### Claim 22 (depends on 21) The system of claim 21, wherein the face-scenario-specific heuristics comprise quality indicators for one or more characteristic features of the video; and wherein to perform said analyze at least the portion of the video with the one or more face-scenario-specific heuristics, the memory stores additional instructions that are executed to apply the quality indicators to corresponding features of the video to determine one or more quality metric measurements; and wherein the output results is based on the determined one or more quality metric measurements. ### Claim 23 (depends on 21) The system of claim 21, wherein the memory stores additional program instructions that are executed to: repeat said evaluate, and said analyze for a plurality of video images of the video to create a plurality of image quality metric measurements from a respective plurality of video images; determine, based at least in part on a comparison of at least one of the plurality of image quality metric measurements to a threshold, whether the image quality metric measurements indicate poor image quality; and in response to a determination that the at least one of the image quality metric measurements exceed the threshold, output an image quality indicator for use in navigation to a particular image associated with the image quality metric measurements. ### Claim 24 (depends on 21) The system of claim 21, wherein the face-scenario-specific heuristics identify high contrasts and sharp detail in a facial area. ### Claim 25 (depends on 21) The system of claim 21, further comprising: an image sensor to capture the video; and an image sensor data processor configured to process the video captured from the image sensor into processed versions of the captured video and evaluate a quality of the processed version of the captured video; wherein the heuristic-based analyzer is configured to work in coordination with the image sensor data processor. ### Claim 26 (depends on 21) The system of claim 21, further comprising: an image sensor to capture the video; and a video data pre-processor configured to correct for one or more distortions in the video from the image sensor, wherein the face-scenario-specific heuristics are applied to determine that the video data pre-processor is functioning correctly. ### Claim 27 (depends on 21) The system of claim 21, further comprising: an image sensor to capture the video; and an encoder configured to compress image data of the video from the image sensor, wherein to perform said analyze at least the portion of the video the memory stores program instructions that cause the heuristic-based analyzer to analyze image data decompressed from the compressed image data. ### Claim 28 (depends on 21) The system of claim 21, wherein to perform said analyze the video with the one or more face-scenario-specific heuristics the memory stores program instructions that are executed to identify an area with chroma that falls within a human skin-tine chromaticity range. ### Claim 29 (depends on 21) The system of claim 21, wherein to perform said analyze the video with the one or more face-scenario-specific heuristics the memory stores program instructions that are executed to: identify one or more facial features; and apply one or more of the face-scenario-specific heuristics in accordance with finding high contrasts and sharp detail in a facial area. ### Claim 30 (depends on 21) The system of claim 21, further comprising; an encoder configured to process regions of frames of the video as macroblocks, or stripes of the video; wherein said analyze the video with one or more face-scenario-specific heuristics is performed on one or more of the regions of the video instead of on an entire frame of the video. ### Claim 31 (depends on 21) The system of claim 21, wherein to perform said output the result of said analyzing in regard to the at least a portion of the video including the detected one or more faces, program instructions cause the one or more processors to perform instructing an indication of image quality of the video as feedback displayed on a device display. ### Claim 32 One or more non-transitory, computer-readable storage media storing instructions that, when executed on or across one or more processors, cause the one or more processors to perform: evaluate captured video to detect one or more faces in the video; and in response to detection of one or more faces in the video, analyze at least a portion of the video including the detected one or more faces with one or more face-scenario-specific heuristics; and output a result of said analyze in regard to the at least a portion of the video including the detected one or more faces. ### Claim 33 (depends on 32) The non-transitory, computer-readable storage media of claim 32, wherein the face-scenario-specific heuristics comprise one or more quality indicators for one or more characteristic features of the video, and wherein to perform said analyze at least a portion of the video with the face-scenario-specific heuristics the media includes program instructions that are further executed to apply the quality indicators to corresponding features of the video to determine one or more image quality metric measurements. ### Claim 34 (depends on 32) The non-transitory, computer-readable storage media of claim 32, wherein the instructions are further executed to: repeat said evaluate, and analyze for a plurality of video images of the video to create a plurality of image quality metric measurements from a respective plurality of video images; determine, based at least in part on a comparison of at least one of the plurality of image quality metric measurements to a threshold, whether the image quality metric measurements indicate poor image quality; and in response to a determination that the image quality metric measurements exceed the threshold, output an image quality indicator for use in navigation to a particular image associated with the image quality metric. ### Claim 35 (depends on 32) The non-transitory, computer-readable storage media of claim 32, wherein the instructions are further executable to determine, based at least in part on application of the face-scenario-specific heuristics, that an image data pre-processor of a device is functioning correctly. ### Claim 36 (depends on 32) The non-transitory, computer-readable storage media of claim 32, wherein to perform said analyze at least a portion of the video with the face-scenario-specific heuristics the media store instructions that, when executed, cause the one or more processors to identify an area with chroma that falls within a human skin-tine chromaticity range. ### Claim 37 (depends on 32) The non-transitory, computer-readable storage media of claim 32, wherein to perform said analyze the video with the face-scenario-specific heuristics the media store instructions that, when executed, cause the one or more processors to perform: identify one or more facial features; and apply a face-scenario-specific heuristic in accordance with finding high contrasts and sharp detail in a facial area associated with the one or more facial features. ### Claim 38 (depends on 32) The non-transitory, computer-readable storage media of claim 32, wherein to perform said analyze the video with the face-scenario-specific heuristics the media store instructions that, when executed, cause the one or more processors to perform: identify one or more facial features of eyes, eyebrows, nose, lips, ears, or, eyeglasses; and apply a face-scenario-specific heuristic in accordance with finding high contrasts and sharp detail in an area of the identified one or more facial features. ### Claim 39 (depends on 32) The non-transitory, computer-readable storage media of claim 32, wherein the media store instructions that implement a scenario-based quality analyzer configured to perform said analyze the video based on the face-scenario-specific heuristics, and to perform one or more other scenario-specific heuristics of a plurality of scenario-specific heuristics in an attempt to match a predetermined scenario with the video. ### Claim 40 (depends on 32) The non-transitory, computer-readable storage media of claim 32, wherein to perform said output a result of said analyze in regard to the at least a portion of the video including the detected one or more faces, the media store program instructions that cause the one or more processors to perform instruction of an indication of image quality of the video as feedback displayed on a device display.