# US 2014/0193030 A1 · Gesture pre-processing of video stream with hold-off period to reduce platform power > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0193030 A1 - Document ID: US20140193030A1 - Status: Published application - Kind: A1 - Application number: 13/737,413 - Filed: 2013-01-09 - Published: 2014-07-10 - Earliest claimed: 2013-01-09 - Assignees: Intel Corporation - Inventors: Jeremy Burr - Web page: https://patentgrep.com/patent/US20140193030A1 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06V 10/94: Hardware or software architectures specially adapted for image or video understanding - G06V 10/96: Management of image or video recognition tasks - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Techniques are disclosed for processing a video stream to reduce platform power by employing a stepped and distributed pipeline process, wherein CPU-intensive processing is selectively performed. In one example case, the techniques are implemented in a user's computer system wherein initial threshold detection (image disturbance) and optionally user presence (e.g., hand image) processing components are proximate to or within the system's camera, and the camera is located in or proximate to the system's primary display. The threshold detection and/or target presence stages can be selectively disabled for a hold-off period. The hold-off period may be, for example, in the range of 50 to 1000 mSec and triggered in response to an indication that a user of the system is unlikely to be making navigational gestures or that the system is not ready to process video, thereby conserving power by avoiding processing of video frames free of navigation gestures. ## Claims ### Claim 1 A system for processing a raw video stream, comprising: a threshold detection stage configured to determine if a current frame of the video stream is sufficiently different from a previous frame of the video stream, and in response to the current frame not being sufficiently different than the previous frame, the threshold detection stage is further configured to stop further processing of the current frame by a subsequent stage; and a target presence stage configured to determine if the current frame includes skin tone colored pixels, and in response to the current frame not including skin tone colored pixels, the target presence stage is further configured to stop further processing of the current frame by a subsequent stage; wherein at least one of the threshold detection and/or target presence stages can be selectively disabled for a hold-off period. ### Claim 2 (depends on 1) The system of claim 1 wherein the threshold detection stage is configured to sum and compare a luma channel of the current frame with a luma channel of the previous frame, and if the delta between the current and previous frames is above a given threshold, then data of the current frame is passed to the target presence stage for further processing. ### Claim 3 (depends on 1) The system of claim 1 wherein the target presence stage is configured to convert data of the current frame into one or more HSV pixels, and if a sufficient number of skin tone colored pixels are found, then data of the current frame is passed to a next processing stage. ### Claim 4 (depends on 3) The system of claim 3 wherein the next processing stage is a segmentation stage configured to separate pixels of interest from pixels that are not of interest, wherein pixels of interest are represented as blobs and pixels not of interest are represented by non-blobs. ### Claim 5 (depends on 4) The system of claim 4 wherein the segmentation stage is configured to at least one of eliminate disperse pixels due to camera noise and/or lighting, and/or process imaged data using erode and/or dilate morphing methods. ### Claim 6 (depends on 4) The system of claim 4 wherein the segmentation stage is implemented in a graphics processing unit (GPU). ### Claim 7 (depends on 4) The system of claim 4 further comprising: a parser stage that is configured to perform spatial interpretation of blob data received from the segmentation stage. ### Claim 8 (depends on 7) The system of claim 7 wherein the parser stage performs spatial interpretation by mapping the blob data to a model of target human anatomical geometry. ### Claim 9 (depends on 8) The system of claim 8 wherein the target human anatomical geometry is a hand. ### Claim 10 (depends on 7) The system of claim 7 further comprising: a gesture recognition stage that is configured to perform temporal interpretation of blob data received from the parser stage. ### Claim 11 (depends on 10) The system of claim 10 wherein at least one of the segmentation, parser, and/or gesture recognition stages are configured to send feedback to at least one of the threshold detection and/or target presence stages. ### Claim 12 (depends on 1) A imaging device comprising the system of claim 1 and a camera operatively coupled to the system. ### Claim 13 (depends on 12) The imaging device of claim 12 wherein at least one of the threshold detection stage and the target presence stage are implemented within sensor circuitry of the camera. ### Claim 14 (depends on 1) A display comprising the system of claim 1. ### Claim 15 (depends on 1) The system of claim 1 wherein the hold-off period is in the range of 50 to 1000 mSec, and is triggered in response to an indication that a user of the system is unlikely to be making navigational gestures, or that the system is not ready to process video. ### Claim 16 A system for processing a raw video stream, comprising: a threshold detection stage configured to determine if a current frame of the video stream is sufficiently different from a previous frame of the video stream, and in response to the current frame not being sufficiently different than the previous frame, the threshold detection stage is further configured to stop further processing of the current frame by a subsequent stage; a target presence stage configured to determine if the current frame includes skin tone colored pixels, and in response to the current frame not including skin tone colored pixels, the target presence stage is further configured to stop further processing of the current frame by a subsequent stage; a segmentation stage configured to receive data from the current frame, and separate pixels of interest from pixels that are not of interest, wherein pixels of interest are represented as blobs and pixels not of interest are represented by non-blobs; a hand parser stage that is configured to perform spatial interpretation of blob data received from the segmentation stage; and a gesture recognition stage that is configured to perform temporal interpretation of blob data received from the hand parser stage and to identify user hand-based navigation gestures; wherein the threshold detection, target presence, segmentation, parser, and gesture recognition stages are configured in a stepped and distributed arrangement, and at least one of the threshold detection and/or target presence stages can be selectively disabled for a hold-off period. ### Claim 17 (depends on 16) The system of claim 16 wherein at least one of the threshold detection and target presence stages are implemented within or proximate to a camera deployable at a first location and having a field of view that includes a user input area, and the segmentation, parser and gesture recognition stages are at one or more locations different from the first location. ### Claim 18 (depends on 17) The system of claim 17 wherein the threshold detection and target presence stages are implemented within a display. ### Claim 19 (depends on 16) The system of claim 16 wherein the segmentation stage is implemented in a graphics processing unit, and the parser and gesture recognition stages are implemented by application software. ### Claim 20 (depends on 16) The system of claim 16 further comprising an application programming interface (API) for operatively coupling one or more of the stages to a computing platform in which the system is configured to operate. ### Claim 21 (depends on 20) The system of claim 20 wherein the API is presented as an extended universal serial bus human interface (USB HID) class device. ### Claim 22 (depends on 16) The system of claim 16 wherein the hold-off period is in the range of 50 to 1000 mSec, and is triggered in response to an indication that a user of the system is unlikely to be making navigational gestures, or that the system is not ready to process video. ### Claim 23 (depends on 16) A media processing system comprising the system of claim 16. ### Claim 24 (depends on 16) A mobile computing system comprising the system of claim 16. ### Claim 25 A computer navigation method using a raw video stream, comprising: determining, at a first processing stage, if a current frame of the video stream is sufficiently different from a previous frame of the video stream, and in response to the current frame not being sufficiently different than the previous frame, stopping further processing of the current frame by a subsequent stage; and determining, at a second processing stage, if the current frame includes skin tone colored pixels, and in response to the current frame not including skin tone colored pixels, stopping further processing of the current frame by a subsequent stage; wherein at least one of the first and second processing stages can be selectively disabled for a hold-off period, the hold-off period being in the range of 50 to 1000 mSec and triggered in response to an indication that no navigational gestures are being made, thereby conserving power by avoiding processing of video frames free of navigation gestures.