# US 2012/0235901 A1 · System and method for control based on face or hand gesture detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2012/0235901 A1 - Document ID: US20120235901A1 - Status: Published application - Kind: A1 - Application number: 13/483,434 - Filed: 2012-05-30 - Published: 2012-09-20 - Earliest claimed: 2009-05-21 - Assignees: May Patents Ltd. - Inventors: Yehuda BINDER - Web page: https://patentgrep.com/patent/US20120235901A1 ## CPC - G01S 3/7864: {T.V. type tracking systems} - G06F 3/005: {Input arrangements through a video camera} - G06F 3/012: {Head tracking input arrangements} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06V 10/242: {by image rotation, e.g. by 90 degrees} - G06V 40/161: {Detection; Localisation; Normalisation} - G06V 40/166: {using acquisition arrangements} - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G09G 5/10: Intensity circuits - H04B 1/38: Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving - H04M 3/02: Calling substations, e.g. by ringing (selective calling H04Q) - H04N 5/63: Generation or supply of power specially adapted for television receivers - H04N 5/645: Mounting of picture tube on chassis or in housing - H04N 7/005: {using at least one opto-electrical conversion device} - H04N 7/015: High-definition television systems - H04N 7/0255: {Display systems therefor} - H04N 7/104: {Switchers or splitters} - H04N 7/163: {by receiver means only} - H04N 9/3141: {Constructional details thereof (details not peculiar to the presence of an electronic spatial light modulator G03B21/14)} - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)} - H04N 21/4131: {home appliance, e.g. lighting, air conditioning system, metering devices (home automation data switching networks exchanging configuration information on appliance services H04L12/2807)} - H04N 21/42204: {User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor (constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205)} - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/43615: {Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805)} - H04N 21/43632: {involving a wired protocol, e.g. IEEE 1394 (high-speed IEEE 1394 serial bus H04L12/40052)} - H04N 21/43635: {HDMI} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} - H04N 23/51: Housings - H04N 23/611: where the recognised objects include parts of the human body - H04W 88/02: Terminal devices - G09G 2330/027: Arrangements or methods related to powering off a display - G09G 2354/00: Aspects of interface with display user - H04N 21/42222: {Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device} - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - H04B: TRANSMISSION - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract System and method for control using face detection or hand gesture detection algorithms in a captured image. Based on the existence of a detected human face or a hand gesture in an image captured by a digital camera, a control signal is generated and provided to a device. The control may provide power or disconnect power supply to the device (or part of the device circuits). The location of the detected face in the image may be used to rotate a display screen to achieve a better line of sight with a viewing person. The difference between the location of the detected face and an optimum is the error to be corrected by rotating the display to the required angular position. A hand gesture detection can be used as a replacement to a remote control for the controlled unit, such as a television set. ## Claims ### Claim 1 A computer implemented method for improving the angular field of view for a person watching an image on a screen, the method comprising: (a) capturing an image of a scene in front of the screen; (b) converting the image into a digital data form; (c) detecting the position of an image of a human face in the captured image; (d) calculating at least one deviation between the detected position of the human face image in the captured image and an imaginary line that passes through the center of the image on the screen and extends substantially perpendicular to the screen; and (e) rotating the screen in a direction to reduce the calculated deviation. ### Claim 2 (depends on 1) The method according to claim 1, further comprising repeatedly performing steps (a) to (e) until the calculated deviation is smaller than a predefined value. ### Claim 3 (depends on 1) The method according to claim 1, wherein steps (a) to (e) are executed repeatedly to form a linear feedback control loop, wherein the calculated deviation serves as a control loop error, the control loop has a set point equal to zero, and the rotation of the display is controlled by the loop. ### Claim 4 (depends on 3) The method according to claim 3, wherein said method defines a linear proportional control loop, wherein the amount of rotation of the screen is proportional to the calculated deviation. ### Claim 5 (depends on 3) The method according to claim 3, wherein said method defines a PID (Proportional, Integral and Derivative) control loop, the amount of rotation of the screen is calculated based on proportional, integral and derivative computations of the calculated deviation. ### Claim 6 (depends on 1) The method according to claim 1, wherein: the deviation is horizontal, the deviation is calculated in the captured image, and the rotation of the display is in the horizontal direction. ### Claim 7 (depends on 1) The method according to claim 1, wherein the deviation is vertical, the deviation is calculated in the captured image, and the rotation of the display is in the vertical direction. ### Claim 8 (depends on 1) The method according to claim 1, wherein the at least one deviation includes a horizontal deviation and a vertical deviation, the horizontal and vertical deviations are calculated in the captured image, and said rotation of the display includes rotating in the horizontal direction in response to the horizontal deviation and rotating in the vertical direction in response to the vertical deviation. ### Claim 9 (depends on 1) The method according to claim 1, wherein if no human face image is detected is step (c), no rotation is performed. ### Claim 10 (depends on 1) The method according to claim 1, wherein if two or more human faces are detected in the captured image, then the average of the positions of the detected faces is calculated, and in step (d) the deviation is calculated between the average and the image center. ### Claim 11 An apparatus for improving the angular field of view for a person watching a display on a screen, said apparatus comprising: a digital camera for capturing an image in digital data form, said camera being fixed in position relative to the screen and oriented to capture an image of the scene substantially in front of the screen; an image processor coupled to receive the image in digital data form from said digital camera, for applying face detection algorithm to detect and locate an image of a human face in the captured image; and a motor mechanically attached to the screen for angularly rotating the screen; wherein said apparatus is operative to rotate said motor in response to the location of the detected image of a face in the captured image. ### Claim 12 (depends on 11) The apparatus according to claim 11, further comprising firmware or software and a controller to execute said firmware or software coupled between said digital camera and said motor for controlling rotation of the screen in response to the location of the detected image of a face in the captured image. ### Claim 13 (depends on 11) The apparatus according to claim 11, wherein said motor is a stepper motor. ### Claim 14 (depends on 11) The apparatus according to claim 11, further operative to calculate the deviation between the detected face image location and the image center, and wherein the angular rotation of the screen is based on the calculated deviation. ### Claim 15 (depends on 14) The apparatus according to claim 14, wherein no motor rotation is executed when the calculated deviation is smaller than a predefined value. ### Claim 16 (depends on 14) The apparatus according to claim 14, wherein said apparatus is operative to continuously rotate said motor in response to the location of the detected face image in the captured image in a linear feedback control loop, wherein the error used by the control loop is the calculated deviation, the set point of the control loop is zero and the angular rotation of the screen is the actuator controlled by the loop. ### Claim 17 (depends on 16) The apparatus according to claim 16, wherein the control loop is a linear proportional control loop, and the amount of angular rotation of the screen is proportional to the calculated deviation. ### Claim 18 (depends on 16) The apparatus according to claim 16, wherein the control loop is a PID (Proportional, Integral and Derivative) control loop, and the amount of angular rotation is calculated based on proportional, integral and derivative computations of the calculated deviation. ### Claim 19 (depends on 16) The apparatus according to claim 16, wherein the deviation is horizontal and is calculated in the captured image, and said motor is attached to effect rotation of the screen in a horizontal plane. ### Claim 20 (depends on 19) The apparatus according to claim 19, further comprising a second motor attached to effect rotation of the screen in a vertical plane, wherein a vertical deviation is calculated in the captured image, and wherein said second motor is attached to effect screen rotation in the vertical plane in response to the calculated vertical deviation. ### Claim 21 (depends on 16) The apparatus according to claim 16, wherein the deviation is vertical and is calculated in the captured image, and said motor is attached to effect rotation of the screen in the vertical plane. ### Claim 22 (depends on 11) The apparatus according to claim 11, wherein if no human face is detected by the image processor, no motor rotation is executed. ### Claim 23 (depends on 14) The apparatus according to claim 14, wherein if two or more human faces are detected in the captured image, then said image processor calculates the average of the positions of the detected face images and the deviation between that average and the image center. ### Claim 24 A computer implemented method for controlling a device based on face detection, the method comprising the steps of: (a) capturing an image of a scene; (b) converting the image into a digital data form; (c) detecting an image of a human face in the captured image; and (d) providing a first control signal associated with the detection of the human face image in the captured image. ### Claim 25 (depends on 24) The method according to claim 24, further comprising performing steps (a) to (d) repeatedly. ### Claim 26 (depends on 25) The method according to claim 25, further comprising waiting a pre-set period before repeating the steps (a) to (d). ### Claim 27 (depends on 24) The method according to claim 24, wherein step (d) comprises supplying power to the device is response to the detection of a human face image in the captured image. ### Claim 28 (depends on 24) The method according to claim 24, wherein step (d) comprises disconnecting power to the device is response to not detecting a human face image in the captured image. ### Claim 29 (depends on 24) The method according to claim 24, wherein said device comprises a display screen. ### Claim 30 (depends on 29) The method according to claim 29, wherein the image captured is of a scene substantially in front of the display screen. ### Claim 31 (depends on 29) The method according to claim 29, wherein in step (d) the display screen is blanked in response to not detecting a human face in said captured image. ### Claim 32 (depends on 24) The method according to claim 24, wherein in step (d) the control signal is generated in response to detecting a human face image in the captured image over a pre-defined period. ### Claim 33 (depends on 24) The method according to claim 24, wherein in step (d) the control signal is generated in response to not detecting a human face in said captured image over a pre-defined period. ### Claim 34 (depends on 24) The method according to claim 24, wherein the first control signal is generated in response to detecting a human face image in the captured image over a first pre-defined period, and said method further comprises generating a second control signal in response to not detecting a human face image in the captured image over a second pre-defined period. ### Claim 35 (depends on 34) The method according to claim 34, wherein the first control signal is operative to supply power to at least part of the device, and the second control signal is operative to disconnect power to at least part of the device. ### Claim 36 (depends on 35) The method according to claim 35, wherein the first control signal is operative to supply power to a selected part of the device, and the second control signal is operative to disconnect power to the selected part of the device. ### Claim 37 An apparatus for face detection based control of a device, said apparatus comprising: a digital camera for capturing an image in a digital data form; an image processor coupled to receive the image in a digital data form from said digital camera, for applying a face detection algorithm to detect the position of a human face image in the captured image; and a controller coupled to said image processor for generating a control signal in response to the detection or non-detection of a human face image in the captured image. ### Claim 38 (depends on 37) The apparatus according to claim 37, further comprising firmware or software executed by said controller. ### Claim 39 (depends on 37) The apparatus according to claim 37, wherein said camera is mechanically fixed to the device. ### Claim 40 (depends on 37) The apparatus according to claim 37, wherein said camera, said image processor and said controller are housed within a single enclosure. ### Claim 41 (depends on 37) The apparatus according to claim 37, further comprising a switch connected to be actuated by the control signal. ### Claim 42 (depends on 41) The apparatus according to claim 41, wherein said switch is connected between a power source and the device, for powering the device in response to the control signal. ### Claim 43 (depends on 42) The apparatus according to claim 42, operative to actuate said switch and to supply power to the device is response to the detection of a human face image in the captured image. ### Claim 44 (depends on 42) The apparatus according to claim 42, operative to actuate said switch and for supplying power to the device is response to not detecting a human face in said captured image. ### Claim 45 (depends on 41) The apparatus according to claim 41, wherein said camera, said image processor, said controller and said switch are housed within a single enclosure. ### Claim 46 (depends on 37) The apparatus according to claim 37, further comprising a first timer for signaling a pre-set first time period, said first timer being operatively associated with said controller to cause the control signal to be generated after elapse of the first time period following detecting a human face in said captured image for a first pre-defined period. ### Claim 47 (depends on 46) The apparatus according to claim 46, further comprising a second timer for signaling a pre-set second time period, said second timer being operatively associated with said controller to cause said controller to generate a second control signal in response to not detecting a human face image in the captured image for the second time period. ### Claim 48 (depends on 47) The apparatus according to claim 47, wherein the first-recited control signal is effective to initiate supplying of power to the device, and the second control signal is effective to disconnect power from the device. ### Claim 49 (depends on 48) The method according to claim 48 wherein the first-recited control signal is effective to initiate supplying of power to part of the device, and the second control signal is effective to disconnect power from the part of the device. ### Claim 50 (depends on 37) The apparatus according to claim 37, further comprising a first timer for signaling a pre-set first period coupled or within said controller, and the control signal is generated in response to not detecting a human face in said captured image for the first pre-defined period. ### Claim 51 (depends on 37) The apparatus according to claim 37, wherein said device is a display or a television set. ### Claim 52 (depends on 37) The apparatus according to claim 37, wherein the device has a display screen and said camera is positioned such that the image captured is substantially in front of the display screen. ### Claim 53 (depends on 37) The apparatus according to claim 37, wherein the device has a display, and said apparatus further comprises a timer associated with said controller for signaling a pre-set time period, and said controller is operative to blank the display in response to not detecting a human face image in the captured image.