# US 2013/0258113 A1 · System and method for control based on face or hand gesture detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0258113 A1
- Document ID: US20130258113A1
- Status: Published application
- Kind: A1
- Application number: 13/903,569
- Filed: 2013-05-28
- Published: 2013-10-03
- Earliest claimed: 2009-05-21
- Assignees: May Patents Ltd.
- Applicants: May Patents Ltd.
- Inventors: Yehuda BINDER
- Web page: https://patentgrep.com/patent/US20130258113A1

## CPC

- G01S 3/7864: {T.V. type tracking systems}
- G06F 3/005: {Input arrangements through a video camera}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06V 10/242: {by image rotation, e.g. by 90 degrees}
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/166: {using acquisition arrangements}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G09G 5/10: Intensity circuits
- H04B 1/38: Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving
- H04M 3/02: Calling substations, e.g. by ringing (selective calling H04Q)
- H04N 5/63: Generation or supply of power specially adapted for television receivers
- H04N 5/645: Mounting of picture tube on chassis or in housing
- H04N 7/005: {using at least one opto-electrical conversion device}
- H04N 7/015: High-definition television systems
- H04N 7/0255: {Display systems therefor}
- H04N 7/104: {Switchers or splitters}
- H04N 7/163: {by receiver means only}
- H04N 9/3141: {Constructional details thereof (details not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)}
- H04N 21/4131: {home appliance, e.g. lighting, air conditioning system, metering devices (home automation data switching networks exchanging configuration information on appliance services H04L12/2807)}
- H04N 21/42204: {User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor (constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205)}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/43615: {Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805)}
- H04N 21/43632: {involving a wired protocol, e.g. IEEE 1394 (high-speed IEEE 1394 serial bus H04L12/40052)}
- H04N 21/43635: {HDMI}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 23/51: Housings
- H04N 23/611: where the recognised objects include parts of the human body
- H04W 88/02: Terminal devices
- G09G 2330/027: Arrangements or methods related to powering off a display
- G09G 2354/00: Aspects of interface with display user
- H04N 21/42222: {Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device}
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04B: TRANSMISSION
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

System and method for control using face detection or hand gesture detection algorithms in a captured image. Based on the existence of a detected human face or a hand gesture in an image captured by a digital camera, a control signal is generated and provided to a device. The control may provide power or disconnect power supply to the device (or part of the device circuits). The location of the detected face in the image may be used to rotate a display screen to achieve a better line of sight with a viewing person. The difference between the location of the detected face and an optimum is the error to be corrected by rotating the display to the required angular position. A hand gesture detection can be used as a replacement to a remote control for the controlled unit, such as a television set.

## Claims

### Claim 1

A television set responsive to an element in an image, the television set comprising:
a flat screen for displaying television channels;
a digital camera for capturing an image, said digital camera having an output port, and being operative to transmit a digital data form of the captured image via said output port, said digital camera being fixed in position relative to said screen to be movable in unison with said screen and being oriented to capture an image of a scene substantially in front of the screen;
an image processor coupled to receive the image in the digital data from said digital camera and for applying an algorithm to detect the element in the captured image;
an antenna for over-the-air radio-frequency communication;
a wireless transceiver coupled between said digital camera output port at d said antenna for transmitting the digital data form of the captured image over the air; and
a single enclosure housing said flat screen, said digital camera, said image processor, is said antenna and said wireless transceiver, wherein information is displayed on said flat screen in response to the detection of the element in the captured image.

### Claim 2 (depends on 1)

The television set according to claim 1, wherein said digital camera comprises:
an optical lens for focusing received light, a photosensitive image sensor array disposed approximately at an image focal point plane of the optical lens for capturing the image and producing an analog form of the captured image; and
an analog-to-digital (A/D) converter coupled to said image sensor for generating the digital data form of the captured image.

### Claim 3 (depends on 2)

The television set according to claim 2, wherein said image sensor array is based on multiple Charge-Coupled Devices (CCD) or Complementary Metal-Oxide-Semiconductor (CMOS) based components.

### Claim 4 (depends on 1)

The television set according to claim 1, wherein the element is an image of a human face, and said image processor applies a face detection algorithm to detect an existence and a. position of the human face in the captured image.

### Claim 5 (depends on 1)

The television set according to claim 1, wherein the element is an image of a human hand, and said image processor applies a hand gesture algorithm to detect an existence and a position of a hand gesture in the captured image.

### Claim 6 (depends on 5)

The television set according to claim 5, wherein said image processor further applies a face detection algorithm to detect an existence and a position of a human face in the captured image.

### Claim 7 (depends on 6)

The television set according to claim 6, wherein the detection algorithm of the hand gesture is based on the position of the human face detected in the captured image.

### Claim 8 (depends on 5)

The television set according to claim 5, wherein the hand gesture includes extending a single finger.

### Claim 9 (depends on 5)

The television set according to claim 5, wherein the hand gesture includes extending multiple fingers.

### Claim 10 (depends on 5)

The television set according to claim 5, wherein the hand gesture includes extending all fingers of one hand.

### Claim 11 (depends on 1)

The television set according to claim 1, further comprising a timer associated with a preset time period, and wherein said screen is blanked in response to not detecting the element in the captured image during the pre-set time period.

### Claim 12

The television set according to claim II, wherein the information is displayed on said screen in response to a detected position of the detected element in the captured image.

### Claim 13 (depends on 1)

The television set according to claim 1, further responsive to a deviation between the position of the detected element in the captured image and a center of the image.

### Claim 14 (depends on 1)

The television set according to claim 1, further connected fur displaying multiple television channels, and wherein a television channel is selected to be displayed on said screen in response to the detection of the element in the captured image.

### Claim 15 (depends on 1)

The television set according to claim 1, wherein said image processor comprises firmware or software and a processor to execute said firmware or software.

### Claim 16 (depends on 1)

The television set according to claim 1, wherein a power is supplied to a load in response to the detection of the element in the captured image.

### Claim 17 (depends on 16)

The television set according to claim 16, further comprising a switch connected to be actuated in response to the detection of the element in the captured image.

### Claim 18 (depends on 1)

The television set according to claim 1, wherein the over-the-air radio frequency communication uses a license-free frequency band.

### Claim 19 (depends on 18)

The television set according to claim 18, wherein the license-free frequency band is an ISM band.

### Claim 20 (depends on 19)

The television set according to claim 19, wherein the ISM band is 5 GHz or 2.4 GHz.

### Claim 21 (depends on 1)

The television set according to claim 1, wherein: the over-the-air radio frequency communication is over a Wireless Personal Area Network (WPAN); said antenna is a WPAN antenna; and said wireless transceiver is a WPAN transceiver.

### Claim 22 (depends on 21)

(Withdrawn, Currently Amended) The television set according to claim 21, wherein the WPAN is according to, or based on, Bluetooth™, IEEE 802.15, Ultra-Wide-Band (UWB), or ZigBee™.

### Claim 23 (depends on 1)

The television set according to claim 1, wherein: the over-the-air radio frequency communication is over a Wireless Local Area Network (WLAN); said antenna is a WLAN antenna; and said wireless transceiver is a WLAN transceiver.

### Claim 24 (depends on 23)

The television set according to claim 23, wherein the WLAN is according to, or based on, IEEE 802.11a, IEEE 802.11b, or IEEE 802.11g.

### Claim 25 (depends on 1)

The television set according to claim 1, wherein: the over-the-air radio frequency communication is over a Wireless Wide Area Network (WWAN); said antenna is a WWAN antenna; and said wireless transceiver is a WWAN transceiver.

### Claim 26 (depends on 1)

The television set according to claim 1, wherein: the over-the-air radio frequency communication is a cellular communication; said antenna is a cellular antenna; and said wireless transceiver is a cellular transceiver.

### Claim 27 (depends on 26)

The television set according to claim 26, wherein the cellular communication is according to, or based on, GSM (Global Television set for Mobile Communications), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), EDGE (Enhanced Data Rates for GSM Evolution), 3GSM, DECT (Digital Enhanced Cordless Telecommunications), Digital AMPS or iDEN (Integrated Digital Enhanced Network).

### Claim 28 (depends on 1)

The television set according to claim 1, wherein: the over-the-air radio frequency communication uses a 60 GHz license-free frequency band and is used for in-room communication.

### Claim 29 (depends on 28)

The television set according to claim 28, wherein the communication is based on beamforming and supports a data rate of above 7 Gb/s.

### Claim 30 (depends on 29)

The television set according to claim 29, wherein the communication is according to, or based on, WirelessHD™.

### Claim 31 (depends on 1)

The television set according to claim 1, further comprising an image or video compressor coupled between said digital camera output port and said wireless transceiver, for compressing the captured image digital data according to a compression scheme.

### Claim 32 (depends on 31)

The television set according to claim 31, wherein the compression scheme is a lossy or lossless type.

### Claim 33 (depends on 311)

The television set according to claim 311, wherein the compression scheme is according to, or based on, JPEG (Joint Photographic Experts Group) and MPEG (Moving Picture Experts Group) standard.

### Claim 34 (depends on 1)

The television set according to claim 1, wherein said flat screen is silicon-based.

### Claim 35 (depends on 34)

The television set according to claim 34 wherein said flat screen is LCD (Liquid Crystal Display) or TFT (Thin-Film Transistor) based.

### Claim 36 (depends on 1)

The television set according to claim 1, further operative for displaying High Definition (HD), and wherein said television set further comprises an HDMI (High-Definition Multimedia Interface) for receiving and displaying HD video.

### Claim 37

A television set operative to receive and display television channels on a screen, the television set being further operative for detecting first and second elements at respective locations in an image and responding to the locations of the first and second elements in the image, the television set comprising a single enclosure, and, in said single enclosure:
a digital camera for capturing the image in a digital data form;
an image processor coupled to receive the image from said digital camera and execute an element detection algorithm to detect the first and second elements in the captured image and to determine the locations of the elements in the image; and
a second processor, and firmware or software executable by said second processor, said second processor being coupled to control said image processor and said digital camera and being operative for generating a control signal in response to the locations of the first and second elements.

### Claim 38 (depends on 37)

The television set according to claim 37, wherein said image processor is further operative to calculate the average location of the first and second elements, and wherein the television set is operative for generating the control signal in response to the center location of the locations of the first and second elements.

### Claim 39 (depends on 37)

The television set according to claim 37, wherein the image is a non-visible image.

### Claim 40 (depends on 39)

The television set according to claim 39, wherein the non-visible image is in the infrared or ultraviolet spectrum.

### Claim 41 (depends on 37)

The television set according to claim 37, wherein the elements are a body parts.

### Claim 42 (depends on 41)

The television set according to claim 41, wherein the body parts are human hands or human faces.

### Claim 43 (depends on 37)

The television set according to claim 37, wherein the elements are hand gestures.

### Claim 44 (depends on 43)

The television set according to claim 43, wherein the hand gestures include extending a. single finger, extending multiple fingers, or extending all fingers of one hand.

### Claim 45 (depends on 37)

The television set according to claim 37, further comprising a switch coupled to be actuated by the control signal.

### Claim 46 (depends on 45)

The television set according to claim 45, wherein said switch is connected between a power source and an apparatus, for powering the apparatus in response to the control signal.

### Claim 47 (depends on 46)

The television set according to claim 46, wherein the apparatus is external to said single enclosure.

### Claim 48 (depends on 37)

The television set according to claim 37, wherein said image processor consists of or comprises, a non-transitory computer readable medium storing an image processing algorithm for detecting the elements in the captured image.

### Claim 49 (depends on 37)

The television set according to claim 37, further comprising a communication port coupled to said second processor for coupling to a network medium; and a transceiver coupled to said communication port for transmitting the control signal to the network medium.

### Claim 50 (depends on 49)

The television set according to claim 49, wherein said transceiver is further coupled to said digital camera for transmitting the captured image over the network medium.

### Claim 51 (depends on 37)

The television set according to claim 37, wherein said digital camera comprises:
an optical lens for focusing received light, said lens being mechanically oriented to guide the image;
a photosensitive image sensor array disposed at least approximately at an image focal point plane of the optical lens for capturing the image and producing an analog signal representing the image; and
an analog-to-digital (A/D) converter coupled to said image sensor array for converting the analog signal to a digital data representation of the image.

### Claim 52 (depends on 51)

The television set according to claim 51, wherein said image sensor array is based on Charge-Coupled Devices (CCD) or Complementary Metal-Oxide-Semiconductor (CMOS) components.
