# US 2012/0249768 A1 · System and method for control based on face or hand gesture detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0249768 A1
- Document ID: US20120249768A1
- Status: Published application
- Kind: A1
- Application number: 13/526,053
- Filed: 2012-06-18
- Published: 2012-10-04
- Earliest claimed: 2009-05-21
- Assignees: May Patents Ltd.
- Inventors: Yehuda BINDER
- Web page: https://patentgrep.com/patent/US20120249768A1

## CPC

- G01S 3/7864: {T.V. type tracking systems}
- G06F 3/005: {Input arrangements through a video camera}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06V 10/242: {by image rotation, e.g. by 90 degrees}
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/166: {using acquisition arrangements}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G09G 5/10: Intensity circuits
- H04B 1/38: Transceivers, i.e. devices in which transmitter and receiver form a structural unit and in which at least one part is used for functions of transmitting and receiving
- H04M 3/02: Calling substations, e.g. by ringing (selective calling H04Q)
- H04N 5/63: Generation or supply of power specially adapted for television receivers
- H04N 5/645: Mounting of picture tube on chassis or in housing
- H04N 7/005: {using at least one opto-electrical conversion device}
- H04N 7/015: High-definition television systems
- H04N 7/0255: {Display systems therefor}
- H04N 7/104: {Switchers or splitters}
- H04N 7/163: {by receiver means only}
- H04N 9/3141: {Constructional details thereof (details not peculiar to the presence of an electronic spatial light modulator G03B21/14)}
- H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards
- H04N 21/4122: {additional display device, e.g. video projector (digital output for controlling a plurality of local displays G06F3/1423)}
- H04N 21/4131: {home appliance, e.g. lighting, air conditioning system, metering devices (home automation data switching networks exchanging configuration information on appliance services H04L12/2807)}
- H04N 21/42204: {User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor (constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205)}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/43615: {Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805)}
- H04N 21/43632: {involving a wired protocol, e.g. IEEE 1394 (high-speed IEEE 1394 serial bus H04L12/40052)}
- H04N 21/43635: {HDMI}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 23/51: Housings
- H04N 23/611: where the recognised objects include parts of the human body
- H04W 88/02: Terminal devices
- G09G 2330/027: Arrangements or methods related to powering off a display
- G09G 2354/00: Aspects of interface with display user
- H04N 21/42222: {Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device}
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04B: TRANSMISSION
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

System and method for control using face detection or hand gesture detection algorithms in a captured image. Based on the existence of a detected human face or a hand gesture in an image captured by a digital camera, a control signal is generated and provided to a device. The control may provide power or disconnect power supply to the device (or part of the device circuits). The location of the detected face in the image may be used to rotate a display screen to achieve a better line of sight with a viewing person. The difference between the location of the detected face and an optimum is the error to be corrected by rotating the display to the required angular position. A hand gesture detection can be used as a replacement to a remote control for the controlled unit, such as a television set.

## Claims

### Claim 1

A television set for being controlled by an element in an image, the television set comprising:
a flat screen for displaying television channels;
a digital camera for capturing an image, said digital camera having an output port, and being operative to transmit a digital data form of the captured image via said output port, said digital camera being fixed in position relative to said screen and being oriented to capture an image of a scene substantially in front of the screen;
an image processor coupled to receive the image in digital data form from said digital camera and for applying an algorithm to detect the element in the captured image;
a communication port for coupling to a network medium; and
a transceiver coupled between said digital camera output port and said communication port for transmitting the digital data form of the captured image over the network medium, wherein information is displayed on said screen in response to the detection of the element in the captured image.

### Claim 2 (depends on 1)

The television set according to claim 1, wherein said digital camera comprises:
an optical lens for focusing received light, a photosensitive image sensor array disposed approximately at an image focal point plane of the optical lens for capturing the image and producing electronic image information representing the image; and
an analog-to-digital (A/D) converter coupled to said image sensor for generating a digital data representation of the image.

### Claim 3 (depends on 2)

The television set according to claim 2 wherein said image sensor array is based on Charge-Coupled Devices (CCD) or Complementary Metal-Oxide-Semiconductor (CMOS) devices.

### Claim 4 (depends on 1)

The television set according to claim 1, wherein the image element is an image of a human face, and said image processor applies a face detection algorithm to detect the existence and the position of the human face image in the captured image.

### Claim 5 (depends on 1)

The television set according to claim 1, wherein said element is an image of a human hand, and said image processor applies a hand gesture algorithm to detect the existence and the position of a hand gesture in the captured image.

### Claim 6 (depends on 5)

The television set according to claim 5, wherein said image processor further applies a face detection algorithm to detect the existence and the position of a human face image in the captured image.

### Claim 7 (depends on 6)

The television set according to claim 6, wherein the detection of the hand gesture is based on the position of a human face detected in the captured image.

### Claim 8 (depends on 5)

The television set according to claim 5, wherein the hand gesture includes extending a single finger.

### Claim 9 (depends on 5)

The television set according to claim 5, wherein the hand gesture includes extending multiple fingers.

### Claim 10 (depends on 5)

The television set according to claim 5, wherein the hand gesture includes extending all fingers of one hand.

### Claim 11 (depends on 1)

The television set according to claim 1, further comprising a timer associated with a pre-set time period, and wherein said screen is blanked in response to not detecting said element in the captured image during said pre-set time period.

### Claim 12 (depends on 1)

The television set according to claim 1, wherein said digital camera, said image processor, said communication port, and said transceiver are housed within a television set single enclosure.

### Claim 13 (depends on 1)

The television set according to claim 1, wherein the information is displayed on said screen in response to the detected position of the element in the captured image.

### Claim 14 (depends on 1)

The television set according to claim 1, further responsive to a deviation between the detected position of the element in the captured image and the center of the image.

### Claim 15 (depends on 1)

The television set according to claim 1, further connected for displaying multiple television channels, and wherein a television channel is selected to be displayed on said screen in response to the detection of the element in the captured image.

### Claim 16 (depends on 1)

The television set according to claim 1, wherein said image processor comprises firmware or software and a processor to execute said firmware or software.

### Claim 17 (depends on 1)

The television set according to claim 1, wherein power is supplied to a load in response to the detection of the element in the captured image.

### Claim 18 (depends on 17)

The television set according to claim 17, further comprising a switch connected to be actuated in response to the detection of the element in the captured image.

### Claim 19 (depends on 1)

The television set according to claim 1, wherein said communication port is an antenna for over-the-air radio frequency communication, and wherein said transceiver is a wireless transceiver.

### Claim 20 (depends on 19)

The television set according to claim 19, wherein said over-the-air radio frequency communication uses a license-free frequency band.

### Claim 21 (depends on 20)

The television set according to claim 20, wherein the license-free frequency band is an ISM band.

### Claim 22 (depends on 21)

The television set according to claim 21, wherein the ISM band is 5 GHz or 2.4 GHz.

### Claim 23 (depends on 19)

The television set according to claim 19, wherein: the over-the-air radio frequency communication is communication over a Wireless Personal Area Network (WPAN); said antenna is a WPAN antenna; and said wireless transceiver is a WPAN transceiver.

### Claim 24 (depends on 23)

The television set according to claim 23, wherein the WPAN is according to, or based on, Bluetooth, IEEE 802.15, Ultra-Wide-Band (UWB), or ZigBee.

### Claim 25 (depends on 19)

The television set according to claim 19, wherein: the over-the-air radio frequency communication is communication over a Wireless Local Area Network (WLAN); said antenna is a WLAN antenna; and said wireless transceiver is a WLAN transceiver.

### Claim 26 (depends on 25)

The television set according to claim 25, wherein the WLAN is according to, or based on, IEEE 802.11a, IEEE802.11b, or IEEE802.11g.

### Claim 27 (depends on 19)

The television set according to claim 19, wherein: the over-the-air radio frequency communication is communication over a Wireless Wide Area Network (WWAN); said antenna is a WWAN antenna; and said wireless transceiver is a WWAN transceiver.

### Claim 28 (depends on 19)

The television set according to claim 19 wherein: the over-the-air radio frequency communication is a cellular communication; said antenna is a cellular antenna; and said wireless transceiver is a cellular transceiver.

### Claim 29 (depends on 28)

The television set according to claim 28, wherein the cellular communication is according to, or based on, GSM (Global Television set for Mobile Communications), GPRS (General Packet Radio Service), CDMA (Code Division Multiple Access), EDGE (Enhanced Data Rates for GSM Evolution), 3GSM, DECT (Digital Enhanced Cordless Telecommunications), Digital AMPS or iDEN (Integrated Digital Enhanced Network).

### Claim 30 (depends on 19)

The television set according to claim 19, wherein: the over-the-air radio frequency communication uses a 60 GHz license-free frequency band and is used for in-room communication.

### Claim 31 (depends on 30)

The television set according to claim 30, wherein said in-building network is based on beamforming and supports a data rate of above 7 Gb/s.

### Claim 32 (depends on 31)

The television set according to claim 31, wherein said in-building network is according to, or based on, WirelessHD™.

### Claim 33 (depends on 1)

The television set according to claim 1, wherein communication over the network medium is based on visible or non-visible light communication.

### Claim 34 (depends on 1)

The television set according to claim 1, wherein: the network medium is a cable; said communication port is a connector for connecting to the cable; and said transceiver is a wired transceiver.

### Claim 35 (depends on 34)

The television set according to claim 34, wherein: the cable is a Local Area Network (LAN) cable; said connector is a LAN connector; and said wired transceiver is a LAN transceiver.

### Claim 36 (depends on 35)

The television set according to claim 35, wherein the network is an Ethernet based LAN, and is according to, or based on, IEEE 802.3-2008 standard.

### Claim 37 (depends on 36)

The television set according to claim 36, wherein: the LAN cable is based on twisted-pair copper cables; said LAN transceiver is according to, or based on, 10Base-T, 100Base-TX, or 1000Base-T; and said LAN connector is RJ-45 type.

### Claim 38 (depends on 34)

The television set according to claim 34, wherein the cable is connectable to simultaneously carry a DC or AC power signal.

### Claim 39 (depends on 38)

The television set according to claim 38, further operative to supply at least in part of the power signal.

### Claim 40 (depends on 38)

The television set according to claim 38, further operative to at least in part be powered from the power signal.

### Claim 41 (depends on 38)

The television set according to claim 38, wherein: the power signal is carried over dedicated wires in the cable, and the wires are distinct from wires in the cable carrying the digital data form of the captured image

### Claim 42 (depends on 38)

The television set according to claim 38, wherein: the power signal and the digital data form of the captured image are carried over the same wires in the cable; said television set further comprises a power/data splitter arrangement having first, second and third ports; only the digital data form of the captured image signal is passed between said first and second ports; only the power signal is passed between said first and third ports; and said first port is coupled to said connector.

### Claim 43 (depends on 42)

The television set according to claim 42, wherein the power and digital data signals are carried using Frequency Division/Domain Multiplexing (FDM), where the digital data signal is carried in a frequency band above and distinct from the power signal frequency or frequency band.

### Claim 44 (depends on 43)

The television set according to claim 43, wherein said power/data splitter comprises a high pass filter between said first and second ports and a low pass filter between said first and third ports.

### Claim 45 (depends on 44)

The television set according to claim 44, wherein said power/data splitter comprising a transformer and a capacitor connected to the transformer windings.

### Claim 46 (depends on 42)

The television set according to claim 42, wherein the power and digital data signals are carried using a phantom scheme.

### Claim 47 (depends on 46)

The television set according to claim 46, wherein said power/data splitter comprises at least two transformers each having a center-tap connection.

### Claim 48 (depends on 42)

The television set according to claim 42, wherein said power and digital data signals are carried substantially according to IEEE 802.3af-2003 or IEEE 802.3at-2009 standard.

### Claim 49 (depends on 1)

The television set according to claim 1, further comprising an image or video compressor coupled between said digital camera output port and said transceiver, for compressing the captured image according to a compression scheme.

### Claim 50 (depends on 49)

The television set according to claim 49, wherein the compression scheme is lossy or lossless type.

### Claim 51 (depends on 49)

The television set according to claim 49, wherein the compression scheme is according to, or based on, JPEG (Joint Photographic Experts Group) and MPEG (Moving Picture Experts Group) standard.

### Claim 52 (depends on 1)

The television set according to claim 1, wherein said flat screen is silicon-based.

### Claim 53 (depends on 52)

The television set according to claim 52 wherein said flat screen is LCD (Liquid Crystal Display) or TFT (Thin-Film Transistor) based.

### Claim 54 (depends on 1)

The television set according to claim 1, further operative for displaying High Definition (HD), and wherein said television set further comprises an HDMI (High-Definition Multimedia Interface) for receiving and displaying HD video.
