# US 10,310,619 B2 · User gesture recognition > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,310,619 B2 - Document ID: US10310619B2 - Status: Granted - Kind: B2 - Application number: 15/128,074 - Filed: 2015-03-05 - Published: 2017-03-30 - Granted: 2019-06-04 - Earliest claimed: 2014-03-21 - Assignees: Artnolens SA - Applicants: Artnolens SA - Inventors: Olivier Raulot - Web page: https://patentgrep.com/patent/US10310619B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/0485: Scrolling or panning - G06F 3/005: {Input arrangements through a video camera} - G06F 2203/04803: Split screen, i.e. subdividing the display area or the window area into separate subareas - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and device for gesture recognition, wherein the gesture is executed by a user in a gesture region which may be defined relative to a display surface. In an embodiment, the gesture comprises a select gesture and the device comprises at least three cameras operating in the visual range where a first camera is used to determine a horizontal location of the select gesture and the other cameras are used to determine a vertical location thereof. A device for providing input to a computing device comprises a rectangular display having a viewing surface and at least three cameras having respective fields of view. A first camera and a second camera are located at respective adjacent corners of the display and a third camera is located at an edge of the display between the first and second cameras. ## Claims ### Claim 1 A method of providing input to a computing device, the computing device comprising a processor, memory, a display having a viewing surface and at least three cameras having respective fields of view, the method comprising the steps of: capturing corresponding images with the cameras; identifying input gestures using the captured images, wherein the input gestures are spaced from the viewing surface of the display, and wherein said identifying input gestures includes processing each image by designating a region of the image as a gesture region and determining changes in the gesture region, wherein the gesture regions are spaced from the viewing surface of the display; and interpreting the identified input gestures as input to the computing device; wherein at least one of the cameras is orientated such that the respective field of view lies adjacent to the viewing surface of the display; wherein said identifying input gestures in the captured images comprises first choosing between images captured by either a first camera or a second camera of the at least three cameras and then using the chosen images to identify the input gestures, wherein said identifying input gestures in the captured images is further based on images captured by a third camera of the at least three cameras, wherein said choosing between images captured by either the first camera or the second camera comprises comparing image information between images captured by the first camera and images captured by the second camera, and wherein said comparing image information comprises comparing a standard deviation of information contained in an image captured by the first camera to a standard deviation of information contained in an image captured by the second camera, wherein each of said at least three cameras operates to capture only two-dimensional image information. ### Claim 2 (depends on 1) The method according to claim 1 wherein a determination of a first dimension for use in identifying input gestures is based on images captured by the third camera. ### Claim 3 (depends on 2) The method according to claim 2 wherein an input gesture is recognized by determining a change in images captured by the first camera at different times and calculating a threshold based on the change. ### Claim 4 (depends on 2) The method according to claim 2 where a determination of a second dimension for use in identifying input gestures is based on images captured by the first camera or the second camera. ### Claim 5 (depends on 4) The method according to claim 4 wherein an input gesture is performed by a user's appendage and wherein the step of determining the second dimension of the input gesture comprises recognizing the user's appendage. ### Claim 6 (depends on 5) The method according to claim 5 wherein the user's appendage is recognized according to a predetermined range of hue, saturation and value of pixels in images captured by the second camera. ### Claim 7 (depends on 4) The method according to claim 4, further comprising establishing a mapping between a co-ordinate system of the first camera and/or the second camera and a co-ordinate system of the viewing surface. ### Claim 8 (depends on 7) The method according to claim 7 wherein the mapping comprises a linear mapping. ### Claim 9 (depends on 8) The method according to claim 8 wherein the mapping comprises a first linear mapping applied to a first section of the co-ordinates of the viewing surface and a second linear mapping applied to a second section of the viewing surface. ### Claim 10 (depends on 1) The method according to claim 1, further comprising tracking an input gesture and determining a slide action in dependence on detected motion of the input gesture. ### Claim 11 (depends on 1) The method according to claim 1 wherein all of the cameras are situated near an edge of the display and orientated with intersecting fields of view. ### Claim 12 (depends on 1) The method according to claim 1 wherein the computing device further comprises a fourth camera which is positioned adjacent the third camera and wherein a determination of a first dimension comprises choosing between images captured by the third camera and images captured by the fourth camera. ### Claim 13 A device for providing input to a computing device, the computing device comprising a processor and memory, the device comprising: a rectangular display having a viewing surface; and at least three cameras having respective fields of view, each of the cameras capturing information in two spatial dimensions only, wherein a first camera and a second camera of the at least three cameras are located at respective adjacent corners of the display, and a third camera of the at least three cameras is located at an edge of the display between the first and second cameras, wherein each camera is orientated to capture image information of a gesture region spaced from the viewing surface of the display to thereby recognize gestures in the gesture region from said image information, wherein the processor is configured to recognized the gestures in the gesture region by first choosing between images captured by either the first camera or the second camera of the at least three cameras and then using the chosen images together with images captured by the third camera of the at least three cameras to recognize the gestures, wherein said choosing between images captured by either the first camera or the second camera comprises comparing image information between images captured by the first camera and images captured by the second camera, wherein said comparing image information comprises comparing a standard deviation of information contained in an image captured by the first camera to a standard deviation of information contained in an image captured by the second camera, and wherein each of said first, second and third cameras operates to capture only two-dimensional image information. ### Claim 14 (depends on 13) The device according to claim 13, further comprising a fourth camera located at an edge of the display between the first and second cameras. ### Claim 15 (depends on 14) The device according to claim 14, further comprising a mounting frame located at a periphery of the display, wherein at least said first and second camera are mounted to the mounting frame. ### Claim 16 (depends on 15) The device according to claim 15 wherein each camera is mounted to the mounting frame. ### Claim 17 (depends on 14) The device according to claim 14 wherein at least one of the cameras is orientated such that the respective field of view lies adjacent to the viewing surface of the display. ### Claim 18 (depends on 14) The device according to claim 14 wherein the input comprises an input gesture and wherein the memory stores instructions which, when processed by the processor, identify input gestures using the captured images, wherein identifying input gestures in the captured images comprises choosing between images captured by either the first camera or the second camera and further using the chosen image to identify input gestures. ### Claim 19 (depends on 18) The device according to claim 18 wherein identifying input gestures in the captured images is further based on images captured by the third camera. ### Claim 20 (depends on 18) The device according to claim 18 wherein choosing between images captured by either the first camera or the second camera comprises comparing information between images captured by the first camera and images captured by the second camera. ### Claim 21 (depends on 20) The device according to claim 20 wherein the input gesture is performed by a user's appendage and wherein the instructions further comprise instructions for recognizing the user's appendage. ### Claim 22 (depends on 21) The device according to claim 21 wherein the user's appendage is recognized according to a predetermined range of hue, saturation and value of pixels in images captured by one or more of the cameras. ### Claim 23 (depends on 18) The device according to claim 18 wherein the instructions, when processed by the processor, determine a first dimension for use in identifying input gestures based on images captured by the third camera and determine a second dimension for use in identifying input gestures based on images captured by the first camera or the second camera, wherein the input gesture is performed by a user's appendage, wherein the step of determining the second dimension of the input gesture comprises recognizing the user's appendage, and wherein the user's appendage is recognized according to a predetermined range of hue, saturation and value of pixels in images captured by the second camera. ### Claim 24 (depends on 14) The device according to claim 14 wherein at least one of the cameras is orientated so that the respective field of view lies adjacent to the viewing surface of the display. ### Claim 25 (depends on 14) The device according to claim 14 wherein all of the cameras are situated near an edge of the display and orientated with intersecting fields of view. ### Claim 26 (depends on 18) The device according to claim 18 wherein the instructions, when processed by the processor, process each image by designating a region of the image as a gesture region and determining changes in the gesture region. ### Claim 27 (depends on 26) The device according to claim 26 wherein the instructions, when processed by the processor, further define respective gesture regions adjacent to the viewing surface of the display. ### Claim 28 (depends on 26) The device according to claim 26 wherein the instructions, when processed by the processor, further define respective gesture regions spaced from the viewing surface of the display.