# US 2014/0157209 A1 · System and method for detecting gestures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0157209 A1 - Document ID: US20140157209A1 - Status: Published application - Kind: A1 - Application number: 13/796,772 - Filed: 2013-03-12 - Published: 2014-06-05 - Earliest claimed: 2012-12-03 - Assignees: Google Inc. - Applicants: Google Inc. - Inventors: Navneet Dalal; Mehul Nariyawala; Ankit Mohan; Varun Gulshan - Web page: https://patentgrep.com/patent/US20140157209A1 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/012: {Head tracking input arrangements} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06V 10/143: Sensing or illuminating at different wavelengths - G06V 30/242: Division of the character sequences into groups prior to recognition; Selection of dictionaries - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06F 2203/011: Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A system and method that includes detecting an application change within a multi-application operating framework; updating an application hierarchy model for gesture-to-action responses with the detected application change; detecting a gesture; according to the hierarchy model, mapping the detected gesture to an action of an application; and triggering the action. ## Claims ### Claim 1 A method comprising: detecting an application change within a multi-application operating system; updating an application hierarchy model for gesture-to-action responses with the detected application change; detecting a gesture; according to the hierarchy model, mapping the detected gesture to an action of an application; and triggering the mapped action of the application. ### Claim 2 (depends on 1) The method of claim 1, wherein detecting an application change comprises detecting a selection of a new top-level application within the operating system; and updating an application-gesture priority queue comprises promoting the new top-level application in the hierarchy model. ### Claim 3 (depends on 2) The method of claim 2, further comprising signaling a change in the application-gesture priority queue upon updating the application-gesture priority queue. ### Claim 4 (depends on 1) The method of claim 1, wherein detecting application change comprises detecting a change of context within an active application. ### Claim 5 (depends on 4) The method of claim 4, wherein detecting a change of context comprises detecting the loading of a media object in an active application. ### Claim 6 (depends on 1) The method of claim 1, wherein mapping the detected gesture to an action of an application comprises: if a gesture is not actionable within an initial application in the hierarchy model, progressively checking gesture-to-action responses of an application in a lower hierarchy of the hierarchy model. ### Claim 7 (depends on 1) The method of claim 1, wherein detecting a gesture comprises limiting gesture detection processing to a subset of gestures defined by gestures in the application hierarchy model. ### Claim 8 (depends on 1) The method of claim 1, further comprising detecting user settings according to facial recognition of a user; and if an active application is set as a preferred application in the detected user settings, promoting the preferred application in the application hierarchy model. ### Claim 9 (depends on 1) The method of claim 1, wherein a gesture is actionable by at least two applications in the hierarchy model; and wherein mapping the detected gesture to an action of an application comprises selecting the action of the application with the highest priority in the hierarchy model. ### Claim 10 (depends on 9) The method of claim 9, wherein at least one gesture in the set of gestures is defined by a thumbs up gesture heuristic that is used for at least voting, approval, and confirming for a first, second, and third application respectively. ### Claim 11 (depends on 1) The method of claim 1, wherein detecting a gesture comprises detecting a gesture presence from a set of gestures characterized in the gesture-to-action responses in the hierarchy model. ### Claim 12 (depends on 11) The method of claim 11, further comprising for at least one gesture in the set of gestures, subsequently initiating the action for at least a second time if a prolonged presence of the at least one gesture is detected. ### Claim 13 (depends on 11) The method of claim 11, wherein for at least one gesture in the set of gestures, initiating a modified form of the action if a translation of the at least one gesture is detected. ### Claim 14 (depends on 13) The method of claim 13, wherein at least one gesture in the set of gestures is defined by a pinch gesture heuristic; wherein the gesture-to-action response is a scrolling response if the detected gesture is a pinch gesture heuristic and a translation of the pinch gesture along an axis scrolls is detected. ### Claim 15 (depends on 11) The method of claim 11, wherein at least a first gesture in the set of gestures is defined by a thumbs up gesture heuristic, at least a second gesture in the set of gestures is defined by a mute gesture heuristic, and at least a third gesture in the set of gestures is defined by an extended sideways thumb gesture heuristic. ### Claim 16 (depends on 15) The method of claim 15, wherein at least a fourth gesture in the set of gestures is defined by a palm up gesture heuristic and at least a fifth gesture in the set of gestures is defined by a palm down heuristic. ### Claim 17 A method comprising: detecting an application change within a multi-application operating framework; updating an application hierarchy model for gesture-to-action responses with the detected application change; detecting a gesture from a set of presence-based gestures; mapping the detected gesture to a gesture-to-action response in the hierarchy model, wherein if a gesture-to-action response is not identified within an initial application in the hierarchy model, progressively checking gesture-to-action responses of an application in a lower hierarchy of the hierarchy model; if the detected gesture is a first gesture, detecting translation of the first gesture; if the detected gesture is a second gesture, detecting rotation of the first gesture; if the detected gesture is a third gesture, detecting prolonged presence of the third gesture; and triggering the action, wherein the action is modified according to a modified action if defined for translation, prolonged presence or rotation. ### Claim 18 (depends on 17) The method of claim 17, wherein at least one gesture in the set of presence-based gestures is defined by a thumbs up gesture heuristic, at least a one gesture in the set of gestures is defined by a mute gesture heuristic, and at least one gesture in the set of gestures is defined by an extended sideways thumb gesture heuristic. ### Claim 19 (depends on 17) The method of claim 17, wherein at least one gesture in the set of presence-based gestures is defined by a palm up gesture heuristic, and at least a one gesture in the set of gestures is defined by a palm down gesture heuristic.