# US 9,606,629 B2 · Systems and methods for gesture interaction with cloud-based applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,606,629 B2 - Document ID: US9606629B2 - Status: Granted - Kind: B2 - Application number: 13/608,971 - Filed: 2012-09-10 - Published: 2013-05-23 - Granted: 2017-03-28 - Earliest claimed: 2011-09-09 - Assignees: Cloudon Ltd. - Applicants: Shuki Binyamin; Jay Zaveri; Assaf Kamil; Tzahi Glik; Doron Yaacoby; Meir Morgenstern - Inventors: Shuki Binyamin; Jay Zaveri; Assaf Kamil; Tzahi Glik; Doron Yaacoby; Meir Morgenstern - Web page: https://patentgrep.com/patent/US9606629B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Various systems and methods described herein relate to server-based computing, where the systems and methods provide a client with access to an application executing remotely from the client device and having access to data (e.g., one or more files) residing on a cloud-based storage (e.g., provided by a third-party cloud-based storage service, such as Dropbox or Box). For some systems and methods, the application may be remotely executed and provided to the client such that the application has in-application/embedded access (hereafter, referred to as “native access”) to the cloud-based storage and files residing on the cloud-based storage. Additionally, some systems and methods may enable a remotely executed application to receive a gesture user input, even when the remotely executed application is not natively configured to receive such user input. ## Claims ### Claim 1 A system comprising: a virtualization module configured to execute a first virtual instance of a first application based on a version of the first application in a virtual computing environment on a first cloud-based server, the virtual computing environment being generated by a virtual machine on the first cloud-based server, an interface associated with the first virtual instance of the first application being presented at a first client device in network communication with the first cloud-based server; a gesture input module configured to receive, from the first client device, gesture data about a gesture input signal received at the first client device, the gesture input signal being recognizable by software on the first client device and received from a user of the first client device, the gesture input signal being associated with a request for a desired function affecting performance or behavior of the first virtual instance of the first application; a gesture mapping module configured to map the gesture data to application control data that is recognizable by the first application, the application control data operable to initiate the desired function affecting performance or behavior of the first virtual instance of the first application, wherein to map the gesture data to application control data comprises translating gesture data that is not native to first application to an alternative data that are compatible with the first application; and a gesture emulation module configured to use the application control data to initiate the desired function affecting performance or behavior of the first virtual instance of the first application and configured in response to execute, in the virtual computing environment, a set of one or more events for causing execution of the desired function affecting performance or behavior of the first virtual instance of the first application, thereby enabling the first instance of the application to respond to non-native input gestures received by the first client device and otherwise not recognizable by the first application. ### Claim 2 (depends on 1) The system of claim 1, wherein executing the set of input events in the virtual computing environment comprises identifying a graphical user interface element of the first virtual instance associated with at least one input event in the set of input events and executing the at least one input event with respect to the graphical user interface element. ### Claim 3 (depends on 2) The system of claim 2, wherein executing the set of input events in the virtual computing environment further comprises moving an input cursor to the graphical user interface element before executing the at least one input event with respect to the graphical user interface element. ### Claim 4 (depends on 1) The system of claim 1, wherein the gesture data comprises a gesture type, a gesture phase, gesture position, gesture velocity, an input pressure, or a gesture timestamp. ### Claim 5 (depends on 1) The system of claim 1, wherein the first gesture input comprises a finger-dragging gesture, a pinch gesture, a swipe gesture, a tap gesture, or a flicking gesture. ### Claim 6 (depends on 1) The system of claim 1, wherein the set of input events comprises a mouse-based input event, a keyboard-based input event, a gesture-based input event, an audio-based input event, or a video input event. ### Claim 7 (depends on 1) The system of claim 1, wherein the set of input events comprises a vertical scroll event, a horizontal scroll event, or a scroll-control-click event. ### Claim 8 (depends on 1) The system of claim 1, wherein the set of input events comprises a zoom command, a pan command, or a rotate command. ### Claim 9 (depends on 1) The system of claim 1, wherein executing the set of input events in the virtual computing environment comprises generating an event handling message operable in the virtual computing environment. ### Claim 10 (depends on 1) The system of claim 1, wherein mapping the first gesture input to the second input is based on a movement physics model. ### Claim 11 (depends on 10) The system of claim 10, wherein the movement physics model comprises a flick model or a polynomial movement model. ### Claim 12 (depends on 1) The system of claim 1, further comprising a storage module configured to establish access to a cloud-based storage provided by a second cloud-based server over a network, the cloud-based storage comprising a file, and the first virtualized instance having access to the file on the cloud-based storage. ### Claim 13 A method comprising: executing a first virtual instance of a first application based on a version of the first application in a virtual computing environment on a first cloud-based server, the virtual computing environment being generated by a virtual machine on the first cloud-based server, an interface associated with the first virtual instance of the first application presented at a first client device in network communication with the first cloud-based server; receiving, from the first client device, gesture data about a gesture input signal received at the first client device, gesture input signal being recognizable by software on the first client device and received from a user of the first client device to request for a desired function affecting performance or behavior of the first virtual instance of the first application; mapping the gesture data to application control data that is recognizable by the first application, the application control data operable to initiate the desired function affecting performance or behavior of the first virtual instance of the first application, wherein mapping the gesture data to application control data comprises translating gesture data that is not native to first application to an alternative data that are compatible with the first application; and executing, in the virtual computing environment, using the application control data to initiate the desired function affecting performance or behavior of the first virtual instance of the first application and configured in response to, a set of one or more events for causing execution of the desired function affecting performance or behavior of the first virtual instance of the first application, thereby enabling the first instance of the application to respond to non-native input gestures received by the first client device and otherwise not recognizable by the first application. ### Claim 14 (depends on 13) The method of claim 13, wherein executing the set of input events comprises issuing a user input message in the virtual computing environment. ### Claim 15 (depends on 13) The method of claim 13, wherein executing the set of input events in the virtual computing environment comprises identifying a graphical user interface element of the first virtual instance of the first application, the graphical user interface element associated with at least one input event in the set of input events, and executing the at least one input event with respect to the graphical user interface element. ### Claim 16 (depends on 15) The method of claim 15, wherein executing the set of input events in the virtual computing environment further comprises moving an input cursor to the graphical user interface element before executing the at least one input event with respect to the graphical user interface element. ### Claim 17 (depends on 13) The method of claim 13, wherein the gesture data comprises a gesture type, a gesture phase, gesture position, gesture velocity, an input pressure, or a gesture timestamp. ### Claim 18 (depends on 13) The method of claim 13, wherein the first gesture input comprises a finger-dragging gesture, a pinch gesture, a swipe gesture, a tap gesture, or a flicking gesture. ### Claim 19 (depends on 13) The method of claim 13, wherein the set of input events comprises a mouse-based input event, a keyboard-based input event, a gesture-based input event, an audio-based input event, or a video input event. ### Claim 20 (depends on 14) The method of claim 14, wherein the set of input events comprises a vertical scroll event, a horizontal scroll event, or a scroll-control-click event. ### Claim 21 (depends on 13) The method of claim 13, wherein the set of input events comprises a zoom command, a pan command, or a rotate command. ### Claim 22 (depends on 13) The method of claim 13, wherein executing the set of input events in the virtual computing environment comprises generating an event handling message operable in the virtual computing environment. ### Claim 23 (depends on 13) The method of claim 13, wherein mapping the first gesture input to the second input is based on a movement physics model. ### Claim 24 (depends on 23) The method of claim 23, wherein the movement physics model comprises a flick model or a polynomial movement model. ### Claim 25 (depends on 13) The method of claim 13, further comprising establishing access to a cloud-based storage provided by a second cloud-based server over a network, the cloud-based storage comprising a file, and the first virtual instance having access to the file on the cloud-based storage. ### Claim 26 A non-transitory computer readable medium comprising executable instructions, the instructions executable by a processor to perform a method, the method comprising: executing a first virtual instance of a first application based on a version of the first application in a virtual computing environment on a first cloud-based server, the virtual computing environment being generated by a virtual machine on the first cloud-based server, an interface associated with the first virtual instance of the first application presented at a first client device in network communication with the first cloud-based server; receiving, from the first client device, gesture data about a gesture input signal received at the first client device, gesture input signal being recognizable by software on the first client device and received from a user of the first client device to request for a desired function affecting performance or behavior of the first virtual instance of the first application; mapping the gesture data to application control data that is recognizable by the first application, the application control data operable to initiate the desired function affecting performance or behavior of the first virtual instance of the first application, wherein mapping the gesture data to application control data comprises translating gesture data that is not native to first application to an alternative data that are compatible with the first application; and executing, in the virtual computing environment, using the application control data to initiate the desired function affecting performance or behavior of the first virtual instance of the first application and configured in response to, a set of one or more events for causing execution of the desired function affecting performance or behavior of the first virtual instance of the first application, thereby enabling the first instance of the application to respond to non-native input gestures received by the first client device and otherwise not recognizable by the first application. ### Claim 27 A system comprising: means for executing a first virtual instance of a first application based on a version of the first application in a virtual computing environment on a first cloud-based server, the virtual computing environment being generated by a virtual machine on the first cloud-based server, an interface associated with the first virtual instance of the first application presented at a first client device in network communication with the first cloud-based server; means for receiving, from the first client device, gesture data about a gesture input signal received at the first client device, gesture input signal being recognizable by software on the first client device and received from a user of the first client device to request for a desired function affecting performance or behavior of the first virtual instance of the first application; means for mapping the gesture data to application control data that is recognizable by the first application, the application control data operable to initiate the desired function affecting performance or behavior of the first virtual instance of the first application, wherein mapping the gesture data to application control data comprises translating gesture data that is not native to first application to an alternative data that are compatible with the first application; and means for executing, in the virtual computing environment, using the application control data to initiate the desired function affecting performance or behavior of the first virtual instance of the first application and configured in response to, a set of one or more events for causing execution of the desired function affecting performance or behavior of the first virtual instance of the first application, thereby enabling the first instance of the application to respond to non-native input gestures received by the first client device and otherwise not recognizable by the first application.