# US 10,503,968 B2 · Identifying a local coordinate system for gesture recognition > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,503,968 B2 - Document ID: US10503968B2 - Status: Granted - Kind: B2 - Application number: 15/076,882 - Filed: 2016-03-22 - Published: 2017-09-28 - Granted: 2019-12-10 - Earliest claimed: 2016-03-22 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Alon Lerner; Maoz Madmony - Web page: https://patentgrep.com/patent/US10503968B2 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 9/52: Program synchronisation; Mutual exclusion, e.g. by means of semaphores - G06F 18/22: Matching criteria, e.g. proximity measures - G06V 10/32: Normalisation of the pattern dimensions - G06V 10/42: Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Identifying a local coordinate system is described for gesture recognition. In one example, a method includes receiving a gesture from a user across a horizontal axis at a depth camera, determining a horizontal vector for the user based on the received user gesture, determining a vertical vector; and determining a rotation matrix to convert positions of user gestures received by the camera to a frame of reference of the user. ## Claims ### Claim 1 A method for aligning a user's coordinate system with a depth camera's coordinate system to facilitate gesture recognition, the method comprising: requesting a user to perform one or more gestures, the request to determine the user's coordinate system; receiving a horizontal swipe gesture from the user at the depth camera, in response to the request, the received horizontal swipe gesture having a path defined by a leftmost endpoint and a rightmost endpoint; determining a horizontal vector based on the leftmost endpoint and the rightmost endpoint of the received horizontal swipe gesture, wherein the horizontal vector is the horizontal axis of the user's coordinate system; determining a vertical vector, wherein the vertical vector is the vertical axis of the user's coordinate system; and determining a rotation matrix relative to the coordinate system of the depth camera, based on the horizontal vector and the vertical vector, to convert positions of user gestures received by the depth camera to a frame of reference of the user, thereby aligning the user's coordinate system with the depth camera's coordinate system. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving an additional user gesture at the depth camera; rotating positions of the additional user gesture to positions of the frame of reference of the user; and recognizing the additional user gesture using the frame of reference of the user. ### Claim 3 (depends on 2) The method of claim 2, further comprising executing a command corresponding to the additional user gesture. ### Claim 4 (depends on 2) The method of claim 2, wherein the horizontal vector is a first horizontal vector, and wherein the method further comprises: determining a second horizontal vector for the user based on the additional user gesture; comparing the second horizontal vector to the first horizontal vector; and if the second horizontal vector differs from the first horizontal vector by more than a threshold, then determining a new rotation matrix using the second horizontal vector. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the vertical vector comprises determining a local direction of gravity using a sensor associated with the depth camera. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the vertical vector comprises: recognizing an object in the field of view of the depth camera; and determining the vertical vector based on the orientation of the recognized object, the object being different from the user. ### Claim 7 (depends on 1) The method of claim 1, wherein determining the vertical vector comprises: receiving, by the depth camera, a vertical swipe gesture from the user at the camera, in response to the request, the received vertical swipe gesture a path defined by a bottom endpoint and a top endpoint; and determining the vertical vector based on the bottom endpoint and the top endpoint. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the horizontal vector comprises: defining a vector based on the leftmost and rightmost end points; and normalizing the vector, thereby providing the horizontal vector. ### Claim 9 (depends on 1) The method of claim 1, wherein the horizontal swipe gesture is a hand gesture. ### Claim 10 (depends on 1) The method of claim 1, wherein determining the rotation matrix comprises: determining an orthonormal coordinate system defined by the horizontal vector, the vertical vector, and a third vector that is orthogonal to one or both the horizontal vector and the vertical vector; normalizing the horizontal, vertical, and third vectors; and generating a matrix by taking the normalized vectors as columns in a 3×3 matrix as the rotation matrix. ### Claim 11 (depends on 10) The method of claim 10, further comprising using one of the leftmost endpoint or the rightmost endpoint as the origin of the orthonormal coordinate system. ### Claim 12 A non-transitory computer-readable medium having instructions thereon that when executed by one or more processors cause a process to be carried out for aligning a user's coordinate system with a depth camera's coordinate system to facilitate gesture recognition, the process comprising: receiving, at a depth camera, a gesture from a user across a horizontal axis, wherein the received gesture has a starting point on the horizontal axis and an ending point on the horizontal axis; determining a horizontal vector for the user based on the received user gesture, wherein the horizontal vector represents the horizontal axis of the user's coordinate system; determining a vertical vector, wherein the vertical vector represents the vertical axis of the user's coordinate system; and determining a rotation matrix relative to the coordinate system of the depth camera, based on the horizontal vector and the vertical vector, to convert positions of user gestures received by the depth camera to a frame of reference of the user, thereby aligning the user's coordinate system with the depth camera's coordinate system. ### Claim 13 (depends on 12) The medium of claim 12, the process further comprising: determining an orthonormal coordinate system defined by the horizontal vector, the vertical vector and a third vector that is orthogonal to one or both the horizontal vector and the vertical vector; and using the orthonormal coordinate system to determine the rotation matrix. ### Claim 14 (depends on 13) The medium of claim 13, wherein determining the rotation matrix comprises: normalizing the horizontal vector, the vertical vector and the third vector; and generating a matrix by taking the normalized vectors as columns in a 3×3 matrix as the rotation matrix. ### Claim 15 (depends on 14) The medium of claim 14, wherein determining an orthonormal coordinate system includes using the starting point as the origin of the orthonormal coordinate system. ### Claim 16 An apparatus for gesture recognition, the apparatus comprising: a depth camera to receive a gesture from a user across a horizontal axis, the depth camera having a first coordinate system, and the user having a second coordinate system; a hand tracking module to determine a horizontal vector for the user based on the received user gesture and to determine a vertical vector, wherein the horizontal vector represents the horizontal axis of the user's coordinate system, and the vertical vector represents the vertical axis of the user's coordinate system; and a coordinate system transformation module to determine a rotation matrix relative to the coordinate system of the depth camera, based on the horizontal vector and the vertical vector, to convert positions of user gestures received by the depth camera to a frame of reference of the user, thereby aligning the user's coordinate system with the depth camera's coordinate system. ### Claim 17 (depends on 16) The apparatus of claim 16, further comprising: a gesture recognition module to recognize user gestures using the frame of reference of the user, wherein the hand tracking module determines the vertical vector by determining a local direction of gravity using a sensor associated with the camera. ### Claim 18 (depends on 16) The apparatus of claim 16, wherein the hand tracking module determines a horizontal vector by determining end positions of a path defined by the gesture, defining a vector based on the end positions, and normalizing the vector. ### Claim 19 (depends on 16) The apparatus of claim 16, wherein the received gesture has a starting point on a horizontal axis and an end point on the horizontal axis, and wherein the hand tracking module determines the horizontal vector using the starting point as the origin of the vector on the horizontal axis and the end point as a magnitude and a direction. ### Claim 20 (depends on 1) The method of claim 1, wherein the depth camera's coordinate system is defined by a horizontal axis and a vertical axis of the depth camera, and by a direction in which the depth camera is pointing.