# US 11,307,718 B2 · Gesture-based small device input > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,718 B2 - Document ID: US11307718B2 - Status: Granted - Kind: B2 - Application number: 17/078,801 - Filed: 2020-10-23 - Published: 2021-02-11 - Granted: 2022-04-19 - Earliest claimed: 2008-10-24 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: James B. Miller; Richard C. Gossweiler, III - Web page: https://patentgrep.com/patent/US11307718B2 ## CPC - G06F 3/044: by capacitive means - G06F 1/163: {Wearable computers, e.g. on a belt} - G06F 1/169: {the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes (G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0418: {for error correction or compensation, e.g. based on parallax, calibration or alignment} - G06F 3/0421: {by interrupting or reflecting a light beam, e.g. optical touch-screen} - G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)} - G06F 3/0426: {tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886)} - G06F 3/04883: for inputting data by handwriting, e.g. gesture or text - G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus - G04G 21/04: using radio waves (radio-controlled time-pieces G04R) - G06F 2203/04101: 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G04G: ELECTRONIC TIME-PIECES ## Abstract A computer-implemented method is described that includes providing a watch defining a body and a watch face on a surface of the body, the body having one or more sensors arranged to sense user inputs in an area adjacent to the body. The method further comprises sensing a motion of an object in the area adjacent to, but not touching, the body using the one or more sensors and changing a display of a pointing element on a graphical user interface on the watch in coordination with the sensed motion. ## Claims ### Claim 1 A computer-implemented method, comprising: displaying, by a computing device including one or more sensors, an image on a surface of a user wearing the computing device, the image being displayed on the surface of the user at an area that is adjacent to the computing device; monitoring, by the computing device, the area adjacent to the computing device for movement of an object relative to the image; determining, by the computing device, that a movement of the object has occurred in the area adjacent to the computing device; and in response to determining that the movement of the object has occurred in the area adjacent to the computing device, performing an operation based at least in part on the movement and the image. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein: displaying, by the computing device, the image on the surface of the user comprises projecting the image onto the surface of the user in the area monitored by the computing device for movement, the image comprising a graphical user interface that includes a user-selectable element; and detecting the movement of the object in the area comprises identifying user interaction with the user-selectable element in the area adjacent to the computing device and the surface of the user. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, further comprising: identifying, by the computing device, a gesture corresponding to the movement of the object in the area that is adjacent to the computing device; and in response to identifying the gesture corresponding to the movement of the object, identifying the operation as corresponding to the gesture; wherein performing the operation is performed in response to identifying the operation as corresponding to the gesture. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, comprising: identifying a path traced by the movement of the object in the area that is adjacent to the computing device; and causing a pointing element displayed in a graphical user interface to move in coordination with the path traced by the object. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: determining, by the computing device, that a detection area on the surface of a body of the user has been contacted. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein determining, by the computing device, that the movement of the object has occurred in the area adjacent to the computing device comprises: detecting, by the computing device using the one or more sensors, a non-touch movement of the object in the area that is adjacent to the computing device. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising: illuminating, using one or more light sources included in the computing device, the area that is adjacent to the computing device; and monitoring, using the one or more sensors of the computing device, light reflected from the area; wherein determining, by the computing device, that the movement of the object has occurred comprises receiving information about the light reflected from the area. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein: the one or more light sources comprise one or more light emitting diodes; and the one or more sensors comprise one or more photodiodes. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, further comprising: identifying a plurality of applications on the computing device; detecting, using an accelerometer included in the computing device, motion of the computing device; in response to detecting the motion of the computing device, selecting one or more applications from a plurality of applications; and providing information to the one or more applications that indicates the motion of the computing device. ### Claim 10 A computing device, comprising: one or more sensors; one or more processors; and one or more non-transitory computer readable media that collectively store instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising: displaying an image on a surface of a user wearing the computing device, the image being displayed on the surface of the user at an area that is adjacent to the computing device; monitoring the area adjacent to the computing device for movement of an object relative to the image; determining that a movement of the object has occurred in the area adjacent to the computing device; and in response to determining that the movement of the object has occurred in the area adjacent to the computing device, performing an operation based at least in part on the movement and the image. ### Claim 11 (depends on 10) The computing device of claim 10, wherein: displaying, by the computing device, the image on the surface of the user comprises projecting the image onto the surface of the user in the area monitored by the computing device for movement, the image comprising a graphical user interface that includes a user-selectable element; and detecting the movement of the object in the area comprises identifying user interaction with the user-selectable element in the area adjacent to the computing device and the surface of the user. ### Claim 12 (depends on 10) The computing device of claim 10, wherein the operations further comprise: identifying a gesture corresponding to the movement of the object in the area that is adjacent to the computing device; and in response to identifying the gesture corresponding to the movement of the object, identifying the operation as corresponding to the gesture; wherein performing the operation is performed in response to identifying the operation as corresponding to the gesture. ### Claim 13 (depends on 10) The computing device of claim 10, wherein the operations further comprise: identifying a path traced by the movement of the object in the area that is adjacent to the computing device; and causing a pointing element displayed in a graphical user interface to move in coordination with the path traced by the object. ### Claim 14 (depends on 10) The computing device of claim 10, wherein the operations further comprise: illuminating, using one or more light sources included in the computing device, the area that is adjacent to the computing device; and monitoring, using the one or more sensors of the computing device, light reflected from the area; wherein determining that the movement of the object has occurred comprises receiving information about the light reflected from the area. ### Claim 15 (depends on 10) The computing device of claim 10, wherein the operations further comprise: identifying a plurality of applications on the computing device; detecting, using an accelerometer included in the computing device, motion of the computing device; in response to detecting the motion of the computing device, selecting one or more applications from a plurality of applications; and providing information to the one or more applications that indicates the motion of the computing device. ### Claim 16 One or more non-transitory computer readable media that collectively store instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising: displaying an image on a surface of a user wearing a computing device, the image being displayed on the surface of the user at an area that is adjacent to the computing device; monitoring the area adjacent to the computing device for movement of an object relative to the image; determining that a movement of the object has occurred in the area adjacent to the computing device; and in response to determining that the movement of the object has occurred in the area adjacent to the computing device, performing an operation based at least in part on the movement and the image. ### Claim 17 (depends on 16) The one or more non-transitory computer readable media of claim 16, wherein displaying the image on the surface of the user comprises: projecting the image onto the surface of the user in the area monitored by the computing device for movement, the image comprising a graphical user interface that includes a user-selectable element; and detecting the movement of the object in the area comprises identifying user interaction with the user-selectable element in the area adjacent to the computing device and the surface of the user. ### Claim 18 (depends on 16) The one or more non-transitory computer readable media of claim 16, wherein the operations further comprise: identifying a gesture corresponding to the movement of the object in the area that is adjacent to the computing device; and in response to identifying the gesture corresponding to the movement of the object, identifying the operation as corresponding to the gesture; wherein performing the operation is performed in response to identifying the operation as corresponding to the gesture. ### Claim 19 (depends on 16) The one or more non-transitory computer readable media of claim 16, wherein the operations further comprise: illuminating, using one or more light sources included in the computing device, the area that is adjacent to the computing device; and monitoring, using one or more sensors of the computing device, light reflected from the area; wherein determining that the movement of the object has occurred comprises receiving information about the light reflected from the area. ### Claim 20 (depends on 16) The one or more non-transitory computer readable media of claim 16, wherein the operations further comprise identifying a plurality of applications on the computing device; detecting, using an accelerometer included in the computing device, motion of the computing device; in response to detecting the motion of the computing device, selecting one or more applications from a plurality of applications; and providing information to the one or more applications that indicates the motion of the computing device.