US 10,592,012 B2 · Five-rectangle method for dispatching touch events from motion-disabled users

Abstract

A method using hardware button(s) dispatches gestures for motion-disabled users. It displays a translucent layer, on top of underlying applications, which intercepts user touch events (preventing inadvertent gestures) and modifies (x′,y′) based on dampened differences of most recent intercepted event coordinates (swabs). The method operates in three modes identified by translucent colors and/or digits (how many button pushes before invocation): mode 1 dispatches short taps at (x′,y′); mode 2 displays extending arrow while the user swabs (x′,y′) until the second push, which dispatches scrolling in arrow direction; mode 3 creates an arrow and pivot. Pivot size and color indicate magnification/demagnification. The third push dispatches zooming when pivot and arrow are collinear; otherwise it dispatches rotation. A special case of mode 2 dispatches a long tap. In one embodiment, dispatched gestures are allowed through a momentary hole centered at (x′,y′) surrounded by four other rectangles in the translucent layer.

Drawings

Drawings are temporarily unavailable. Open the PDF at USPTO

Drawings

Claims21

Dependent claims

A method for dispatching gestures on behalf of a motion-disabled user assisted by an accessory with at least a first button for a touchscreen device having a pixel display and visible coordinate range w by h pixels, wherein w defines the width of the pixel display and h defines the height of the pixel display and having a processor running an underlying application and running an extra software layer, said method carried out by said extra software layer including the steps of:

initializing coordinates values (x′,y′), (swipeX,swipeY), (zoomX,zoomY) and a mode value;
displaying on said touchscreen a translucent layer that provides a visual indicator of the most recent values of said (x′,y′) and of said mode value on each occasion said values change;
capturing motion events from said user by said translucent layer, whereby said capturing of said motion events can be enabled and disabled, thereby preventing said motion events from reaching said underlying application when said capturing is enabled, and thereby allowing at least some of said motion events to reach said underlying application when said capturing is disabled;
adjusting said (x′,y′) by the product of a constant and a vector difference between coordinates of the two most recent of said motion events obtained by said step of capturing;
changing said mode value, thereby yielding a new mode value, in response to a request from said user;
selecting one of the following steps when said user pushes said first button:
performing a first gesture composed from said (x′,y′) when said mode value equals 1;
saving said (x′,y′) as said (swipeX ,swipeY) when said mode value equals 2 and said swipeX equals −1;
performing a second gesture composed from said (x′,y′) and said (swipeX,swipeY) when said mode value equals 2 and said swipeX does not equal −1;
saving said (x′,y′) as said (zoomX,zoomY) when said mode value equals 3 and said zoomX equals −1;
saving said (x′,y′) as said (swipeX,swipeY) when said mode value equals 3 and said zoomX does not equal −1 and said swipeX equals −1;
performing a third gesture composed from said (x′,y′), said (swipeX,swipeY) and said (zoomX,zoomY) when said mode value equals 3 and said zoomX does not equal −1 and said swipeX does not equal −1.

9 more elements

initializing coordinate values:
(x′,y′), (swipeX,swipeY), (zoomX,zoomY), and distance values:
sx, sy, and mode value;
computing, based on said (x′,y′), said sx, said sy, said h and said w, updated values for top as max(y′−sy,0), left as max(x′−sx,0), bottom as min(y′+sy,h), right as min(x′+sx,w), top2s as max(y′+sy,0), bottom2s as min(y′−sy,h) and right2s as min(x′−sy,w);
displaying on said touchscreen a translucent layer, which provides a clear rectangular visual indicator, proportional in size to sx and sy centered on the most recent value of said (x′,y′), said step of displaying said translucent layer including the steps of:
displaying a digit near said (x′,y′), wherein said digit corresponds to the integer value of said mode value;
displaying an arrow from said (swipeX,swipeY) to said (x′,y′) when said mode value equals 2 and said swipeX does not equal −1;
displaying an arrow from said (zoomX,zoomY) to said (x′,y′) when said mode value equals 3 and said zoomX does not equal −1 and said swipeX equals −1;
displaying a pivot symbol and an arrow from said (zoomX,zoomY) to said (swipeX,swipeY) when said mode value equals 3 and said zoomX does not equal −1 and said swipeX does not equal −1, wherein the size of said pivot symbol is a logarithmic function involving said (x′,y′), said (zoomX,zoomY) and said (swipeX,swipeY);
displaying a first translucent rectangle with corners (0,0) and (left2s,top);
displaying a second translucent rectangle with corners (0,bottom2s) and (left,h);
displaying a third translucent rectangle with corners (right2s,bottom) and (w,h);
and displaying a fourth translucent rectangle with corners (right,0) and (w,top2s);
capturing motion events from said user by said translucent layer, whereby said capturing of said motion events can be enabled and disabled, thereby preventing said motion events from reaching said underlying application when said capturing is enabled, and thereby allowing at least some of said motion events to reach said underlying application when said capturing is disabled;
adjusting said (x′,y′) by the product of a constant and a vector difference between coordinates of the two most recent of said motion events obtained by said step of capturing;
changing said mode value, thereby yielding a new value of said mode value, when said step of adjusting initially makes said (x′,y′) be outside the bounds of said visible coordinate range;
and
selecting one of the following steps when said user pushes said first button:
performing a first gesture composed from said (x′,y′) when said mode value equals 1;
saving said (x′,y′) as said (swipeX,swipeY) when said mode value equals 2 and said swipeX equals −1;
performing a second gesture composed from said (x′,y′) and said (swipeX,swipeY) when said mode value equals 2 and said swipeX does not equal −1;
saving said (x′,y′) as said (zoomX,zoomY) when said mode value equals 3 and said zoomX equals −1;
saving said (x′,y′) as said (swipeX,swipeY) when said mode value equals 3 and said zoomX does not equal −1 and said swipeX equals −1;
performing a third gesture composed from said (x′,y′), said (swipeX,swipeY) and said (zoomX,zoomY) when said mode value equals 3 and said zoomX does not equal −1 and said swipeX does not equal −1.

21 more elements

Initializing coordinate values:
(x′,y′), (swipeX,swipeY), (zoomX,zoomY) , and distance values:
sx, sy, and mode value;
computing, based on said (x′,y′), said sx, said sy, said h and said w, updated values for top as max(y′−sy,0), left as max(x′−sx,0), bottom as min(y′+sy,h), right as min(x′+sx,w), top2s as max(y′+sy,0), left2s as max(x′+sx,0), bottom2s as min(y′−sy,h) and right2s as min(x′−sy,w);
displaying on said touchscreen a translucent layer, in response to changes in said (x′,y′) and said mode value, including the steps of:
displaying a digit near said (x′,y′), wherein said digit corresponds to the integer value of said mode value;
displaying an arrow form said (swipeX,swipeY) to said (x′,y′) when said mode value equals 2 and said swipeX does not equal −1;
displaying an arrow from said (zoomX,zoomY) to said (x′,y′) when said mode value equals 3 and said zoomX does not equal −1 and said swipeX equals −1;
displaying a pivot symbol and an arrow from said (zoomX,zoomY) to said (swipeX,swipeY) when said mode value equals 3 and said zoomX does not equal −1 and said swipeX does not equal −1, wherein the size of said pivot symbol is a logarithmic function involving said (x′,y′), said (zoomX,zoomY) and said (swipeX,swipeY);
displaying a first translucent rectangle with corners (0,0) and (left2s,top) that captures said motion events, whereby the visual appearance of said first translucent rectangle is based on said mode value;
displaying a second translucent rectangle with corners (0,bottom2s) and (left,h) that captures said motion events, whereby the visual appearance of said second translucent rectangle is based on said mode value;
displaying a third translucent rectangle with corners (right2s,bottom) and (w,h) that captures said motion events, whereby the visual appearance of said third translucent rectangle is based on said mode value;
displaying a fourth translucent rectangle with corners (right,0) and (w,top2s) that captures said motion events, whereby the visual appearance of said fourth translucent rectangle is based on said mode value;
and
displaying a fifth transparent rectangle with corners (right,top) and (left2s,bottom2s) whose capturing of said motion events can be enabled and disabled;
adjusting said (x′,y′) by the product of a constant and a vector difference between coordinates of the two most recent of said motion events obtained by said step of capturing;
changing said mode value, thereby yielding a new value of said mode value, in response to a request from said user;
selecting one of the following steps when said user pushes said first button:
saving said (x′,y′) as said (swipeX,swipeY) when said mode value equals 2and said swipeX equals −1;
saving said (x′,y′) as said (zoomX,zoomY) when said mode value equals 3and said zoomX equals −1;
saving said (x′,y′) as said (swipeX,swipeY) when said mode value equals 3and said zoomX does not equal −1 and said swipeX equals −1;
otherwise performing the following steps:
creating a gesture based one of the following:
said (x′,y′) if said mode value equals 1;
said (x′,y′) and said (swipeX,swipeY) if said mode value equals 2;
said (x′,y′), said (swipeX,swipeY) and said (zoomX,zoomY) if said mode value equals 3;
setting said (swipeX,swipeY) and said (zoomX,zoomY) to (−1,−1);
disabling said capturing by said fifth transparent rectangle;
dispatching said gesture, whereby said gesture will be received by said underlying application;
and
enabling said capturing by said fifth transparent rectangle.

28 more elements

Dependencies are inferred from the claim text.

Description11,615 words

Summary2,464 words

#The following is a tabulation of some of the prior art that appears relevant:

#U.S. Patents Patent Number Kind Code Issue Date Patentee 9,110,523 B2 2015 Aug. 18 Mann 8,947,405 B2 2015 Feb. 3 Cho, et al. 8,803,844 B1 2014 Aug. 12 Green, et al. 8,766,543 B2 2014 Jul. 1 Vuppu, et. al. 8,648,837 B2 2014 Feb. 11 Tran, et al. 8,564,553 B2 2013 Nov. 22 I-Hau Yew, et al. 8,528,117 B2 2013 Sep. 10 Asiaghi 8,125,469 B2 2012 Feb. 28 Badeye, et al. 8,094,133 B2 2010 Mar. 18 Sato, et al.,

#U.S. Patent Application Publications Patent Number Kind Code Publ. Date Applicant 20140104224 A1 2014 Apr. 17 Ih, et al. 20130268094 A1 2013 Oct. 10 Wiemeersch,et al. 20130257804 A1 2013 Oct. 3 Vu, et al. 20130234986 A1 2013 Sep. 12 Elias 20130127791 A1 2013 May 23 Siuta 20130106714 A1 2013 May Shahparnia 20120242618 A1 2012 Sep. 27 Everest, et al. 20110304577 A1 2011 December Brown 20110285671 A1 2011 Nov. 24 Wu, et al. 20110248946 A1 2011 October Michaelis 20110234623 A1 2011 Sep. 29 Ure 20100053113 A1 2010 Mar. 4 Wu, et al. 20100053120 A1 2010 Mar. 4 Chang, et al. 20090256824 A1 2009 Oct. 15 Hanizl, et al. 20090211821 A1 2009 Aug. 27 Yy-Cho Yee 20090167727 A1 2009 Jul. 2 Yi-Ching Liu, et al. 20050231471 A1 2005 Oct. 20 Mallard, et al.

Brief description of the drawings15 figures

#FIG. 1 shows the context in which the unaided human hand interacts capacitively with a prior-art touchscreen device.

#FIG. 2 shows a simplified view of the rectangular array of translucent capacitive plates and associated electronics composing a prior-art touchscreen together with the CPU that enables software to run on the touchscreen device.

#FIG. 3 shows conceptually the layers of software running in a prior-art touchscreen device relevant to the present invention, including operating system kernel driver, gesture recognition, user interface and user application(s).

Detailed description8,460 words

#Referring to FIG. 1, the prior-art capacitive touchscreen device 100 includes a touchscreen 101. The touchscreen 101 senses the change in capacitance that results when it is touched by a user's finger 204. Underneath the translucent material of touchscreen 101 is a pixel display 109, allowing an easy-to-understand user interface, in which the user touches the portion of touchscreen 101 at coordinate (x,y) co-located with the portion of an image displayed on pixel display 109. The (x,y) sensed by touchscreen 101 identifies an object displayed on touchscreen 109 of interest to the user. Typically, a prior-art touchscreen device 100 also includes several additional input-output interfaces, such as microphone 102, audio jack 103, wireless transceiver 104 and USB port 105. Patent application Ser. No. 14/708,572 envisions the proposed extra-software layer could operate without using such additional input-output interfaces by receiving signals encoded via touchscreen 101. The five-rectangle filtering approach disclosed below is compatible with both the touchscreen-signaling approach of patent application Ser. No. 14/708,572 as well as prior-art accessories described earlier that use additional input-output interfaces such as microphone 102, audio jack 103, wireless transceiver 104 and USB port 105.

#The majority of the area of touchscreen 101 and the corresponding area of pixel display 109 are available to be configured arbitrarily by user applications, which are software programs created by third parties (other than the manufacturer) that the user may choose to download onto the device. Often, the operating-system reserves a small menu region 199 of touchscreen 101 that is unavailable to user applications. Touching portions of menu region 199 invoke operating-system functions, such are returning to an operating-system home screen or back to a previous screen.

#The touchscreen 101 recognizes when it is touched by the user's hand 200 (usually by a single finger 204) because of the electrical circuit formed between the case of touchscreen device 100 through the finger-to-case capacitance 203 (typically with air as the dielectric and observed as CC picofarads), continuing through the finger-to-screen capacitance 201 (typically with glass or other similar material as the dielectric and observed as CS picofarads). Most regions of the touchscreen 101 are not connected through the user's hand in this fashion, and instead are connected through a simpler circuit formed between the case of touchscreen device 100 through the unconnected capacitance 190 (with air and glass as dielectrics observed as CU picofarads). Referring to FIGS. 1 and 2, the touchscreen 101 consists of a large two-dimensional array of individual capacitor plates (of which plates 111, 112, 113, 121, 122, 123, 131, 132 and 133 are shown with ellipses in between to indicate many similar plates) embedded within the translucent dielectric material on the surface of touchscreen 101. The plates are connected to a plurality of voltage generators (of which generators 110, 120 and 130 are shown with ellipses to indicate many similar generators) corresponding to each row of the array via a plurality of row wires (of which wires 119, 129 and 139 are shown with ellipses to indicate many similar wires). The generators create a voltage spike relative to the case ground 100. The plates are also connected along the columns to a plurality of sensors (of which voltage sensors 161, 162 and 163 are shown with ellipses to indicate many similar sensors) via a plurality of column wires (of which wires 151, 152 and 153 are shown with ellipses to indicate many similar wires). The voltage generators 110, 120, 130, . . . and sensors 161, 162, 163, . . . cooperate to measure the capacitances of plates 111, 112, 113, 121, 122, 123, 131, 132, 133, . . . in a time-multiplexed (row-by-row) fashion under the control of proprietary-hardware interface 171. Conceptually, this control is communicated via internal bus 170, although those skilled in the art can envision many alternative control arrangements. For example, the arrangement in FIG. 2 is capable of multi-touch, that is recognizing when more than one finger touches the screen. Those skilled in the art can envision a simpler arrangement only capable of recognizing one finger that time multiplexes rows and columns. The intelligence and functionality of the touchscreen device 100 (for instance, as a phone or a tablet computer) is implemented largely by a CPU 173, which connects to the proprietary hardware interface 171 (and therefore all the elements that function as touchscreen 101) via the CPU bus 172. Those skilled in the art will realize there are a wide variety of CPU bus architectures used in the prior art, and that the simple abstract connections shown in FIG. 2 are intended to represent any of these.

Facts

Application

15/932,139Filed 7 Feb 2018

Published

US 2019/0243467 A18 Aug 2019

Granted

17 Mar 2020

Term ends

7 Feb 2038

Estimated · how

20 years from earliest known nonprovisional domestic filing plus recorded patent term adjustment. Maintenance-fee lapses, terminal disclaimers and statutory extensions have not been checked. The continuity chain may be incomplete; this is not an enforceability determination.

Classification17 codes

G06F 3/038
Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor: control and interface…
G06F 3/03545
With detection of two-dimensional [2D] relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D…
G06F 3/0362
Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor: with detection of…
G06F 3/0383
Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry: signal control means within the pointing device

Five-rectangle method for dispatching touch events from motion-disabled users

Life of this patent

Five-rectangle method for dispatching touch events from motion-disabled users

Life of this patent

Life of this patent

Abstract

Drawings

Drawings

Claims21

Description11,615 words

Drawings

Classification17 codes

Life of this patent

Description11,615 words