# US 2012/0075194 A1 · Adaptive virtual keyboard for handheld device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0075194 A1
- Document ID: US20120075194A1
- Status: Published application
- Kind: A1
- Application number: 13/376,620
- Filed: 2009-12-31
- Published: 2012-03-29
- Earliest claimed: 2009-06-16
- Assignees: Intel Corporation
- Inventors: Bran Ferren
- Web page: https://patentgrep.com/patent/US20120075194A1

## CPC

- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G02B 13/0065: {having a beam-folding prism or mirror}
- G02B 13/009: {having zoom function}
- G03B 3/00: Focusing arrangements of general interest for cameras, projectors or printers
- G06F 3/02: Input arrangements using manually operated switches, e.g. using keyboards or dials
- G06F 3/0236: {using selection techniques to select from displayed items}
- G06F 3/0237: {using prediction or retrieval techniques}
- G06F 3/041: Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
- G06F 3/0412: {Digitisers structurally integrated in a display}
- G06F 3/0414: {using force sensing means to determine a position}
- G06F 3/0418: {for error correction or compensation, e.g. based on parallax, calibration or alignment}
- G06F 3/04186: {Touch location disambiguation}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 9/451: Execution arrangements for user interfaces
- G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints
- G06F 21/40: by quorum, i.e. whereby two or more security principals are required
- G06K 7/10811: {Focalisation}
- G06Q 20/3276: {using a pictured code, e.g. barcode or QR-code, being read by the M-device}
- G06Q 20/4012: {Verifying personal identification numbers [PIN]}
- G06Q 20/40145: {Biometric identity checks}
- G06Q 30/02: Marketing; Price estimation or determination; Fundraising
- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/19: Sensors therefor
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06V 40/45: {Detection of the body part being alive}
- H04L 63/083: {using passwords (cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226)}
- H04N 7/183: {for receiving images from a single remote source}
- H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence)
- H04N 13/243: using three or more two-dimensional [2D] image sensors
- H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
- H04N 23/63: by using electronic viewfinders
- H04N 23/67: Focus control based on electronic image sensor signals
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- G06K 2207/1011: Aiming
- H04M 1/72448: with means for adapting the functionality of the device according to specific conditions
- H04M 2250/52: including functional features of a camera
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

In various embodiments, the size, shape, and arrangement of keys on a virtual keyboard may be determined based on touchscreen contacts made by the user. Further, the actual contact patch made by the user may be analyzed to interpret which point of contact was intended, and other factors such as spelling and context may also be considered. These factors may be determined based on a calibration session and/or on continuing inputs during operation of the keyboard, and applied to future operational interpretations of the touchscreen contacts.

## Claims

### Claim 1

An apparatus, comprising:
a handheld electronic device having a touchscreen with a virtual keyboard, wherein the virtual keyboard has first and second arc-shaped tiered rows of keys.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the virtual keyboard has two portions, the first portion having at least the first and second rows of keys arranged for operation by one of a user's thumbs, and the second portion having at least a third arc-shaped row of keys arranged for operation by the user's other thumb, when the user is gripping the device with both right and left hands.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein at least one item selected from the following list is programmable by user action:
a) a distance between the first and second rows;
b) a distance between adjacent keys in a row; and
c) which character is assigned to each key.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein a display of a key on the touchscreen is to be shifted to a position visible to a user when the key is activated by a touch.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the device is to perform a keyboard calibration procedure comprising:
a) prompting a user to draw a calibration arc on the touchscreen with the user's thumb; and
b) generating the first row of keys for the virtual keyboard along the calibration arc drawn by the user's thumb.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the calibration procedure further comprises generating the second row of keys in relation to the first row of keys.

### Claim 7

A method, comprising:
entering a calibration mode;
detecting a first calibration arc drawn on a touchscreen; and
generating a first row of soft keys for a virtual keyboard along the first calibration arc.

### Claim 8 (depends on 7)

The method of claim 7, further comprising prompting a user, prior to said detecting, to draw the first calibration arc on the touchscreen.

### Claim 9 (depends on 7)

The method of claim 7, further comprising generating a second row of soft keys having a tiered relation with the first row of soft keys.

### Claim 10 (depends on 7)

The method of claim 7, further comprising:
detecting a second calibration arc drawn on the touchscreen; and
generating a second row of soft keys for the virtual keyboard along the second calibration arc.

### Claim 11 (depends on 10)

The method of claim 10, wherein the first and second calibration arcs are tiered with respect to each other.

### Claim 12 (depends on 10)

The method of claim 10, wherein:
the first and second calibration arcs are drawn by the user using different thumbs; and
the first and second calibration arcs are not tiered with respect to each other.

### Claim 13

An article comprising
a computer-readable storage medium that contains instructions, which when executed by one or more processors result in performing operations comprising:
entering a calibration mode;
detecting a first calibration arc drawn on a touchscreen; and
generating a first row of soft keys for a virtual keyboard along the first calibration arc.

### Claim 14 (depends on 13)

The article of claim 13, wherein the operations further comprises prompting a user, prior to said detecting, to draw the first calibration arc on the touchscreen.

### Claim 15 (depends on 13)

The article of claim 13, wherein the operations further comprise generating a second row of soft keys having a tiered relation with the first row of soft keys.

### Claim 16 (depends on 13)

The article of claim 13, wherein the operations further comprise:
detecting a second calibration arc drawn on the touchscreen; and
generating a second row of soft keys for the virtual keyboard along the second calibration arc.

### Claim 17 (depends on 16)

The article of claim 16, wherein the first and second calibration arcs are tiered with respect to each other.

### Claim 18 (depends on 16)

The article of claim 16, wherein:
the first and second calibration arcs are drawn by the user using different thumbs; and
the first and second calibration arcs are not tiered with respect to each other.

### Claim 19

An apparatus, comprising:
a handheld device including a touchscreen for displaying a virtual keyboard, wherein the device is to:
detect a touch on the touchscreen by a user;
determine a location for a centroid of a contact patch for the touch; and
determine a location of an active point with respect to the centroid.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein said determining the location of the centroid comprises:
receiving readings from multiple sensor points on the touchscreen located within the contact patch; and
calculating the centroid from the multiple sensor points.

### Claim 21 (depends on 19)

The apparatus of claim 19, wherein said determining the location of the active point is based at least in part on one or more factors selected from a list consisting of:
the centroid of the contact patch;
a size of the contact patch;
a shape of the contact patch;
a difference in pressure sensed at the multiple points;
a location on the keyboard of the contact patch;
a positional relationship between the centroid and a keyboard key that is at least partly overlapped by the contact patch; and
a previous history of the positional relationship between the contact patch and the keyboard key.

### Claim 22 (depends on 19)

The apparatus of claim 19, wherein said detecting a touch is to be performed during a calibration procedure for the keyboard.

### Claim 23

A method, comprising:
sensing a touch on a touchscreen of a handheld electronic device;
determining a centroid for a contact patch of the touch; and
determining an active point for the contact patch, wherein the active point is a different location than the centroid.

### Claim 24 (depends on 23)

The method of claim 23, wherein said determining the active point is based at least in part on one or more factors selected from a list consisting of:
the centroid of the contact patch;
a size of the contact patch;
a shape of the contact patch;
a difference in pressure sensed at multiple points within the contact patch;
a location on the keyboard of the contact patch;
a positional relationship between the contact patch and a keyboard key that is at least partly overlapped by the contact patch; and
a previous history of the positional relationship between the contact patch and the keyboard key.

### Claim 25 (depends on 23)

The method of claim 23, wherein said sensing is performed during an initial calibration procedure for the keyboard.

### Claim 26 (depends on 23)

The method claim 23, wherein said adjusting is performed during operational usage of the keyboard.

### Claim 27

An article comprising
a computer-readable storage medium that contains instructions, which when executed by one or more processors result in performing operations comprising:
sensing a contact patch on a touchscreen of a handheld electronic device;
determining a centroid for a contact patch for the touch; and
determining an active point for the contact patch.

### Claim 28 (depends on 27)

The article of claim 27, wherein the operation of determining the active point is based at least in part on one or more factors selected from a list consisting of:
a centroid of the contact patch;
a size of the contact patch;
a shape of the contact patch;
a difference in pressure sensed at the multiple points;
a location on the keyboard of the contact patch;
a positional relationship between the contact patch and a keyboard key that is at least partly overlapped by the contact patch;
a previous history of the positional relationship between the contact patch and the keyboard key.

### Claim 29 (depends on 27)

The article of claim 27, wherein the operation of adjusting is performed during an initial calibration procedure for the keyboard.

### Claim 30 (depends on 27)

The article claim 27, wherein the operation of adjusting is performed during operational usage of the keyboard.

### Claim 31

An apparatus, comprising:
a handheld electronic device including a touchscreen for displaying a virtual keyboard, wherein the device is to:
detect a contact patch when the touchscreen is touched by a user;
determine an active point for the contact patch;
determine a hot spot for each of multiple keys located near the active point;
select a particular key from among the multiple keys as a key intended by the user; and
include a particular character represented by the particular key as a character in an ongoing stream of text input by the user.

### Claim 32 (depends on 31)

The apparatus of claim 31, wherein said selection is based on at least one criteria selected from a list consisting of:
location of each of the hotspots with respect to the active point; and
a probability based at least in part on a past history of the active point and at least some of the multiple keys.

### Claim 33 (depends on 31)

The apparatus of claim 31, wherein the device is to relocate the hot spot for the particular key for subsequent operations, based at least in part on determinations leading to the selection of the particular key.

### Claim 34 (depends on 31)

The apparatus of claim 31, wherein the device is further to:
detect additional contact patches;
select additional keys based on the additional contact patches;
include a corresponding character from each of the selected keys into the ongoing stream of text until at least a word has been formed;
determine the word is an invalid word; and
change the particular character to another character represented by another of the multiple keys to form a valid word.

### Claim 35 (depends on 34)

The apparatus of claim 34, wherein the device is further to move a hot spot for the particular key based at least in part on determinations leading to the changing of the particular character.

### Claim 36

A method, comprising:
determining an active point of a contact patch when a touchscreen of a handheld electronic device is touched by a user;
determining the active point is near a hot spot for each of multiple keys on a virtual keyboard on the touchscreen;
selecting a particular key from among the multiple keys as a key intended by the user; and
including a particular character represented by the particular key as a character in an ongoing stream of text input by the user.

### Claim 37 (depends on 36)

The method of claim 36, wherein said selecting is based on at least one criteria selected from a list consisting of:
location of each of the hotspots with respect to the contact patch; and
a probability based at least in part on a past history of the multiple keys and corresponding contact patches.

### Claim 38 (depends on 36)

The method of claim 36, further comprising moving a hot spot for the particular key based at least in part on determinations leading to said selecting the particular key.

### Claim 39 (depends on 36)

The method of claim 36, further comprising:
determining additional active points for additional contact patches generated by additional touches;
selecting additional keys based on the additional touches;
including a corresponding character from each of the selected additional keys into the ongoing stream of text until at least a word has been formed;
determining the word is an invalid word; and
changing the particular character to another character represented by another of the multiple keys to form a valid word.

### Claim 40 (depends on 39)

The method of claim 39, further comprising moving a hot spot for the particular key based at least in part on determinations leading to said changing of the particular character.

### Claim 41

An article comprising
a computer-readable storage medium that contains instructions, which when executed by one or more processors result in performing operations comprising:
determining an active point of a contact patch when a touchscreen of a handheld electronic device is touched by a user;
determining the active point is near a hot spot for each of multiple keys on a virtual keyboard on the touchscreen;
selecting a particular key from among the multiple keys as a key intended by the user; and
including a particular character represented by the particular key as a character in an ongoing stream of text input by the user.

### Claim 42 (depends on 41)

The article of claim 41, wherein said selecting is based on at least one criteria selected from a list consisting of:
a location of each of the hotspots with respect to the contact patch; and
a probability based at least in part on a past history of the multiple keys and corresponding contact patches.

### Claim 43 (depends on 41)

The article of claim 41, wherein the operations further comprise moving a hot spot for the particular key based at least in part on determinations leading to said selecting the particular key.

### Claim 44 (depends on 41)

The article of claim 41, wherein the operations further comprise:
determining additional active points for additional contact patches generated by additional touches;
selecting additional keys based on the additional touches;
including a corresponding character from each of the selected additional keys into the ongoing stream of text until at least a word has been formed;
determining the word is an invalid word; and
changing the particular character to another character represented by another of the multiple keys to form a valid word.

### Claim 45 (depends on 44)

The article of claim 44, wherein the operations further comprise moving a hot spot for the particular key based at least in part on determinations leading to said changing of the particular character.
