# US 2016/0246498 A1 · Touch screen finger tracing device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0246498 A1
- Document ID: US20160246498A1
- Status: Published application
- Kind: A1
- Application number: 15/048,782
- Filed: 2016-02-19
- Published: 2016-08-25
- Earliest claimed: 2015-02-23
- Assignees: Capit Learning
- Applicants: Capit Learning
- Inventors: Eyal Rav-Noy; Tzippy Rav-Noy
- Web page: https://patentgrep.com/patent/US20160246498A1

## CPC

- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0486: Drag-and-drop
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06V 30/32: Digital ink
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A portable electronic device includes a touch display configured to sense contact with a finger or stylus, a speaker, and a processing system that communicates with the touch display and the speaker. The device is configured to display a symbol using a set of trace indicator line segments comprising a trace direction indicator and a trace termination indicator and to highlight a region of a first of the trace indicator line segments. The device is configured to detect user touch input at multiple locations on the touch display and determine whether the first trace indicator line segment has been traced from the highlighted region to the termination indicator in a direction corresponding to the trace direction indicator, and in response, to draw and maintain a color line corresponding to the detected user touch input at the multiple locations.

## Claims

### Claim 1

A portable electronic device having a touch user interface, the portable electronic device comprising:
a touch display configured to sense contact with a finger or stylus and to display visual content;
a speaker;
a processing system comprising at least one computing device, the processing system configured to communicate with the touch display and the speaker; and
non-transitory memory that stores instructions that when executed by the processing system cause the portable electronic device to perform operations comprising:
display a first iteration of a first symbol, wherein the first iteration of the first symbol is displayed without trace indicator line segments;
generate a sound corresponding to the first symbol using the speaker;
display a second iteration of the first symbol using a set of trace indicator line segments comprising a trace direction indicator and a trace termination indicator;
highlight an end region of a first of the trace indicator line segments;
detect user touch input at a plurality of locations on the touch display;
based at least in part on the detected user touch input at the plurality of locations, determine whether the user has traced the first trace indicator line segment from the highlighted end region to the termination indicator in a direction corresponding to the trace direction indicator;
draw and maintain a color line corresponding to the detected user touch input at the plurality of locations at least partly in response to a determination that the user has traced the first trace indicator line segment from the highlighted end region to the termination indicator in a direction corresponding to the trace direction indicator.

### Claim 2 (depends on 1)

The portable electronic device having a touch user interface, as defined in claim 1, wherein the determination as to whether the user has traced the first trace indicator line segment from the highlighted end region to the termination indicator in a direction corresponding to the trace direction indicator comprises:
a comparison of a vector corresponding to the detected user touch input at the plurality of locations to a reference vector.

### Claim 3 (depends on 1)

The portable electronic device having a touch user interface, as defined in claim 1, the portable electronic device further comprising:
an accelerometer,
a 3-axis gyroscope,
a temperature sensor, and
a camera
a speaker.

### Claim 4 (depends on 1)

The portable electronic device having a touch user interface, as defined in claim 1, wherein the operations further comprise:
display a third iteration of the first symbol using the set of trace indicator line segments comprising the trace direction indicator and the trace termination indicator, wherein the first symbol overlays a symbol mnemonic graphic having a shape corresponding to that of the first symbol;
highlight an end region of the first of the trace indicator line segments of the third iteration of the first symbol while the third iteration of the first symbol overlays the symbol mnemonic graphic;
detect user touch input at two or more locations on the touch display while the third iteration of the first symbol overlays the symbol mnemonic graphic;
based at least in part on the detected user touch input at the two or more locations while the third iteration of the first symbol overlays the symbol mnemonic graphic, determine whether the user has traced the first trace indicator line segment of the third iteration of the first symbol from the highlighted end region to the termination indicator in a direction corresponding to the trace direction indicator;
maintain a color line corresponding to the detected user touch input at the two or more locations while the third iteration of the first symbol overlays the symbol mnemonic graphic at least partly in response to a determination that the user has traced the first trace indicator line segment from the highlighted end region to the termination indicator in a direction corresponding to the trace direction indicator.

### Claim 5 (depends on 1)

The portable electronic device having a touch user interface, as defined in claim 1, wherein the operations further comprise:
display a fourth iteration of the first symbol using a set of segments, wherein display of a given segment in the set of segments does not include display of the trace direction indicators;
highlight a first end of the first of the segments of the fourth iteration of the first symbol;
detect user touch input at two or more locations on the touch display while the fourth iteration of the first symbol is displayed using the set of segments;
based at least in part on the detected user touch input at the two or more locations while the fourth iteration of the first symbol is displayed using the set of segments, determine whether the user has traced the first segment from the highlighted first end to a termination location in the direction corresponding to the trace direction indicator of the second iteration of the first symbol;
maintain a color line corresponding to the detected user touch input at the two or more locations while the fourth iteration of the first symbol is displayed using the set of segments at least partly in response to a determination that the user has traced the first segment from the highlighted end to the termination area in the direction corresponding to the trace direction indicator.

### Claim 6 (depends on 1)

The portable electronic device having a touch user interface, as defined in claim 1, wherein the operations further comprise:
provide a symbol selection user interface comprising a plurality of symbols via the touch display;
receive a user symbol selection via the touch display;
display via the touch display the selected symbol in conjunction with a symbol mnemonic graphic;
monitor user interaction with the touch display at least while the selected symbol is displayed in conjunction with the symbol mnemonic graphic;
determined whether the user has dragged the symbol mnemonic graphic to overlay the selected symbol at least a first threshold amount;
at least partly in response to a determination that the user has dragged the symbol mnemonic graphic to overlay the selected symbol at least a first threshold amount, generate a sound, using the speaker, corresponding to a sound associated with the symbol.

### Claim 7 (depends on 1)

The portable electronic device having a touch user interface, as defined in claim 1, wherein the operations further comprise:
provide a symbol selection user interface;
receive a user symbol selection via the touch display;
display via the touch display the selected symbol in conjunction with a symbol mnemonic graphic;
display via the touch display the symbol mnemonic graphic as a plurality of displaced components;
display via the touch display an outline corresponding to the symbol mnemonic graphic;
monitor interaction of a user touch input with the symbol mnemonic graphic components;
determine whether the user has moved, via a user touch input, each of the symbol mnemonic graphic components into the assembled outline corresponding to the symbol mnemonic graphic to form an assembled symbol mnemonic graphic;
at least partly in response to a determination that the user has moved, via a user a touch input, each of the symbol mnemonic graphic components into the assembled outline corresponding to the symbol mnemonic graphic to form an assembled symbol mnemonic graphic, enable the user to move, via a touch input, the symbol over the assembled symbol mnemonic graphic;
determine whether the user has moved the symbol to over lay the assembled symbol mnemonic graphic;
at least partly in response to a determination that the user has moved the symbol to over lay the assembled symbol mnemonic graphic, play a sound file that includes a sound that corresponds to the symbol and a word that corresponds to the symbol mnemonic graphic.

### Claim 8 (depends on 1)

The portable electronic device having a touch user interface, as defined in claim 1, wherein the operations further comprise:
display via the touch display a first version of the first symbol;
display via the touch display a second version of the first symbol;
display a first target in association with the first version of the first symbol;
display a second target in association with the second version of the first symbol;
display a first symbol mnemonic graphic whose shape corresponds to the first version of the first symbol;
display a second symbol mnemonic graphic whose shape corresponds to the second version of the first symbol;
determine if the user has moved, via touch input, the first symbol to overlay the first target;
determine if the user has moved, via touch input, the second symbol to overlay the second target;
at least partly in response to a determination that the user has moved, via touch input, the first symbol to overlay the first target and the second symbol to overlay the second target, generate a confirmation sound.

### Claim 9 (depends on 1)

The portable electronic device having a touch user interface, as defined in claim 1, wherein the operations further comprise:
display a keyboard via the touch display, including a first key that displays the first symbol, a second key the displays a second symbol, and a plurality of keyboard keys that do not display symbols;
determine if the user has touched the first key;
at least partly in response to detecting that the user has touched the first key, display the first symbol in a symbol field and generate a sound corresponding to the first symbol.

### Claim 10

A method for processing touch inputs received via a touch device, the method comprising:
displaying via a touch display of the touch device a first iteration of a first symbol, wherein the first iteration of the first symbol is displayed without trace indicator line segments;
generating a sound corresponding to the first symbol;
displaying via the touch display of the touch device a second iteration of the first symbol using a set of trace indicator line segments comprising a trace direction indicator and a trace termination indicator;
highlighting via the touch display of the touch device a first region of a first of the trace indicator line segments;
detecting user touch input at a plurality of locations on the touch display;
based at least in part on the detected user touch input at the plurality of locations, determining whether the user has traced the first trace indicator line segment from the highlighted region to the termination indicator in a direction corresponding to the trace direction indicator; and
drawing and maintaining a color line corresponding to the detected user touch input at the plurality of locations at least partly in response to a determination that the user has traced the first trace indicator line segment from the highlighted first region to the termination indicator in a direction corresponding to the trace direction indicator.

### Claim 11 (depends on 10)

The method as defined in claim 10, wherein determining whether the user has traced the first trace indicator line segment from the highlighted first region to the termination indicator in a direction corresponding to the trace direction indicator comprises:
a comparison of a vector corresponding to the detected user touch input at the plurality of locations to a reference vector.

### Claim 12 (depends on 10)

The method as defined in claim 10, the method further comprising:
displaying a third iteration of the first symbol using the set of trace indicator line segments comprising the trace direction indicator and the trace termination indicator, wherein the first symbol overlays a symbol mnemonic graphic having a shape corresponding to that of the first symbol;
highlighting a region of the first of the trace indicator line segments of the third iteration of the first symbol while the third iteration of the first symbol overlays the symbol mnemonic graphic;
detecting user touch input at two or more locations on the touch display while the third iteration of the first symbol overlays the symbol mnemonic graphic;
based at least in part on the detected user touch input at the two or more locations while the third iteration of the first symbol overlays the symbol mnemonic graphic, determining whether the user has traced the first trace indicator line segment of the third iteration of the first symbol from the highlighted region to the termination indicator in a direction corresponding to the trace direction indicator;
maintaining a color line corresponding to the detected user touch input at the two or more locations while the third iteration of the first symbol overlays the symbol mnemonic graphic at least partly in response to a determination that the user has traced the first trace indicator line segment from the highlighted region to the termination indicator in a direction corresponding to the trace direction indicator.

### Claim 13 (depends on 10)

The method as defined in claim 10, the method further comprising:
displaying a fourth iteration of the first symbol using a set of segments, wherein display of a given segment in the set of segments does not include display of the trace direction indicators;
highlighting a region of the first of the segments of the fourth iteration of the first symbol;
detecting user touch input at two or more locations on the touch display while the fourth iteration of the first symbol is displayed using the set of segments;
based at least in part on the detected user touch input at the two or more locations while the fourth iteration of the first symbol is displayed using the set of segments, determine whether the user has traced the first segment from the highlighted region to a termination location in the direction corresponding to the trace direction indicator of the second iteration of the first symbol;
maintaining a color line corresponding to the detected user touch input at the two or more locations while the fourth iteration of the first symbol is displayed using the set of segments at least partly in response to a determination that the user has traced the first segment from the highlighted region to the termination area in the direction corresponding to the trace direction indicator.

### Claim 14 (depends on 10)

The method as defined in claim 10, the method further comprising:
providing a symbol selection user interface comprising a plurality of symbols via the touch display;
receiving a user symbol selection via the touch display;
displaying via the touch display the selected symbol in conjunction with a symbol mnemonic graphic;
monitoring user interaction with the touch display at least while the selected symbol is displayed in conjunction with the symbol mnemonic graphic;
determining whether the user has dragged the symbol mnemonic graphic to overlay the selected symbol at least a first threshold amount;
at least partly in response to a determination that the user has dragged the symbol mnemonic graphic to overlay the selected symbol at least a first threshold amount, generating a sound corresponding to a sound associated with the symbol.

### Claim 15 (depends on 10)

The method as defined in claim 10, the method further comprising:
providing a symbol selection user interface;
receiving a user symbol selection via the touch display;
displaying via the touch display the selected symbol in conjunction with a symbol mnemonic graphic;
displaying via the touch display the symbol mnemonic graphic as a plurality of displaced components;
displaying via the touch display an outline corresponding to the symbol mnemonic graphic;
monitoring interaction of a user touch input with the symbol mnemonic graphic components;
determining whether the user has moved, via a user touch input, each of the symbol mnemonic graphic components into the assembled outline corresponding to the symbol mnemonic graphic to form an assembled symbol mnemonic graphic;
at least partly in response to a determination that the user has moved, via a user a touch input, each of the symbol mnemonic graphic components into the assembled outline corresponding to the symbol mnemonic graphic to form an assembled symbol mnemonic graphic, enabling the user to move, via a touch input, the symbol over the assembled symbol mnemonic graphic;
determining whether the user has moved the symbol to over lay the assembled symbol mnemonic graphic;
at least partly in response to determining that the user has moved the symbol to over lay the assembled symbol mnemonic graphic, play a sound file that includes a sound that corresponds to the symbol and a word that corresponds to the symbol mnemonic graphic.

### Claim 16 (depends on 10)

The method as defined in claim 10, the method further comprising:
displaying via the touch display a first version of the first symbol;
displaying via the touch display a second version of the first symbol;
displaying a first target in association with the first version of the first symbol;
displaying a second target in association with the second version of the first symbol;
displaying a first symbol mnemonic graphic whose shape corresponds to the first version of the first symbol;
displaying a second symbol mnemonic graphic whose shape corresponds to the second version of the first symbol;
determining if the user has moved, via touch input, the first symbol to overlay the first target;
determining if the user has moved, via touch input, the second symbol to overlay the second target;
at least partly in response to determining that the user has moved, via touch input, the first symbol to overlay the first target and the second symbol to overlay the second target, generating a confirmation sound.

### Claim 17 (depends on 10)

The method as defined in claim 10, the method further comprising:
displaying a keyboard via the touch display, including a first key that displays the first symbol, a second key the displays a second symbol, and a plurality of keyboard keys that do not display symbols;
determining if the user has touched the first key;
at least partly in response to detecting that the user has touched the first key, displaying the first symbol in a symbol field and generate a sound corresponding to the first symbol.

### Claim 18

Non-transitory memory that stores instructions that when executed by a computing system cause the computing to perform operations comprising:
display a first iteration of a first symbol, wherein the first iteration of the first symbol is displayed without trace indicator line segments;
generate a sound corresponding to the first symbol;
display a second iteration of the first symbol using a set of trace indicator line segments comprising a trace direction indicator and a trace termination indicator;
highlight an first region of a first of the trace indicator line segments;
detect user touch input at a plurality of locations on the touch display;
based at least in part on the detected user touch input at the plurality of locations, determine whether the user has traced the first trace indicator line segment from the highlighted first region to the termination indicator in a direction corresponding to the trace direction indicator; and
draw and maintain a color line corresponding to the detected user touch input at the plurality of locations at least partly in response to a determination that the user has traced the first trace indicator line segment from the highlighted first region to the termination indicator in a direction corresponding to the trace direction indicator.

### Claim 19 (depends on 18)

The non-transitory memory as defined in claim 18, wherein the determination as to whether the user has traced the first trace indicator line segment from the highlighted first region to the termination indicator in a direction corresponding to the trace direction indicator comprises:
a comparison of a vector corresponding to the detected user touch input at the plurality of locations to a reference vector.

### Claim 20 (depends on 18)

The non-transitory memory as defined in claim 18, wherein the instructions are further configured to cause the computing to perform operations comprising:
display a third iteration of the first symbol using the set of trace indicator line segments comprising the trace direction indicator and the trace termination indicator, wherein the first symbol overlays a symbol mnemonic graphic having a shape corresponding to that of the first symbol;
highlight a region of the first of the trace indicator line segmentstrace indicator line segments of the third iteration of the first symbol while the third iteration of the first symbol overlays the symbol mnemonic graphic;
detect user touch input at two or more locations on the touch display while the third iteration of the first symbol overlays the symbol mnemonic graphic;
based at least in part on the detected user touch input at the two or more locations while the third iteration of the first symbol overlays the symbol mnemonic graphic, determine whether the user has traced the first trace indicator line segment of the third iteration of the first symbol from the highlighted region to the termination indicator in a direction corresponding to the trace direction indicator;
maintain a color line corresponding to the detected user touch input at the two or more locations while the third iteration of the first symbol overlays the symbol mnemonic graphic at least partly in response to a determination that the user has traced the first trace indicator line segment from the highlighted region to the termination indicator in a direction corresponding to the trace direction indicator.

### Claim 21 (depends on 18)

The non-transitory memory as defined in claim 18, wherein the instructions are further configured to cause the computing to perform operations comprising:
display a fourth iteration of the first symbol using a set of segments, wherein display of a given segment in the set of segments does not include display of the trace direction indicators;
highlight a region of the first of the segments of the fourth iteration of the first symbol;
detect user touch input at two or more locations on the touch display while the fourth iteration of the first symbol is displayed using the set of segments;
based at least in part on the detected user touch input at the two or more locations while the fourth iteration of the first symbol is displayed using the set of segments, determine whether the user has traced the first segment from the highlighted region to a termination location in the direction corresponding to the trace direction indicator of the second iteration of the first symbol;
maintain a color line corresponding to the detected user touch input at the two or more locations while the fourth iteration of the first symbol is displayed using the set of segments at least partly in response to a determination that the user has traced the first segment from the highlighted region to the termination area in the direction corresponding to the trace direction indicator.

### Claim 22 (depends on 18)

The non-transitory memory as defined in claim 18, wherein the instructions are further configured to cause the computing to perform operations comprising:
provide a symbol selection user interface comprising a plurality of symbols via the touch display;
receive a user symbol selection via the touch display;
display via the touch display the selected symbol in conjunction with a symbol mnemonic graphic;
monitor user interaction with the touch display at least while the selected symbol is displayed in conjunction with the symbol mnemonic graphic;
determined whether the user has dragged the symbol mnemonic graphic to overlay the selected symbol at least a first threshold amount;
at least partly in response to a determination that the user has dragged the symbol mnemonic graphic to overlay the selected symbol at least a first threshold amount, generate a sound corresponding to a sound associated with the symbol.

### Claim 23 (depends on 18)

The non-transitory memory as defined in claim 18, wherein the instructions are further configured to cause the computing to perform operations comprising:
provide a symbol selection user interface;
receive a user symbol selection via the touch display;
display via the touch display the selected symbol in conjunction with a symbol mnemonic graphic;
display via the touch display the symbol mnemonic graphic as a plurality of displaced components;
display via the touch display an outline corresponding to the symbol mnemonic graphic;
monitor interaction of a user touch input with the symbol mnemonic graphic components;
determine whether the user has moved, via a user touch input, each of the symbol mnemonic graphic components into the assembled outline corresponding to the symbol mnemonic graphic to form an assembled symbol mnemonic graphic;
at least partly in response to a determination that the user has moved, via a user a touch input, each of the symbol mnemonic graphic components into the assembled outline corresponding to the symbol mnemonic graphic to form an assembled symbol mnemonic graphic, enable the user to move, via a touch input, the symbol over the assembled symbol mnemonic graphic;
determine whether the user has moved the symbol to over lay the assembled symbol mnemonic graphic;
at least partly in response to a determination that the user has moved the symbol to over lay the assembled symbol mnemonic graphic, play a sound file that includes a sound that corresponds to the symbol and a word that corresponds to the symbol mnemonic graphic.

### Claim 24 (depends on 18)

The non-transitory memory as defined in claim 18, wherein the instructions are further configured to cause the computing to perform operations comprising:
display via the touch display a first version of the first symbol;
display via the touch display a second version of the first symbol;
display a first target in association with the first version of the first symbol;
display a second target in association with the second version of the first symbol;
display a first symbol mnemonic graphic whose shape corresponds to the first version of the first symbol;
display a second symbol mnemonic graphic whose shape corresponds to the second version of the first symbol;
determine if the user has moved, via touch input, the first symbol to overlay the first target;
determine if the user has moved, via touch input, the second symbol to overlay the second target;
at least partly in response to a determination that the user has moved, via touch input, the first symbol to overlay the first target and the second symbol to overlay the second target, generate a confirmation sound.

### Claim 25 (depends on 18)

The non-transitory memory as defined in claim 18, wherein the instructions are further configured to cause the computing to perform operations comprising:
display a keyboard via the touch display, including a first key that displays the first symbol, a second key the displays a second symbol, and a plurality of keyboard keys that do not display symbols;
determine if the user has touched the first key;
at least partly in response to detecting that the user has touched the first key, display the first symbol in a symbol field and generate a sound corresponding to the first symbol.
