# US 2024/0393941 A1 · Handwriting entry on an electronic device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0393941 A1
- Document ID: US20240393941A1
- Status: Published application
- Kind: A1
- Application number: 18/796,097
- Filed: 2024-08-06
- Published: 2024-11-28
- Earliest claimed: 2019-05-06
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Julian MISSIG; Matan STAUBER; Guillaume ARDAUD; Jeffrey Traer BERNSTEIN; Christopher D. SOLI
- Web page: https://patentgrep.com/patent/US20240393941A1

## CPC

- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/03545: {Pens or stylus}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0485: Scrolling or panning
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- G06F 40/166: Editing, e.g. inserting or deleting
- G06T 13/80: Two-dimensional [2D] animation, e.g. using sprites
- G06V 30/333: {Preprocessing; Feature extraction}
- G06F 2203/04803: Split screen, i.e. subdividing the display area or the window area into separate subareas
- G06F 2203/04807: Pen manipulated menu
- G06F 2203/04808: Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

In some embodiments, an electronic device receives handwritten inputs in text entry fields and converts the handwritten inputs into font-based text. In some embodiments, an electronic device selects and deletes text based on inputs from a stylus. In some embodiments, an electronic device inserts text into pre-existing text based on inputs from a stylus. In some embodiments, an electronic device manages the timing of converting handwritten inputs into font-based text. In some embodiments, an electronic device presents a handwritten entry menu. In some embodiments, an electronic device controls the characteristic of handwritten inputs based on selections on the handwritten entry menu. In some embodiments, an electronic device presents autocomplete suggestions. In some embodiments, an electronic device converts handwritten input to font-based text. In some embodiments, an electronic device displays options in a content entry palette.

## Claims

### Claim 1

A method comprising:
at an electronic device in communication with a touch-sensitive display:
displaying, on the touch-sensitive display, a text entry user interface including a first sequence of characters that includes a first portion of the first sequence of characters and a second portion of the first sequence of characters;
while displaying the text entry user interface, receiving, via the touch-sensitive display, a user input in the text entry user interface in between the first portion of the first sequence of characters and the second portion of the first sequence of characters; and
in response to receiving the user input:
in accordance with a determination that the user input corresponds to a request to enter respective font-based text in between the first portion of the first sequence of characters and the second portion of the first sequence of characters using handwritten input, updating the text entry user interface by creating a space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, wherein the space between the first portion and the second portion is configured to receive the handwritten input for inserting the respective font-based text between the first portion and the second portion of the first sequence of characters.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
after updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, receiving, via the touch-sensitive display, a handwritten input in the space between the first portion and the second portion of the first sequence of characters; and
after receiving the handwritten input, converting the handwritten input into font-based text in between the first portion and the second portion of the first sequence of characters.

### Claim 3 (depends on 2)

The method of claim 2, wherein the handwritten input is detected after detecting the user input in between the first portion and the second portion of the first sequence of characters without detecting lift-off from the touch-sensitive display.

### Claim 4 (depends on 1)

The method of claim 1, wherein the user input corresponds to the request to enter respective text in between the first portion and the second portion of the first sequence of characters using handwritten input when the user input comprises touchdown of a stylus on the touch-sensitive display in between the first portion and the second portion of the first sequence of characters, and updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters occurs in response to detecting the touchdown of the stylus before detecting further input from the stylus.

### Claim 5 (depends on 4)

The method of claim 4, wherein the touchdown of the stylus is between two words of the first sequence of characters.

### Claim 6 (depends on 1)

The method of claim 1, wherein the user input corresponds to the request to enter respective text in between the first portion and the second portion of the first sequence of characters using handwritten input when the user input comprises touchdown of a stylus on the touch-sensitive display for longer than a time threshold, and updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters occurs in response to detecting the touchdown of a stylus on the touch-sensitive display for longer than the time threshold.

### Claim 7 (depends on 1)

The method of claim 1, wherein the user input corresponds to the request to enter respective text in between the first portion and the second portion of the first sequence of characters using handwritten input when the user input comprises a respective gesture, and updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters occurs in response to detecting the respective gesture.

### Claim 8 (depends on 1)

The method of claim 1, wherein the user input comprises touchdown of a stylus on the touch-sensitive display, the method further comprising:
in response to detecting the touchdown of the stylus in between the first and second portions of the first sequence of characters on the touch-sensitive display, displaying, on the touch-sensitive display, a selectable option for creating the space between the first and second portions of the first sequence of characters; and
while displaying the selectable option for creating the space between the first and second portions of the first sequence of characters, receiving, via the touch-sensitive display, selection of the selectable option,
wherein updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters occurs in response to detecting the selection of the selectable option.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
after updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, receiving, via the touch-sensitive display, a handwritten input in the space between the first portion and the second portion of the first sequence of characters; and
in response to receiving the handwritten input:
displaying a representation of the handwritten input in the space between the first and second portions of the first sequence of characters; and
in accordance with a determination that the handwritten input satisfies one or more criteria, expanding the space between the first and second portions of the first sequence of characters.

### Claim 10 (depends on 9)

The method of claim 9, wherein the handwritten input satisfies the one or more criteria when the handwritten input includes a first respective gesture, and does not satisfy the one or more criteria when the handwritten input includes a second respective gesture, different than the first respective gesture.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
after updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, receiving, via the touch-sensitive display, a handwritten input in the space between the first portion and the second portion of the first sequence of characters; and
in response to receiving the handwritten input:
displaying a representation of the handwritten input in the space between the first and second portions of the first sequence of characters; and
in accordance with a determination that one or more new line criteria are satisfied, updating the user interface to create a new line configured to receive additional handwritten input for inserting additional respective text in the new line.

### Claim 12 (depends on 11)

The method of claim 11, wherein the one or more new line criteria include a criterion that is satisfied when the handwritten input reaches an end of a current line in the user interface.

### Claim 13 (depends on 11)

The method of claim 11, wherein the one or more new line criteria include a criterion that is satisfied when the additional handwritten input is detected below existing font-based text in the user interface.

### Claim 14 (depends on 11)

The method of claim 11, wherein the one or more new line criteria include a criterion that is satisfied when a tap input is detected below existing font-based text in the user interface.

### Claim 15 (depends on 11)

The method of claim 11, further comprising:
in response to receiving the handwritten input:
in accordance with a determination that the handwritten input is within a threshold distance of an end of a current line in the user interface, displaying, in the user interface, a selectable option for creating a new line in the user interface,
wherein the one or more new line criteria include a criterion that is satisfied when selection of the selectable option for creating the new line in the user interface is detected.

### Claim 16 (depends on 11)

The method of claim 11, further comprising:
while the new line configured to receive the additional handwritten input is included in the user interface, receiving, via the touch-sensitive display, a respective user input; and
in response to receiving the respective user input:
in accordance with a determination that the respective user input comprises a tap input detected at an end of a last word in a previous line, previous to the new line, in the user interface, or a tap input detected at a beginning of a first word in the new line in the user interface, displaying, in the user interface, a selectable option for removing the new line from the user interface.

### Claim 17 (depends on 11)

The method of claim 11, further comprising:
while the new line configured to receive the additional handwritten input is included in the user interface and the new line includes a respective sequence of characters, receiving, via the touch-sensitive display, a respective input including a touchdown of a stylus on the respective sequence of characters and a movement of the stylus to a respective line, different than the new line, in the user interface; and
in response to receiving the respective input:
moving the respective sequence of characters to the respective line in the user interface; and
removing the new line from the user interface.

### Claim 18 (depends on 1)

The method of claim 1, further comprising:
after updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, receiving, via the touch-sensitive display, a handwritten input in the space between the first portion and the second portion of the first sequence of characters; and
in response to receiving the handwritten input:
displaying, in the user interface, a representation of the handwritten input in the space between the first and second portions of the first sequence of characters;
in accordance with a determination that the handwritten input has not reached an end of a current line in the user interface, ceasing to display the representation of the handwritten input after a first elapsed time since receiving the handwritten input; and
in accordance with a determination that the handwritten input has reached the end of the current line in the user interface, ceasing to display the representation of the handwritten input after a second elapsed time, shorter than the first elapsed time, since receiving the handwritten input.

### Claim 19 (depends on 1)

The method of claim 1, further comprising:
after updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, receiving, via the touch-sensitive display, a handwritten input in the space between the first portion and the second portion of the first sequence of characters; and
after receiving the handwritten input:
in accordance with a determination that no additional handwritten input is received for a time threshold after an end of the handwritten input, reducing a size of the space between the first portion and the second portion of the first sequence of characters to remove space not consumed by the handwritten input in the user interface.

### Claim 20 (depends on 1)

The method of claim 1, further comprising:
after updating the text entry user interface by creating the space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, receiving, via the touch-sensitive display, a handwritten input in the space between the first portion and the second portion of the first sequence of characters; and
after receiving the handwritten input:
in accordance with a determination that no additional handwritten input is received for a time threshold after an end of the handwritten input, converting the handwritten input into font-based text in the space between the first and second portions of the first sequence of characters.

### Claim 21 (depends on 1)

The method of claim 1, further comprising:
displaying, in the text entry user interface, a second sequence of characters that includes a first portion of the second sequence of characters and a second portion of the second sequence of characters;
while displaying the text entry user interface, receiving, via the touch-sensitive display, a second user input in the text entry user interface in between the first portion of the second sequence of characters and the second portion of the second sequence of characters; and
in response to receiving the second user input:
in accordance with a determination that the second user input corresponds to a request to enter second respective font-based text in between the first portion of the second sequence of characters and the second portion of the second sequence of characters using handwritten input:
displaying, in the user interface, a handwritten input user interface element configured to receive handwritten input for inserting the second respective font-based text between the first portion and the second portion of the second sequence of characters.

### Claim 22 (depends on 21)

The method of claim 21, further comprising:
while displaying the handwritten input user interface element, receiving, via the touch-sensitive display, a second handwritten input in the handwritten input user interface element; and
in response to receiving the second handwritten input in the handwritten input user interface element:
inserting font-based text corresponding to the second handwritten input into the text entry user interface; and
while the handwritten input user interface element remains stationary on the touch-sensitive display, scrolling the text entry user interface in accordance with movement of a current text insertion point in the text entry user interface.

### Claim 23 (depends on 21)

The method of claim 21, further comprising:
while displaying the handwritten input user interface element, receiving, via the touch-sensitive display, a second handwritten input in the handwritten input user interface element; and
in response to receiving the second handwritten input in the handwritten input user interface element:
displaying, in the handwritten input user interface element, a representation of the second handwritten input;
in accordance with a determination that the second handwritten input has not reached an end of the handwritten input user interface element, ceasing to display the representation of the second handwritten input after a first elapsed time since receiving the second handwritten input; and
in accordance with a determination that the second handwritten input has reached the end of the handwritten input user interface element, ceasing to display the representation of the second handwritten input after a second elapsed time, shorter than the first elapsed time, since receiving the second handwritten input.

### Claim 24 (depends on 1)

The method of claim 1, further comprising:
while displaying the text entry user interface including the first sequence of characters, receiving, via the touch-sensitive display, a respective user input including a movement across a respective portion of the first sequence of characters while maintaining contact with the touch-sensitive display at a location between a first character and a second character in the first sequence of characters; and
in response to receiving the respective user input:
in accordance with a determination that no characters separate the first character and the second character in the first sequence of characters, updating the text entry user interface by adding a whitespace character between the first character and the second character in the first sequence of characters; and
in accordance with a determination that only a whitespace character separates the first character and the second character in the first sequence of characters, updating the text entry user interface by removing the whitespace character between the first character and the second character in the first sequence of characters.

### Claim 25

An electronic device, comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
displaying, on a touch-sensitive display in communication with the electronic device, a text entry user interface including a first sequence of characters that includes a first portion of the first sequence of characters and a second portion of the first sequence of characters;
while displaying the text entry user interface, receiving, via the touch-sensitive display, a user input in the text entry user interface in between the first portion of the first sequence of characters and the second portion of the first sequence of characters; and
in response to receiving the user input:
in accordance with a determination that the user input corresponds to a request to enter respective font-based text in between the first portion of the first sequence of characters and the second portion of the first sequence of characters using handwritten input, updating the text entry user interface by creating a space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, wherein the space between the first portion and the second portion is configured to receive the handwritten input for inserting the respective font-based text between the first portion and the second portion of the first sequence of characters.

### Claim 26

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform a method comprising:
displaying, on a touch-sensitive display in communication with the electronic device, a text entry user interface including a first sequence of characters that includes a first portion of the first sequence of characters and a second portion of the first sequence of characters;
while displaying the text entry user interface, receiving, via the touch-sensitive display, a user input in the text entry user interface in between the first portion of the first sequence of characters and the second portion of the first sequence of characters; and
in response to receiving the user input:
in accordance with a determination that the user input corresponds to a request to enter respective font-based text in between the first portion of the first sequence of characters and the second portion of the first sequence of characters using handwritten input, updating the text entry user interface by creating a space between the first portion of the first sequence of characters and the second portion of the first sequence of characters, wherein the space between the first portion and the second portion is configured to receive the handwritten input for inserting the respective font-based text between the first portion and the second portion of the first sequence of characters.
