# US 11,307,763 B2 · Portable touch screen device, method, and graphical user interface for using emoji characters

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,307,763 B2
- Document ID: US11307763B2
- Status: Granted
- Kind: B2
- Application number: 17/210,255
- Filed: 2021-03-23
- Published: 2021-07-22
- Granted: 2022-04-19
- Earliest claimed: 2008-11-19
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Bradford Allen Moore; Bas Ording; Elizabeth Caroline Cranfill; Stephen J. Chick; Kenneth Kocienda
- Web page: https://patentgrep.com/patent/US11307763B2

## CPC

- G06F 3/04883: for inputting data by handwriting, e.g. gesture or text
- G06F 3/0219: {Special purpose keyboards}
- G06F 3/0236: {using selection techniques to select from displayed items}
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04886: by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
- H04M 1/7243: with interactive means for internal management of messages
- H04M 2250/22: including a touch pad, a touch sensor or a touch detector
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

In some embodiments, a computer-implemented method performed at a portable electronic device with a touch screen display includes simultaneously displaying a character input area operable to display text character input and emoji character input selected by a user, a keyboard display area, and a plurality of emoji category icons. In response to detecting a gesture on a respective emoji category icon, the method also includes simultaneously displaying: a first subset of emoji character keys for the respective emoji category in the keyboard display area and a plurality of subset-sequence-indicia icons for the respective emoji category. The method also includes detecting a gesture in the keyboard display area and, in response: replacing display of the first subset of emoji character keys with display of a second subset of emoji character keys for the respective emoji category, and updating the information provided by the subset-sequence-indicia icons.

## Claims

### Claim 1

An electronic device, comprising:
a touch screen display;
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
simultaneously displaying:
a character input area operable to display text character input and emoji character input selected by a user,
a keyboard display area operable to display text character keys and emoji character keys,
a first plurality of emoji character keys in the keyboard display area, and
a plurality of emoji category icons in the keyboard display area, wherein the emoji category icons correspond to a plurality of emoji categories;
detecting a gesture on a respective emoji category icon in the plurality of emoji category icons;
in response to detecting the gesture on the respective emoji category icon:
maintaining display of the character input area;
maintaining display of the plurality of emoji category icons in the keyboard display area;
ceasing to display the first plurality of emoji character keys in the keyboard display area; and
displaying a first subset of emoji character keys for the respective emoji category in the keyboard display area, wherein the respective emoji category contains a number of subsets of emoji character keys that are configured to be separately displayed as a sequence of subsets of emoji character keys in the keyboard display area;
while displaying the first subset of emoji character keys for the respective emoji category in the keyboard display area, detecting a gesture in the keyboard display area; and
in response to detecting the gesture in the keyboard display area:
maintaining display of the character input area;
maintaining display of the plurality of emoji category icons in the keyboard display area; and
replacing display of the first subset of emoji character keys for the respective emoji category with display of a second subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 2 (depends on 1)

The electronic device of claim 1, wherein the plurality of emoji category icons includes a recently and frequently used emoji category icon that corresponds to a plurality of emoji characters recently and frequently selected by the user.

### Claim 3 (depends on 1)

The electronic device of claim 1, wherein emoji character keys in at least one emoji category are ordered in accordance with emoji character usage data.

### Claim 4 (depends on 3)

The electronic device of claim 3, wherein the emoji character usage data includes emoji character usage data from software applications running on the electronic device.

### Claim 5 (depends on 3)

The electronic device of claim 3, wherein the emoji character usage data includes emoji character usage data received from a service provider of the electronic device.

### Claim 6 (depends on 1)

The electronic device of claim 1, wherein replacing display of the first subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area comprises displaying an animation of the first subset of emoji character keys moving off the touch screen display.

### Claim 7 (depends on 1)

The electronic device of claim 1, wherein replacing display of the first subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area comprises displaying an animation of the second subset of emoji character keys moving on the touch screen display.

### Claim 8 (depends on 1)

The electronic device of claim 1, wherein replacing display of the first subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area comprises displaying an animation of the first subset of emoji character keys moving off the touch screen display and displaying an animation of the second subset of emoji character keys moving on the touch screen display.

### Claim 9 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
while displaying the second subset of emoji character keys for the respective emoji category in the keyboard display area, detecting a gesture in the keyboard display area; and
in response to detecting the gesture in the keyboard display area:
replacing display of the second subset of emoji character keys for the respective emoji category with display of a third subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 10 (depends on 9)

The electronic device of claim 9, wherein the gesture detected while displaying the second subset of emoji character keys is a swipe gesture in a first direction; and the one or more programs further including instructions for:
detecting a second swipe gesture on the touch screen display in the keyboard display area in a direction that is opposite the first direction; and
in response to detecting the second swipe gesture in the keyboard display area:
replacing display of the third subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 11 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
displaying a keyboard selection icon;
detecting a gesture on the keyboard selection icon; and
in response to detecting the gesture on the keyboard selection icon, replacing display of the second subset of emoji character keys for the respective emoji category and the plurality of emoji category icons with display of a text character input keyboard in the keyboard display area.

### Claim 12 (depends on 11)

The electronic device of claim 11, the one or more programs further including instructions for:
receiving one or more words input by the user with the text character input keyboard;
displaying the one or more words input by the user in the character input area; and
in response to receiving the one or more words input by the user with the text character input keyboard:
determining a recommended emoji character to be inserted after the one or more words input by the user based at least in part on the one or more words input by the user, and
displaying the recommended emoji character on the touch screen display.

### Claim 13 (depends on 12)

The electronic device of claim 12, the one or more programs further including instructions for:
detecting a gesture on the touch screen display; and
in response to detecting the gesture, inserting and displaying the recommended emoji character after the one or more words input by the user in the character input area.

### Claim 14 (depends on 11)

The electronic device of claim 11, the one or more programs further including instructions for:
displaying the keyboard selection icon while the text character input keyboard is displayed in the keyboard display area;
detecting a gesture on the keyboard selection icon; and
in response to detecting the gesture on the keyboard selection icon, replacing display of the text character input keyboard with simultaneous display of the second subset of emoji character keys for the respective emoji category in the keyboard display area and the plurality of emoji category icons in the keyboard display area.

### Claim 15 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
detecting a gesture on a respective emoji character key in the second subset of emoji character keys; and
in response to detecting the gesture on the respective emoji character key in the second subset of emoji character keys, inserting and displaying a corresponding emoji character in the character input area.

### Claim 16 (depends on 1)

The electronic device of claim 1, the one or more programs further including instructions for:
detecting a predefined user action for initiating a predefined user interface reconfiguration process; and
in response to detecting the predefined user action, varying positions of two or more emoji character keys in the second subset of emoji character keys, wherein the varying includes varying the positions of each emoji character key of the two or more emoji character keys about a respective average position distinct from the respective average positions of other emoji character keys of the two or more emoji character keys.

### Claim 17 (depends on 16)

The electronic device of claim 16, wherein each respective emoji character key of the two or more emoji character keys oscillates in a region centered on the respective average position of the respective emoji character key.

### Claim 18 (depends on 16)

The electronic device of claim 16, the one or more programs further including instructions for:
detecting the user making a point of contact with the touch screen display at a first position corresponding to a first emoji character key in the second subset of emoji character keys and detecting movement of the point of contact to a second position on the touch screen display; and
in response to detecting the point of contact and detecting movement of the point of contact, displaying movement of the first emoji character key to the second position on the touch screen display in accordance with the detected movement of the point of contact.

### Claim 19 (depends on 18)

The electronic device of claim 18, the one or more programs further including instructions for fixing a position of the first emoji character key at the second position in response to detecting a second predefined user action for terminating the predefined user interface reconfiguration process.

### Claim 20 (depends on 1)

The electronic device of claim 1, wherein the keyboard display area is operable to display the text character keys in a text character input keyboard to replace display of a respective subset of emoji character keys and the plurality of emoji category icons.

### Claim 21 (depends on 1)

The electronic device of claim 1, wherein the gesture detected in the keyboard display area while displaying the first subset of emoji character keys for the respective emoji category in the keyboard display area is a swipe gesture.

### Claim 22

A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a touch screen display, the one or more programs including instructions for:
simultaneously displaying:
a character input area operable to display text character input and emoji character input selected by a user,
a keyboard display area operable to display text character keys and emoji character keys,
a first plurality of emoji character keys in the keyboard display area, and
a plurality of emoji category icons in the keyboard display area, wherein the emoji category icons correspond to a plurality of emoji categories;
detecting a gesture on a respective emoji category icon in the plurality of emoji category icons;
in response to detecting the gesture on the respective emoji category icon:
maintaining display of the character input area;
maintaining display of the plurality of emoji category icons in the keyboard display area;
ceasing to display the first plurality of emoji character keys in the keyboard display area; and
displaying a first subset of emoji character keys for the respective emoji category in the keyboard display area, wherein the respective emoji category contains a number of subsets of emoji character keys that are configured to be separately displayed as a sequence of subsets of emoji character keys in the keyboard display area;
while displaying the first subset of emoji character keys for the respective emoji category in the keyboard display area, detecting a gesture in the keyboard display area; and
in response to detecting the gesture in the keyboard display area:
maintaining display of the character input area;
maintaining display of the plurality of emoji category icons in the keyboard display area; and
replacing display of the first subset of emoji character keys for the respective emoji category with display of a second subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 23 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein the plurality of emoji category icons includes a recently and frequently used emoji category icon that corresponds to a plurality of emoji characters recently and frequently selected by the user.

### Claim 24 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein emoji character keys in at least one emoji category are ordered in accordance with emoji character usage data.

### Claim 25 (depends on 24)

The non-transitory computer-readable storage medium of claim 24, wherein the emoji character usage data includes emoji character usage data from software applications running on the electronic device.

### Claim 26 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein replacing display of the first subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area comprises displaying an animation of the first subset of emoji character keys moving off the touch screen display.

### Claim 27 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein replacing display of the first subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area comprises displaying an animation of the second subset of emoji character keys moving on the touch screen display.

### Claim 28 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, the one or more programs further including instructions for:
while displaying the second subset of emoji character keys for the respective emoji category in the keyboard display area, detecting a gesture in the keyboard display area; and
in response to detecting the gesture in the keyboard display area:
replacing display of the second subset of emoji character keys for the respective emoji category with display of a third subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 29 (depends on 28)

The non-transitory computer-readable storage medium of claim 28, wherein the gesture detected while displaying the second subset of emoji character keys is a swipe gesture in a first direction; and the one or more programs further including instructions for:
detecting a second swipe gesture on the touch screen display in the keyboard display area in a direction that is opposite the first direction; and
in response to detecting the second swipe gesture in the keyboard display area:
replacing display of the third subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 30 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, the one or more programs further including instructions for:
displaying a keyboard selection icon;
detecting a gesture on the keyboard selection icon; and
in response to detecting the gesture on the keyboard selection icon, replacing display of the second subset of emoji character keys for the respective emoji category and the plurality of emoji category icons with display of a text character input keyboard in the keyboard display area.

### Claim 31 (depends on 30)

The non-transitory computer-readable storage medium of claim 30, the one or more programs further including instructions for:
receiving one or more words input by the user with the text character input keyboard;
displaying the one or more words input by the user in the character input area; and
in response to receiving the one or more words input by the user with the text character input keyboard:
determining a recommended emoji character to be inserted after the one or more words input by the user based at least in part on the one or more words input by the user, and
displaying the recommended emoji character on the touch screen display.

### Claim 32 (depends on 31)

The non-transitory computer-readable storage medium of claim 31, the one or more programs further including instructions for:
detecting a gesture on the touch screen display; and
in response to detecting the gesture, inserting and displaying the recommended emoji character after the one or more words input by the user in the character input area.

### Claim 33 (depends on 30)

The non-transitory computer-readable storage medium of claim 30, the one or more programs further including instructions for:
displaying the keyboard selection icon while the text character input keyboard is displayed in the keyboard display area;
detecting a gesture on the keyboard selection icon; and
in response to detecting the gesture on the keyboard selection icon, replacing display of the text character input keyboard with simultaneous display of the second subset of emoji character keys for the respective emoji category in the keyboard display area and the plurality of emoji category icons in the keyboard display area.

### Claim 34 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, the one or more programs further including instructions for:
detecting a gesture on a respective emoji character key in the second subset of emoji character keys; and
in response to detecting the gesture on the respective emoji character key in the second subset of emoji character keys, inserting and displaying a corresponding emoji character in the character input area.

### Claim 35 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein the keyboard display area is operable to display the text character keys in a text character input keyboard to replace display of a respective subset of emoji character keys and the plurality of emoji category icons.

### Claim 36 (depends on 22)

The non-transitory computer-readable storage medium of claim 22, wherein the gesture detected in the keyboard display area while displaying the first subset of emoji character keys for the respective emoji category in the keyboard display area is a swipe gesture.

### Claim 37

A computer-implemented method, comprising:
at an electronic device with a touch screen display:
simultaneously displaying:
a character input area operable to display text character input and emoji character input selected by a user,
a keyboard display area operable to display text character keys and emoji character keys,
a first plurality of emoji character keys in the keyboard display area, and
a plurality of emoji category icons in the keyboard display area, wherein the emoji category icons correspond to a plurality of emoji categories;
detecting a gesture on a respective emoji category icon in the plurality of emoji category icons;
in response to detecting the gesture on the respective emoji category icon:
maintaining display of the character input area;
maintaining display of the plurality of emoji category icons in the keyboard display area;
ceasing to display the first plurality of emoji character keys in the keyboard display area; and
displaying a first subset of emoji character keys for the respective emoji category in the keyboard display area, wherein the respective emoji category contains a number of subsets of emoji character keys that are configured to be separately displayed as a sequence of subsets of emoji character keys in the keyboard display area;
while displaying the first subset of emoji character keys for the respective emoji category in the keyboard display area, detecting a gesture in the keyboard display area; and
in response to detecting the gesture in the keyboard display area:
maintaining display of the character input area;
maintaining display of the plurality of emoji category icons in the keyboard display area; and
replacing display of the first subset of emoji character keys for the respective emoji category with display of a second subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 38 (depends on 37)

The method of claim 37, wherein the plurality of emoji category icons includes a recently and frequently used emoji category icon that corresponds to a plurality of emoji characters recently and frequently selected by the user.

### Claim 39 (depends on 37)

The method of claim 37, wherein emoji character keys in at least one emoji category are ordered in accordance with emoji character usage data.

### Claim 40 (depends on 39)

The method of claim 39, wherein the emoji character usage data includes emoji character usage data from software applications running on the electronic device.

### Claim 41 (depends on 37)

The method of claim 37, wherein replacing display of the first subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area comprises displaying an animation of the first subset of emoji character keys moving off the touch screen display.

### Claim 42 (depends on 37)

The method of claim 37, wherein replacing display of the first subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area comprises displaying an animation of the second subset of emoji character keys moving on the touch screen display.

### Claim 43 (depends on 37)

The method of claim 37, further comprising:
while displaying the second subset of emoji character keys for the respective emoji category in the keyboard display area, detecting a gesture in the keyboard display area; and
in response to detecting the gesture in the keyboard display area:
replacing display of the second subset of emoji character keys for the respective emoji category with display of a third subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 44 (depends on 43)

The method of claim 43, wherein the gesture detected while displaying the second subset of emoji character keys is a swipe gesture in a first direction; and further comprising:
detecting a second swipe gesture on the touch screen display in the keyboard display area in a direction that is opposite the first direction; and
in response to detecting the second swipe gesture in the keyboard display area:
replacing display of the third subset of emoji character keys for the respective emoji category with display of the second subset of emoji character keys for the respective emoji category in the keyboard display area.

### Claim 45 (depends on 37)

The method of claim 37, further comprising:
displaying a keyboard selection icon;
detecting a gesture on the keyboard selection icon; and
in response to detecting the gesture on the keyboard selection icon, replacing display of the second subset of emoji character keys for the respective emoji category and the plurality of emoji category icons with display of a text character input keyboard in the keyboard display area.

### Claim 46 (depends on 45)

The method of claim 45, further comprising:
receiving one or more words input by the user with the text character input keyboard;
displaying the one or more words input by the user in the character input area; and
in response to receiving the one or more words input by the user with the text character input keyboard:
determining a recommended emoji character to be inserted after the one or more words input by the user based at least in part on the one or more words input by the user, and
displaying the recommended emoji character on the touch screen display.

### Claim 47 (depends on 46)

The method of claim 46, further comprising:
detecting a gesture on the touch screen display; and
in response to detecting the gesture, inserting and displaying the recommended emoji character after the one or more words input by the user in the character input area.

### Claim 48 (depends on 45)

The method of claim 45, further comprising:
displaying the keyboard selection icon while the text character input keyboard is displayed in the keyboard display area;
detecting a gesture on the keyboard selection icon; and
in response to detecting the gesture on the keyboard selection icon, replacing display of the text character input keyboard with simultaneous display of the second subset of emoji character keys for the respective emoji category in the keyboard display area and the plurality of emoji category icons in the keyboard display area.

### Claim 49 (depends on 37)

The method of claim 37, further comprising:
detecting a gesture on a respective emoji character key in the second subset of emoji character keys; and
in response to detecting the gesture on the respective emoji character key in the second subset of emoji character keys, inserting and displaying a corresponding emoji character in the character input area.

### Claim 50 (depends on 37)

The method of claim 37, wherein the keyboard display area is operable to display the text character keys in a text character input keyboard to replace display of a respective subset of emoji character keys and the plurality of emoji category icons.

### Claim 51 (depends on 37)

The method of claim 37, wherein the gesture detected in the keyboard display area while displaying the first subset of emoji character keys for the respective emoji category in the keyboard display area is a swipe gesture.