# US 10,311,144 B2 · Emoji word sense disambiguation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,144 B2
- Document ID: US10311144B2
- Status: Granted
- Kind: B2
- Application number: 15/678,618
- Filed: 2017-08-16
- Published: 2018-11-22
- Granted: 2019-06-04
- Earliest claimed: 2017-05-16
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Jerome R. Bellegarda; Bishal Barman
- Web page: https://patentgrep.com/patent/US10311144B2

## CPC

- G06F 40/30: Semantic analysis
- G06F 3/0237: {using prediction or retrieval techniques}
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure generally relates to systems and processes for emoji word sense disambiguation. In one example process, a word sequence is received. A word-level feature representation is determined for each word of the word sequence and a global semantic representation for the word sequence is determined. For a first word of the word sequence, an attention coefficient is determined based on a congruence between the word-level feature representation of the first word and the global semantic representation for the word sequence. The word-level feature representation of the first word is adjusted based on the attention coefficient. An emoji likelihood is determined based on the adjusted word-level feature representation of the first word. In accordance with the emoji likelihood satisfying one or more criteria, an emoji character corresponding to the first word is presented for display.

## Claims

### Claim 1

An electronic device, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving natural language input comprising a word sequence;
for each respective word of the word sequence, determining a feature representation of the respective word based on a left word context for the respective word and the right word context for the respective word;
determining a global semantic representation for the word sequence based on the left word context and the right word context for each respective word of the word sequence; and
for each respective word of the word sequence that corresponds to a predefined word of a predefined emoji character:
determining a respective attention coefficient based on a congruence between the feature representation of the respective word of the word sequence and the global semantic representation for the word sequence;
adjusting the feature representation of the respective word of the word sequence based on the determined respective attention coefficient;
determining a respective emoji likelihood based on the adjusted feature representation of the respective word of the word sequence; and
in accordance with the respective emoji likelihood satisfying one or more criteria, causing an emoji character corresponding to the respective word of the word sequence to be presented for display.

### Claim 2 (depends on 1)

The device of claim 1, wherein a word of the word sequence that corresponds to a predefined word of a predefined emoji character has a plurality of semantic meanings.

### Claim 3 (depends on 1)

The device of claim 1, wherein the one or more programs further include instructions for:
for each respective word of the word sequence, determining the left word context for the respective word based on the respective word and a respective previous left word context.

### Claim 4 (depends on 3)

The device of claim 3, wherein for each respective word of the word sequence, the left word context for the respective word is determined by combining a vector representation for the respective word with the respective previous left word context.

### Claim 5 (depends on 3)

The device of claim 3, wherein for each respective word of the word sequence, the respective previous left word context comprises a vector representation of a portion of the word sequence received prior to the respective word.

### Claim 6 (depends on 1)

The device of claim 1, wherein the one or more programs further include instructions for:
for each respective word of the word sequence, determining the right word context for the respective word based on the respective word and a respective subsequent right word context.

### Claim 7 (depends on 6)

The device of claim 6, wherein for each respective word of the word sequence, the right word context for the respective word is determined by combining a vector representation for the respective word with the respective subsequent right word context.

### Claim 8 (depends on 1)

The device of claim 1, wherein for each respective word of the word sequence, the respective subsequent right word context comprises a vector representation of a portion of the word sequence received after the respective word.

### Claim 9 (depends on 1)

The device of claim 1, wherein for each respective word of the word sequence, the left word context and the right word context are each vectors of a same dimension.

### Claim 10 (depends on 1)

The device of claim 1, wherein for each respective word of the word sequence, the feature representation of the respective word is determined by concatenating or adding the left word context for the respective word and the right word context for the respective word.

### Claim 11 (depends on 1)

The device of claim 1, wherein the global semantic representation for the word sequence is determined using a hierarchy of stages comprising at least one pooling layer and at least one long short-term memory (LSTM) network layer.

### Claim 12 (depends on 11)

The device of claim 11, wherein a pooling layer of the at least one pooling layer is configured to:
combine respective left word contexts for adjacent word pairs in the word sequence; and
combine respective right word contexts for adjacent word pairs in the word sequence.

### Claim 13 (depends on 1)

The device of claim 1, wherein the respective attention coefficient for a respective word of the word sequence represents a degree of semantic similarity between the word sequence and the respective word.

### Claim 14 (depends on 1)

The device of claim 1, wherein the respective attention coefficient for a respective word of the word sequence is determined based on a product of a transpose of the global semantic representation for the word sequence and the feature representation of the respective word.

### Claim 15 (depends on 1)

The device of claim 1, wherein the emoji character corresponding to the respective word of the word sequence is an ideogram.

### Claim 16 (depends on 1)

The device of claim 1, wherein the one or more programs further include instructions for:
determining whether one or more input criteria related to the natural language input is satisfied, wherein the feature representation of each respective word of the word sequence is determined in response to determining that the one or more input criteria related to the natural language input is satisfied.

### Claim 17

A method for emoji word sense disambiguation, the method comprising:
at an electronic device:
receiving natural language input comprising a word sequence;
for each respective word of the word sequence, determining a feature representation of the respective word based on a left word context for the respective word and the right word context for the respective word;
determining a global semantic representation for the word sequence based on the left word context and the right word context for each respective word of the word sequence; and
for each respective word of the word sequence that corresponds to a predefined word of a predefined emoji character:
determining a respective attention coefficient based on a congruence between the feature representation of the respective word of the word sequence and the global semantic representation for the word sequence;
adjusting the feature representation of the respective word of the word sequence based on the determined respective attention coefficient;
determining a respective emoji likelihood based on the adjusted feature representation of the respective word of the word sequence; and
in accordance with the respective emoji likelihood satisfying one or more criteria, causing an emoji character corresponding to the respective word of the word sequence to be presented for display.

### Claim 18 (depends on 17)

The method of claim 17, wherein a word of the word sequence that corresponds to a predefined word of a predefined emoji character has a plurality of semantic meanings.

### Claim 19 (depends on 17)

The method of claim 17, further comprising:
for each respective word of the word sequence, determining the left word context for the respective word based on the respective word and a respective previous left word context.

### Claim 20 (depends on 19)

The method of claim 19, wherein for each respective word of the word sequence, the left word context for the respective word is determined by combining a vector representation for the respective word with the respective previous left word context.

### Claim 21 (depends on 19)

The method of claim 19, wherein for each respective word of the word sequence, the respective previous left word context comprises a vector representation of a portion of the word sequence received prior to the respective word.

### Claim 22 (depends on 17)

The method of claim 17, further comprising:
for each respective word of the word sequence, determining the right word context for the respective word based on the respective word and a respective subsequent right word context.

### Claim 23 (depends on 22)

The method of claim 22, wherein for each respective word of the word sequence, the right word context for the respective word is determined by combining a vector representation for the respective word with the respective subsequent right word context.

### Claim 24 (depends on 17)

The method of claim 17, wherein for each respective word of the word sequence, the respective subsequent right word context comprises a vector representation of a portion of the word sequence received after the respective word.

### Claim 25 (depends on 17)

The method of claim 17, wherein for each respective word of the word sequence, the left word context and the right word context are each vectors of a same dimension.

### Claim 26 (depends on 17)

The method of claim 17, wherein for each respective word of the word sequence, the feature representation of the respective word is determined by concatenating or adding the left word context for the respective word and the right word context for the respective word.

### Claim 27 (depends on 17)

The method of claim 17, wherein the global semantic representation for the word sequence is determined using a hierarchy of stages comprising at least one pooling layer and at least one long short-term memory (LSTM) network layer.

### Claim 28 (depends on 27)

The method of claim 27, wherein a pooling layer of the at least one pooling layer configured to:
combine respective left word contexts for adjacent word pairs in the word sequence; and
combine respective right word contexts for adjacent word pairs in the word sequence.

### Claim 29 (depends on 17)

The method of claim 17, wherein the respective attention coefficient for a respective word of the word sequence represents a degree of semantic similarity between the word sequence and the respective word.

### Claim 30 (depends on 17)

The method of claim 17, wherein the respective attention coefficient for a respective word of the word sequence is determined based on a product of a transpose of the global semantic representation for the word sequence and the feature representation of the respective word.

### Claim 31 (depends on 17)

The method of claim 17, wherein the emoji character corresponding to the respective word of the word sequence is an ideogram.

### Claim 32 (depends on 17)

The method of claim 17, further comprising:
determining whether one or more input criteria related to the natural language input satisfied, wherein the feature representation of each respective word of the word sequence is determined in response to determining that the one or more input criteria related to the natural language input is satisfied.

### Claim 33

A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a display and memory, the one or more programs including instructions for:
receiving natural language input comprising a word sequence;
for each respective word of the word sequence, determining a feature representation of the respective word based on a left word context for the respective word and the right word context for the respective word;
determining a global semantic representation for the word sequence based on the left word context and the right word context for each respective word of the word sequence; and
for each respective word of the word sequence that corresponds to a predefined word of a predefined emoji character:
determining a respective attention coefficient based on a congruence between the feature representation of the respective word of the word sequence and the global semantic representation for the word sequence;
adjusting the feature representation of the respective word of the word sequence based on the determined respective attention coefficient;
determining a respective emoji likelihood based on the adjusted feature representation of the respective word of the word sequence; and
in accordance with the respective emoji likelihood satisfying one or more criteria, causing an emoji character corresponding to the respective word of the word sequence to be presented for display.

### Claim 34 (depends on 33)

The computer-readable storage medium of claim 33, wherein a word of the word sequence that corresponds to a predefined word of a predefined emoji character has a plurality of semantic meanings.

### Claim 35 (depends on 33)

The computer-readable storage medium of claim 33, wherein the one or more programs further include instructions for:
for each respective word of the word sequence, determining the left word context for the respective word based on the respective word and a respective previous left word context.

### Claim 36 (depends on 35)

The computer-readable storage medium of claim 35, wherein for each respective word of the word sequence, the left word context for the respective word is determined by combining a vector representation for the respective word with the respective previous left word context.

### Claim 37 (depends on 35)

The computer-readable storage medium of claim 35, wherein for each respective word of the word sequence, the respective previous left word context comprises a vector representation of a portion of the word sequence received prior to the respective word.

### Claim 38 (depends on 33)

The computer-readable storage medium of claim 33, wherein the one or more programs further include instructions for:
for each respective word of the word sequence, determining the right word context for the respective word based on the respective word and a respective subsequent right word context.

### Claim 39 (depends on 38)

The computer-readable storage medium of claim 38, wherein for each respective word of the word sequence, the right word context for the respective word is determined by combining a vector representation for the respective word with the respective subsequent right word context.

### Claim 40 (depends on 33)

The computer-readable storage medium of claim 33, wherein for each respective word of the word sequence, the respective subsequent right word context comprises a vector representation of a portion of the word sequence received after the respective word.

### Claim 41 (depends on 33)

The computer-readable storage medium of claim 33, wherein for each respective word of the word sequence, the left word context and the right word context are each vectors of a same dimension.

### Claim 42 (depends on 33)

The computer-readable storage medium of claim 33, wherein for each respective word of the word sequence, the feature representation of the respective word is determined by concatenating or adding the left word context for the respective word and the right word context for the respective word.

### Claim 43 (depends on 33)

The computer-readable storage medium of claim 33, wherein the global semantic representation for the word sequence is determined using a hierarchy of stages comprising at least one pooling layer and at least one long short-term memory (LSTM) network layer.

### Claim 44 (depends on 43)

The computer-readable storage medium of claim 43, wherein a pooling layer of the at least one pooling layer is configured to:
combine respective left word contexts for adjacent word pairs in the word sequence; and
combine respective right word contexts for adjacent word pairs in the word sequence.

### Claim 45 (depends on 33)

The computer-readable storage medium of claim 33, wherein the respective attention coefficient for a respective word of the word sequence represents a degree of semantic similarity between the word sequence and the respective word.

### Claim 46 (depends on 33)

The computer-readable storage medium of claim 33, wherein the respective attention coefficient for a respective word of the word sequence is determined based on a product of a transpose of the global semantic representation for the word sequence and the feature representation of the respective word.

### Claim 47 (depends on 33)

The computer-readable storage medium of claim 33, wherein the emoji character corresponding to the respective word of the word sequence is an ideogram.

### Claim 48 (depends on 33)

The computer-readable storage medium of claim 33, wherein the one or more programs further include instructions for:
determining whether one or more input criteria related to the natural language input is satisfied, wherein the feature representation of each respective word of the word sequence is determined in response to determining that the one or more input criteria related to the natural language input is satisfied.