# US 2017/0286379 A1 · Generating and rendering inflected text

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0286379 A1
- Document ID: US20170286379A1
- Status: Published application
- Kind: A1
- Application number: 15/090,392
- Filed: 2016-04-04
- Published: 2017-10-05
- Earliest claimed: 2016-04-04
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Unnati Jigar Dani; Jiwon Choi; David Nissimoff; Vineeth Karanam
- Web page: https://patentgrep.com/patent/US20170286379A1

## CPC

- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 40/103: Formatting, i.e. changing of presentation of documents (automatic justification G06F40/189; automatic line break hyphenation G06F40/191)
- G06F 40/109: Font handling; Temporal or kinetic typography
- G06F 40/117: Tagging; Marking up (details of markup languages G06F40/143); Designating a block; Setting of attributes (style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154)
- G10L 13/00: Speech synthesis; Text to speech systems
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A facility for using gestures to attach visual inflection to displayed text is described. The facility receives first user input specifying text, and causes the text specified by the first user input to be displayed in a first manner. The facility receives second user input corresponding to a gesture performed with respect to at least a portion of the displayed text, the performed gesture specifying an inflection type. Based at least in part on receiving the second user input, the facility causes the text specified by the first user input to be displayed in a manner that visually reflects application of the inflection type specified by the performed gesture to the at least a portion of the displayed text with respect to which the gesture was performed.

## Claims

### Claim 1

A processor-based device, comprising:
at least one processor; and
memory having contents that, based on execution by the at least one processor, configure the at least one processor to:
receive first input from a user specifying text;
cause the text specified by the first input to be displayed in a first manner;
receive second input from a user corresponding to a gesture performed with respect to some or all of the displayed text, the performed gesture specifying an inflection type; and
based at least in part on receiving the second input, cause the text specified by the first input to be displayed in a manner that visually reflects application of the inflection type specified by the performed gesture to the displayed text with respect to which the gesture was performed.

### Claim 2 (depends on 1)

The device of claim 1, further comprising a touch digitizer, wherein the second input reflects a multi-point touch gesture sensed by the touch digitizer.

### Claim 3 (depends on 1)

The device of claim 1 wherein the gesture to which the received second input corresponds is horizontal pinch in, horizontal pinch out, non-horizontal pinch in, non-horizontal pinch out, vertical pinch in, vertical pinch out, tap, double tap, flick up, flick down, curve, or skew.

### Claim 4 (depends on 1)

The device of claim 1 wherein the inflection type specified by the performed gesture is curious, happy, mad, quiet, loud, swelling, excited, or uncertain.

### Claim 5 (depends on 1)

The device of claim 1, further comprising a speaker, the memory having contents that, based on execution by the at least one processor, configure the at least one processor to further:
cause synthesized speech to be played by the speaker that recites the specified text in a manner that vocally reflects application of the inflection type specified by the performed gesture to the displayed text with respect to which the gesture was performed.

### Claim 6 (depends on 1)

The device of claim 1, the memory having contents that, based on execution by the at least one processor, configure the at least one processor to further:
store in the memory in Speech Synthesis Markup Language format a representation of the specified text in which the displayed text with respect to which the gesture was performed is demarcated with a tag specifying the inflection type specified by the performed gesture.

### Claim 7 (depends on 1)

The device of claim 1, the memory having contents that, based on execution by the at least one processor, configure the at least one processor to further:
after receiving the first input and before receiving the second input, receiving third input from a user selecting the displayed text with respect to which the gesture is performed.

### Claim 8 (depends on 1)

The device of claim 1 wherein causing the text specified by the first input to be displayed in a manner that visually reflects application of the inflection type is performed substantially in real-time relative to receiving the second user input.

### Claim 9 (depends on 1)

The device of claim 1, the memory having contents that, based on execution by the at least one processor, configure the at least one processor to further:
include the text specified by the first input, qualified by the inflection type specified by the performed gesture, to be included in a message transmitted from the processor-based device to a second processor-based device, enabling the second processor-based device to (1) display the text specified by the first input in a manner that visually reflects application of the inflection type specified by the performed gesture to the displayed text with respect to which the gesture was performed, and (2) output synthesized speech that recites the body of text in a manner that vocally reflects application of the inflection type specified by the performed gesture to the displayed text with respect to which the gesture was performed.

### Claim 10 (depends on 1)

The device of claim 1, the memory having contents that, based on execution by the at least one processor, configure the at least one processor to further:
based at least in part on the inflection type specified by the performed gesture, determine a value reflecting the importance of the displayed text with respect to which the gesture was performed within the displayed text; and
evaluate a search query against the displayed text in a manner that considers the determined value.

### Claim 11

A computer-readable medium storing an inflected text data structure, the data structure comprising:
a sequence of characters; and
for each of one or more contiguous portions of the sequence of characters, an indication of an inflection type specified for the contiguous portion of the sequence of characters by performing a user input gesture with respect to the contiguous portion of the sequence of characters, the contents of the data structure being usable to render the sequence of characters in a manner that reflects, for each of the one or more contiguous portions of the sequence of characters, the inflection type specified for the contiguous portion of the sequence of characters.

### Claim 12 (depends on 11)

The computer-readable medium of claim 11 wherein the indications of inflection types each comprise a set of one or more Speech Synthesis Markup Language tags.

### Claim 13 (depends on 11)

The computer-readable medium of claim 11 wherein, for a distinguished one of the contiguous portions, the indicated inflection type reflects an automatic inference as to inflection type based upon at least content of the distinguished portion.

### Claim 14 (depends on 11)

The computer-readable medium of claim 11 wherein, for a distinguished one of the contiguous portions, the indicated inflection type reflects an automatic inference as to inflection type based upon at least (1) content of the distinguished portion, and (2) one or words preceding the distinguished portion in the sequence.

### Claim 15

A computer readable medium having contents configured to cause a computing system to:
access a representation of a body of text, the representation specifying, for each of one or more portions of the body of text, an inflection type applied to the portion;
cause the body of text to be displayed in a manner that, for each portion, visually reflects application of the inflection type specified for the portion to the portion; and
cause synthesized speech to be outputted that recites the body of text in a manner that, for each portion, vocally reflects application of the inflection type specified for the portion.

### Claim 16 (depends on 15)

The computer readable medium of claim 15 wherein the synthesized speech is caused to be outputted at least in part based on receiving user input corresponding to an interaction with the displayed body of text.

### Claim 17 (depends on 15)

The computer readable medium of claim 15 wherein the synthesized speech is caused to be outputted at least in part based on (1) receiving user input corresponding to selecting the displayed body of text, (2) receiving user input corresponding to touching the displayed body of text, (3) receiving user input corresponding to selecting a visual user interface control displayed in connection with the body of text, (4) receiving user input corresponding to touching a visual user interface control displayed in connection with the body of text, or (5) receiving the representation of the body of text from another device.

### Claim 18 (depends on 15)

The computer readable medium of claim 15 wherein causing synthesized speech to be outputted comprises calling a text-to-speech API function, passing parameters separately specifying portions of the body of text and the inflection type specified for each portion.

### Claim 19 (depends on 15)

The computer readable medium of claim 15 wherein causing synthesized speech to be outputted comprises calling a text-to-speech API function, passing a version of the body of text containing markup language tags conveying the inflection type specified for each portion.

### Claim 20 (depends on 15)

The computer readable medium of claim 15 wherein, for a selected one of the portions of the body of text, the inflection type specified by the representation was selected from a palette of inflection types.
