# US 11,122,335 B2 · Wishing television

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,122,335 B2
- Document ID: US11122335B2
- Status: Granted
- Kind: B2
- Application number: 16/693,204
- Filed: 2019-11-22
- Published: 2021-05-27
- Granted: 2021-09-14
- Earliest claimed: 2019-11-22
- Assignees: Tcl Research America Inc.
- Applicants: Tcl Research America Inc.
- Inventors: Haohong Wang
- Web page: https://patentgrep.com/patent/US11122335B2

## CPC

- H04N 21/47205: {for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally}
- H04N 21/8541: involving branching, e.g. to different story endings
- G06F 16/40: of multimedia data, e.g. slideshows comprising image and additional audio data (retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70)
- G10L 15/20: Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence)
- H04N 21/2393: {involving handling client requests (scheduling or organising the servicing of application requests H04L67/60)}
- H04N 21/262: Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists {(scheduling strategies for dispatcher in multiprogramming arrangements G06F9/4881; arrangements for scheduling broadcast services or broadcast-related services H04H60/06; flow control in packet networks H04L47/10; establishing a time schedule or organising the servicing of application requests H04L67/62)}
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/47202: {for requesting content on demand, e.g. video on demand}
- H04N 21/4755: {for defining user preferences, e.g. favourite actors or genre (retrieval personalisation and generation of user profiles for the retrieval of video data G06F16/739; user profiles in network data switching protocols H04L67/306)}
- H04N 21/4882: {for displaying messages, e.g. warnings, reminders (arrangements for providing short real-time information to substation in data switching networks H04L12/1895)}
- G10L 2015/223: {Execution procedure of a spoken command}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

An interaction method includes receiving an interaction indicating a user's wish, interpreting the interaction to obtain an interpreted wish, identifying a realization story in a hyperstory according to the interpreted wish, sending a feedback message indicating a time the user's wish will be realized and a response-to-wish confidence level, generating a realization video according to the realization story, and outputting the realization video.

## Claims

### Claim 1

An interaction method, comprising:
receiving an interaction indicating a user's wish;
interpreting the interaction to obtain an interpreted wish;
identifying a realization story in a hyperstory according to the interpreted wish;
sending a feedback message indicating a time the user's wish will be realized and a response-to-wish confidence level, wherein the feedback message includes a realization latency, the response-to-wish confidence level, and a quality level;
generating a realization video according to the realization story; and
outputting the realization video.

### Claim 2 (depends on 1)

The method of claim 1, wherein interpreting the interaction to obtain the interpreted wish includes:
interpreting the interaction to derive wish information about who, what, when, and where; and
converting the wish information to the interpreted wish, the interpret wish including a template of wish command consistent with the derived information of who, what, when, and where.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
obtaining the hyperstory representing status of one or more characters and events causing status changes of the one or more characters.

### Claim 4 (depends on 3)

The method of claim 3, wherein obtaining the hyperstory includes:
representing the hyperstory using a directed acyclic graph (DAG), each vertex in the DAG representing a status of a character and an edge of any connected vertex pair representing an event causing status changes of the character.

### Claim 5 (depends on 4)

The method of claim 4, wherein identifying the realization story in the hyperstory according to the interpreted wish includes:
determining a realization vertex satisfying the interpreted wish and a realization path from a wishing point vertex to the realization vertex from existing story branches in the hyperstory based on a realization latency, a quality level, and the response-to-wish confidence level, the wishing point vertex being a vertex close to a point when the interaction is received.

### Claim 6 (depends on 5)

The method of claim 5, wherein determining the realization vertex and the realization path includes:
determining the realization vertex and the realization path from the existing story branches in the hyperstory according to a wish-realization-distance (WRD).

### Claim 7 (depends on 6)

The method of claim 6, wherein determining the realization vertex and the realization path according to the WRD includes:
for each possible realization vertex in the hyperstory:
determining a possible realization path corresponding to the realization video with a minimum time period;
determining a matching degree between the interpreted wish and the possible realization path;
calculating the quality level of the possible realization path; and
calculating a WRD function of the possible realization path according to the obtained minimum time period, the matching degree of the possible realization path, and the quality level of the possible realization path; and
determining the realization vertex and the realization path corresponding to a minimum value of WRD as the realization vertex and the realization path.

### Claim 8 (depends on 7)

The method of claim 7, wherein determining the possible realization path corresponding to the realization video with the minimum time period includes:
obtaining the edges bridging the wishing point vertex to the realization vertex by calculating the minimum time period; and
determining the possible realization path as a path of obtained edges.

### Claim 9 (depends on 7)

The method of claim 7, wherein calculating the minimum time period includes:
calculating the minimum time period T(Vxy, Vrx,ry) by recursively calculating Minimum [T(Vx,y, Vx+1,yy)+T(Vx+1,yy, Vrx,ry)] for all vertices Vx+1,yy connected directly to Vx,y, Vx,y being the wishing point vertex, the Vrx,ry being the possible realization vertex, Vx+1,yy being a vertex directly connected to the Vxy.

### Claim 10 (depends on 7)

The method of claim 7, wherein determining the matching degree between the interpreted wish and the possible realization path includes:
calculating a trajectory of the character's status change in the realization path to determine the matching degree between the interpreted wish and the possible realization path.

### Claim 11 (depends on 7)

The method of claim 7, wherein calculating the quality level of the possible realization path includes:
calculating the quality level of the possible realization path as a minimum value of the quality levels of steps within the possible realization path.

### Claim 12 (depends on 7)

The method of claim 7, wherein calculating the WRD function of the possible realization path includes:
calculating the WRD function as a sum of a matching degree weight times a difference between one and the matching degree of the possible realization path, a latency weight times a time period of video corresponding to the possible realization path, and a quality level weight times a difference between an upper-bound threshold for the quality level and the quality level of the possible realization path, the quality level weight being one minus the matching degree weight and the latency weight, and the matching degree weight, the latency weigh, and the quality level weight being in a range of 0 to 1.

### Claim 13 (depends on 1)

The method of claim 1, wherein generating the realization video according to the realization story includes:
retrieving steps of the realization story in the hyperstory, according to the realization story, including an index of the wishing point vertex, an index of the determined realization vertex, and indexes of other vertices in the determined path; and
generating the realization video from the steps of the realization story using an auto-cinematography method.

### Claim 14

A television (TV) comprising:
a processor;
a memory coupled to the processor and storing instructions that, when executed by the processor, cause the processor to:
receive an interaction indicating a user's wish;
interpret the interaction to obtain an interpreted wish;
identify a realization story in a hyperstory according to the interpreted wish;
send a feedback message indicating a time the user's wish will be realized and a response-to-wish confidence level, wherein the feedback message includes a realization latency, the response-to-wish confidence level, and a quality level;
generate a realization video according to the realization story; and
output the realization video; and
a screen coupled to the memory and configured to display the realization video.

### Claim 15 (depends on 14)

The TV of claim 14, wherein the instructions further cause the processor to:
interpret the interaction to derive wish information about who, what, when, and where; and
convert the wish information to the interpreted wish, the interpret wish including a template of wish command consistent with the derived wish information of who, what, when, and where.

### Claim 16 (depends on 14)

The TV of claim 14, wherein the instructions further cause the processor to:
obtain the hyperstory representing status of one or more characters and events causing status changes of the one or more characters.

### Claim 17 (depends on 16)

The TV of claim 16, wherein the instructions further cause the processor to:
represent the hyperstory using a directed acyclic graph (DAG), each vertex in the DAG representing a status of a character and an edge of any connected vertex pair representing an event causing status changes of the character.

### Claim 18 (depends on 17)

The TV of claim 17, wherein the instructions further cause the processor to:
determine a realization vertex satisfying the interpreted wish and a realization path from a wishing point vertex to the realization vertex from an existing availability of story branches in the hyperstory based on a realization latency, a quality level, and the response-to-wish confidence level, the wishing point vertex being a vertex close to a point when the interaction is received.

### Claim 19 (depends on 18)

The TV of claim 18, wherein the instructions further cause the processor to:
determine the realization vertex and the realization path from the story branches in the hyperstory according to a wish-realization-distance (WRD).

### Claim 20 (depends on 19)

The TV of claim 19, wherein the instructions further cause the processor to:
for each possible realization vertex in the hyperstory:
determine a possible realization path corresponding to the realization video with a minimum time period;
analyze a matching degree between the interpreted wish and the possible realization path;
calculate the quality level of the possible realization path; and
calculate a WRD function of the possible realization path according to the obtained minimum time period, the matching degree of the possible realization path, and the quality level of the possible realization path; and
determine the realization vertex and the realization path corresponding to a minimum value of WRD as the realization vertex and the realization path.

### Claim 21 (depends on 20)

The TV of claim 20, wherein the instructions further cause the processor to:
obtain the edges bridging the wishing point vertex to the possible realization vertex by calculating the minimum time period; and
determine the possible realization path as a path of obtained edges.

### Claim 22

A non-transitory computer readable medium storing instructions that, when executed by a computer, cause the computer to:
receive an interaction indicating a user's wish;
interpret the interaction to obtain an interpreted wish;
identify a realization story in a hyperstory according to the interpreted wish;
send a feedback message indicating a time the user's wish will be realized and a response-to-wish confidence level, wherein the feedback message includes a realization latency, the response-to-wish confidence level, and a quality level;
generate a realization video according to the realization story; and
output the realization video.

### Claim 23 (depends on 22)

The computer readable medium of claim 22, wherein the instructions further cause the computer to:
interpret the interaction to derive wish information about who, what, when, and where; and
convert the wish information to the interpreted wish, the interpret wish including a template of wish command consistent with the derived wish information of who, what, when, and where.

### Claim 24 (depends on 22)

The computer readable medium of claim 22, wherein the instructions further cause the computer to:
obtain the hyperstory representing status of one or more characters and events causing status changes of the one or more characters.

### Claim 25 (depends on 24)

The computer readable medium of claim 24, wherein the instructions further cause the computer to:
represent the hyperstory using a directed acyclic graph (DAG), each vertex in the DAG representing a status of a character and an edge of any connected vertex pair representing an event causing status changes of the character.

### Claim 26 (depends on 25)

The computer readable medium of claim 25, wherein the instructions further cause the computer to:
determine a realization vertex satisfying the interpreted wish and a realization path from a wishing point vertex to the realization vertex from an existing availability of story branches in the hyperstory based on a realization latency, a quality level, and the response-to-wish confidence level, the wishing point vertex being a vertex close to a point when the interaction is received.

### Claim 27 (depends on 26)

The computer readable medium of claim 26, wherein the instructions further cause the computer to:
determine the realization vertex and the realization path from the story branches in the hyperstory according to a wish-realization-distance (WRD).

### Claim 28 (depends on 27)

The computer readable medium of claim 27, wherein the instructions further cause the computer to:
for each possible realization vertex in the hyperstory:
determine a possible realization path corresponding to the realization video with a minimum time period;
analyze a matching degree between the interpreted wish and the possible realization path;
calculate the quality level of the possible realization path; and
calculate a WRD function of the possible realization path according to the obtained minimum time period, the matching degree of the possible realization path, and the quality level of the possible realization path; and
determine the realization vertex and the realization path corresponding to a minimum value of WRD as the realization vertex and the realization path.

### Claim 29 (depends on 28)

The computer readable medium of claim 28, wherein the instructions further cause the computer to:
obtain the edges bridging the wishing point vertex to the possible realization vertex by calculating the minimum time period; and
determine the possible realization path as a path of obtained edges.