# US 12,009,011 B2 · Method and device of presenting audio/video files, computing device, and readable storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,009,011 B2
- Document ID: US12009011B2
- Status: Granted
- Kind: B2
- Application number: 16/904,124
- Filed: 2020-06-17
- Published: 2020-12-24
- Granted: 2024-06-11
- Earliest claimed: 2019-06-18
- Assignees: Shanghai Bilibili Technology Co., Ltd.
- Applicants: Shanghai Bilibili Technology Co., LTD.
- Inventors: Huyang Sun; Zhaoxin Tan; Weijia Li; Jiangiang Ding; Yifei Pei; Yuxuan Gao; Kan Hu; Xiaofeng Ji; Chaoran Li; Fuling Chen; Dejun Shan
- Web page: https://patentgrep.com/patent/US12009011B2

## CPC

- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- H04N 21/8541: involving branching, e.g. to different story endings
- G11B 27/102: {Programmed access in sequence to addressed parts of tracks of operating record carriers (access by moving the head G11B3/08, G11B5/54, G11B7/085, G11B21/022; by moving the record carrier G11B15/005, G11B17/005, by driving of both record carrier and head G11B15/1816)}
- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- H04N 21/47217: {for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks (specific graphical features in visual interfaces H04N21/4312)}
- H04N 21/8456: {by decomposing the content in the time domain, e.g. in time segments}
- H04N 21/8545: for generating interactive applications
- G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

The present disclosure describes techniques of presenting audio/video (AV) files. The disclosed techniques include displaying a first interface, wherein the first interface comprises a list of a plurality of AV files and an editing area; generating a plot graph based at least in part on an input by a first user via the first interface, wherein the plot graph has a directed structure, and the at least one first type of AV file and the at least one second type of AV file are associated with different nodes of the plot graph; playing a first type of AV file; detecting that a preset instruction is received in response to determining that a playing progress of the first type of AV file reaches a first predetermined position; and playing another first type of AV file or a second type of AV file based on the preset instruction.

## Claims

### Claim 1

A method of implementing different plots of playing audio/video (AV) files, comprising:
generating and displaying a first interface, wherein the first interface comprises a list of a plurality of AV files and an editing area, wherein the plurality of AV files comprises at least one first type of AV file and at least one second type of AV file, wherein the at least one first type of AV file is a process video file corresponding to a developing process of at least two plots, wherein the at least one second type of AV file is an ending video file corresponding to an end of the at least two plots, and wherein the AV files comprise audio data and video data;
generating a plot graph based at least in part on an input by a first user via the first interface, wherein the plot graph comprises a directed cyclic graph and is configured to implement the at least two plots of playing the AV files, wherein the at least one first type of AV file and the at least one second type of AV file are associated with different nodes of the plot graph, wherein the at least one first type of AV file corresponds to a starting node of the plot graph, and wherein a first one of the at least one second type of AV file corresponds to a first ending node of the plot graph and a second one of the at least one second type of AV file corresponds to a second ending node of the plot graph;
playing a first type of AV file among the at least one first type of AV file;
determining whether a playing progress of the first type of AV file reaches a first predetermined position;
detecting whether a preset instruction is received in response to determining that the playing progress of the first type of AV file reaches the first predetermined position;
in response to detecting a receipt of the preset instruction, playing, based on the preset instruction, one of another first type of AV file among the at least one first type of AV file, the first one of the at least one second type of AV file, or the second one of the at least one second type of AV file, wherein the preset instruction is triggered by a user input via a second interface, and the method further comprises:
configuring the preset instruction;
establishing an association between the preset instruction and another first type of AV file or the second type of AV file;
determining whether the user input is received within a predetermined time; and
triggering the preset instruction in response to determining that the user input is received within the predetermined time.

### Claim 2 (depends on 1)

The method of claim 1, wherein the detecting that a preset instruction is received further comprises:
suspending playing the first type of AV file when the playing progress of the first type of AV file reaches the first predetermined position; and
displaying a second interface.

### Claim 3 (depends on 1)

The method of claim 1, wherein the preset instruction is a system instruction, and the method further comprises:
configuring the system instruction;
establishing an association between the system instruction and another first type of AV file or the second type of AV file;
determining whether a user input is received via a second interface within a predetermined time; and
performing the system instruction in response to a determination that the user input is not received within the predetermined time.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
configuring at least one variable;
assigning a value to the at least one variable in response to a determination that a user input is received via a second interface within a predetermined time; and
in response to determining that the playing progress of the first type of AV file reaches a second predetermined position, playing a corresponding second type of AV file or a corresponding first type of AV file based at least in part on the value of the at least one variable.

### Claim 5 (depends on 1)

The method of claim 1, wherein the plurality of AV files further comprise at least one third type of AV file, and the method further comprises when the playing progress of the first type of AV file reaches the first predetermined position,
loading a third type of AV file among the at least one third type of AV file; or
adjusting at least one of an amount of audio output or pixel values of video output frames of the first type of AV file.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
configuring at least one display marker;
establishing an association between the at least one display marker and the first type of AV file;
displaying the at least one display marker while playing the first type of AV file; and
wherein the at least one display marker comprises a first element and a second element, the first element corresponds to a first part of the first type of AV file that has been or is being played, and the second element corresponds to a second part of the first type of AV file that has not been played.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
activating a timer; and
performing the preset instruction when an output value of the timer reaches a threshold value.

### Claim 8 (depends on 2)

The method of claim 2, wherein the second interface comprises a text control, a pattern control, a button control, a polygon input box, or a polygon trigger control, and the method further comprises:
receiving information from a user, the information indicating a content of the second interface and a position of displaying the second interface;
storing the information in the first type of AV file corresponding to a first node of the plot graph;
generating the second interface based on the information; and
displaying the second interface based on the information when the playing progress of the first type of AV file reaches the first predetermined position.

### Claim 9

A system of implementing different plots of playing audio/video (AV) files, comprising
at least one processor; and
at least one memory communicatively coupled to the at least one processor and storing instructions that upon execution by the at least one processor cause the system to:
generate and display a first interface, wherein the first interface comprises a list of a plurality of AV files and an editing area, and wherein the plurality of AV files comprises at least one first type of AV file and at least one second type of AV file, wherein the at least one first type of AV file is a process video file corresponding to a developing process of at least two plots, wherein the at least one second type of AV file is an ending video file corresponding to an end of the at least two plots, and wherein the AV files comprise audio data and video data;
generate a plot graph based at least in part on an input by a first user via the first interface, wherein the plot graph comprises a directed cyclic graph and is configured to implement the at least two plots of playing the AV files, wherein the at least one first type of AV file and the at least one second type of AV file are associated with different nodes of the plot graph, wherein the at least one first type of AV file corresponds to a starting node of the plot graph, and wherein a first one of the at least one second type of AV file corresponds to a first ending node of the plot graph and a second one of the at least one second type of AV file corresponds to a second ending node of the plot graph;
play a first type of AV file among the at least one first type of AV file;
determine whether a playing progress of the first type of AV file reaches a first predetermined position;
detect whether a preset instruction is received in response to determining that the playing progress of the first type of AV file reaches the first predetermined position;
in response to detecting a receipt of the preset instruction, play, based on the preset instruction, one of another first type of AV file among the at least one first type of AV file, the first one of the at least one second type of AV file, or the second one of the at least one second type of AV file, wherein the preset instruction is triggered by a user input via a second interface, and the instructions that upon execution by the at least one processor cause the system to:
configure the preset instruction;
establish an association between the preset instruction and another first type of AV file or the second type of AV file;
determine whether the user input is received within a predetermined time; and
trigger the preset instruction in response to determining that the user input is received within the predetermined time.

### Claim 10 (depends on 9)

The system of claim 9, the at least one memory further storing instructions that upon execution by the at least one processor cause the system to:
suspend playing the first type of AV file when the playing progress of the first type of AV file reaches the first predetermined position; and
display a second interface.

### Claim 11 (depends on 9)

The system of claim 9, wherein the preset instruction is a system instruction, and the at least one memory further stores instructions that upon execution by the at least one processor cause the system to:
configure the system instruction;
establish an association between the system instruction and another first type of AV file or the second type of AV file;
determine whether a user input is received via a second interface within a predetermined time; and
perform the system instruction in response to a determination that the user input is not received within the predetermined time.

### Claim 12 (depends on 9)

The system of claim 9, the at least one memory further storing instructions that upon execution by the at least one processor cause the system to:
configure at least one variable;
assign a value to the at least one variable in response to a determination that a user input is received via a second interface within a predetermined time; and
in response to determining that the playing progress of the first type of AV file reaches a second predetermined position, play a corresponding second type of AV file or a corresponding first type of AV file based at least in part on the value of the at least one variable.

### Claim 13 (depends on 10)

The system of claim 10, wherein the second interface comprises a text control, a pattern control, a button control, a polygon input box, or a polygon trigger control, and wherein the at least one memory further stores instructions that upon execution by the at least one processor cause the system to:
receive information from a user, the information indicating a content of the second interface and a position of displaying the second interface;
store the information in the first type of AV file corresponding to a first node of the plot graph;
generate the second interface based on the information; and
display the second interface when the playing progress of the first type of AV file reaches the first predetermined position.

### Claim 14 (depends on 9)

The system of claim 9, the at least one memory further storing instructions that upon execution by the at least one processor cause the system to:
configure at least one display marker;
establish an association between the at least one display marker and the first type of AV file;
display the at least one display marker while playing the first type of AV file; and
wherein the at least one display marker comprises a first element and a second element, the first element corresponds to a first part of the first type of AV file that has been or is being played, and the second element corresponds to a second part of the first type of AV file that has not been played.

### Claim 15

A non-transitory computer-readable storage medium, which stores computer programs that upon execution by a processor cause the processor to perform operations, the operations comprising:
generating and displaying a first interface, wherein the first interface comprises a list of a plurality of AV files and an editing area, and wherein the plurality of AV files comprises at least one first type of AV file and at least one second type of AV file, wherein the at least one first type of AV file is a process video file corresponding to a developing process of at least two plots, wherein the at least one second type of AV file is an ending video file corresponding to an end of the at least two plots, and wherein the plurality of AV files comprise audio data and video data;
generating a plot graph based at least in part on an input by a first user via the first interface, wherein the plot graph comprises a directed cyclic graph and is configured to implement the at least two plots of playing the AV files, wherein the at least one first type of AV file and the at least one second type of AV file are associated with different nodes of the plot graph, wherein the at least one first type of AV file corresponds to a starting node of the plot graph, and wherein a first one of the at least one second type of AV file corresponds to a first ending node of the plot graph and a second one of the at least one second type of AV file corresponds to a second ending node of the plot graph;
playing a first type of AV file among the at least one first type of AV file;
determining whether a playing progress of the first type of AV file reaches a first predetermined position;
detecting whether a preset instruction is received in response to determining that the playing progress of the first type of AV file reaches the first predetermined position;
in response to detecting a receipt of the preset instruction, playing, based on the preset instruction, one of another first type of AV file among the at least one first type of AV file, the first one of the at least one second type of AV file, or the second one of the at least one second type of AV file, wherein the preset instruction is triggered by a user input via a second interface, and the causing the processor to further perform the operations comprising:
configuring the preset instruction;
establishing an association between the preset instruction and another first type of AV file or the second type of AV file;
determining whether the user input is received within a predetermined time; and
triggering the preset instruction in response to determining that the user input is received within the predetermined time.