# US 2021/0342385 A1 · Interactive method and system of bullet screen easter eggs > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0342385 A1 - Document ID: US20210342385A1 - Status: Published application - Kind: A1 - Application number: 17/222,107 - Filed: 2021-04-05 - Published: 2021-11-04 - Earliest claimed: 2020-04-30 - Assignees: Shanghai Bilibili Technology Co., Ltd. - Applicants: Shanghai Bilibili Technology Co.,Ltd. - Inventors: Lulu YU - Web page: https://patentgrep.com/patent/US20210342385A1 ## CPC - G06F 16/489: {using time information} - G06F 16/35: Clustering; Classification - G06F 16/435: Filtering based on additional data, e.g. user or group profiles - G06F 16/437: {Administration of user profiles, e.g. generation, initialisation, adaptation, distribution} - G06F 16/5866: {using information manually generated, e.g. tags, keywords, comments, manually generated location and time information} - G06F 16/9535: Search customisation based on user profiles and personalisation - G06F 40/134: Hyperlinking - G06F 40/30: Semantic analysis - G06N 3/08: Learning methods - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract The embodiment of the present application provides an interactive method of bullet screen easter eggs, including: obtaining information sent by a client when playing a multimedia file, wherein the information comprises comment content and comment time; determining a target interface element for replacing the comment content; determining a timestamp of the target interface element according to the comment time; generating bullet screen data comprising the target interface element and the timestamp; and sending the bullet screen data to the client for displaying the target interface element in a form of a bullet screen on a playback interface of the multimedia file when the multimedia file is played to a time point that matches the timestamp of the target interface element. ## Claims ### Claim 1 A method, comprising: obtaining data sent from a client computing device while the client computing device is playing a multimedia file, wherein the data comprise information indicative of comment content and comment time; determining a target interface element based at least in part on the comment content; determining a timestamp of the target interface element based on the comment time; generating bullet screen data, the bullet screen data comprising the target interface element and the timestamp; and sending bullet screen data to the client computing device for displaying the target interface element in a form of a bullet screen on a playback interface of the multimedia file when the multimedia file is played to a time point that matches the timestamp of the target interface element. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining the target interface element based at least in part on the multimedia file. ### Claim 3 (depends on 1) The method of claim 1, wherein the data sent from the client computing device further comprise information indicative of a user account, and wherein the method further comprises: determining a user preference based on the information indicative of the user account; determining whether an acquisition operation for acquiring the target interface element is performed based on the user preference; and generating a text bullet screen for displaying the comment content in response to a determination that the acquisition operation is not performed. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining whether a word segmentation in the comment content triggers a first rule, wherein the first rule comprises a first plurality of texts, a first plurality of interface elements, and a first mapping relationship between the first plurality of texts and the first plurality of interface elements; and determining that the word segmentation triggers the first rule based on determining that the word segmentation matches at least one text among the plurality of texts; and selecting the target interface element from the plurality of interface elements based on the at least one text and the first mapping relationship. ### Claim 5 (depends on 1) The method of claim 1, further comprising: determining whether a word segmentation in the comment content triggers a blacklist rule based on determining whether the word segmentation matches at least one text among a plurality of texts included in the blacklist rule; determining whether an acquisition operation for acquiring the target interface element is to be performed based on determining whether the word segmentation triggers the blacklist rule; in response to a determination that the word segmentation triggers the blacklist rule, determining not to perform the acquisition operation; and in response to the determination that the word segmentation triggers the blacklist rule, generating a text bullet screen for displaying the comment content. ### Claim 6 (depends on 1) The method of claim 1, further comprising: performing a semantic recognition operation on the comment content to obtain semantic information of the comment content; determining whether the semantic information of the comment content triggers a second rule, wherein the second rule comprises a second plurality of texts, a second plurality of interface elements, and a second mapping relationship between the second plurality of texts and the second plurality of interface elements; determining that the semantic information triggers the second rule based on determining that the semantic information matches at least one text among the second plurality of texts; and selecting the target interface element from the second plurality of interface elements based on the at least one text and the second mapping relationship. ### Claim 7 (depends on 1) The method of claim 1, further comprising: inputting the information indicative of the comment content into a trained model; outputting a plurality of confidence degrees corresponding to a plurality of tags from the trained model, wherein the plurality of tags are associated with the multimedia file and correspond to a plurality of interface elements, respectively; identifying a tag with a highest confidence degree among the plurality of tags as a target tag; and obtaining the target interface element based on the target tag in response to a determination that the confidence degree of the target tag is greater than a predetermined threshold. ### Claim 8 (depends on 1) The method of claim 1, further comprising: obtaining a third rule, the third rule being uploaded by a content uploader through a submission interface, wherein the third rule is associated with the multimedia file, and wherein the third rule comprises a third plurality of customized texts, a third plurality of customized interface elements, and a third mapping relationship between the third plurality of customized texts and the third plurality of customized interface elements; and determining the target interface element based at least in part on the third rule. ### Claim 9 (depends on 1) The method of claim 1, wherein the data sent from the client computing device further comprise information indicative of a user account, and wherein the method further comprises: determining an authority of the user account based on the information indicative of the user account, wherein there are a plurality of authorities comprising a first authority and a second authority, the first authority represents that a corresponding user account has a authority of launching interface elements, and the second authority represents that the corresponding user account does not have the authority of launching the interface elements; obtaining the target interface element in response to a first determination that the user account has the first authority; and generating a text bullet screen for displaying the comment content in response to a second determination that the user account has the second authority. ### Claim 10 (depends on 1) The method of claim 1, wherein the target interface element comprises a picture, the picture is formed by embedding the comment content in an image, and the image is associated with the comment content. ### Claim 11 (depends on 1) The method of claim 1, wherein the target interface element comprises at least one of a hyperlink or an interactive component. ### Claim 12 A system, comprising: at least one processor; and at least one memory communicatively coupled to the at least one processor and storing instructions that upon execution by the at least one processor cause the system to perform operations, the operations comprising: obtaining data sent from a client computing device while the client computing device is playing a multimedia file, wherein the data comprise information indicative of comment content and comment time; determining a target interface element based at least in part on the comment content; determining a timestamp of the target interface element based on the comment time; generating bullet screen data, the bullet screen data comprising the target interface element and the timestamp; and sending bullet screen data to the client computing device for displaying the target interface element in a form of a bullet screen on a playback interface of the multimedia file when the multimedia file is played to a time point that matches the timestamp of the target interface element. ### Claim 13 (depends on 12) The system of claim 12, the operations further comprising: determining whether a word segmentation in the comment content triggers a first rule, wherein the first rule comprises a first plurality of texts, a first plurality of interface elements, and a first mapping relationship between the first plurality of texts and the first plurality of interface elements; and determining that the word segmentation triggers the first rule based on determining that the word segmentation matches at least one text among the plurality of texts; and selecting the target interface element from the plurality of interface elements based on the at least one text and the first mapping relationship. ### Claim 14 (depends on 12) The system of claim 12, the operations further comprising: performing a semantic recognition operation on the comment content to obtain semantic information of the comment content; determining whether the semantic information of the comment content triggers a second rule, wherein the second rule comprises a second plurality of texts, a second plurality of interface elements, and a second mapping relationship between the second plurality of texts and the second plurality of interface elements; determining that the semantic information triggers the second rule based on determining that the semantic information matches at least one text among the second plurality of texts; and selecting the target interface element from the second plurality of interface elements based on the at least one text and the second mapping relationship. ### Claim 15 (depends on 12) The system of claim 12, the operations further comprising: obtaining a third rule, the third rule being uploaded by a content uploader through a submission interface, wherein the third rule is associated with the multimedia file, and wherein the third rule comprises a third plurality of customized texts, a third plurality of customized interface elements, and a third mapping relationship between the third plurality of customized texts and the third plurality of customized interface elements; and determining the target interface element based at least in part on the third rule. ### Claim 16 (depends on 12) The system of claim 12, the operations further comprising: inputting the information indicative of the comment content into a trained model; outputting a plurality of confidence degrees corresponding to a plurality of tags from the trained model, wherein the plurality of tags are associated with the multimedia file and correspond to a plurality of interface elements, respectively; identifying a tag with a highest confidence degree among the plurality of tags as a target tag; and obtaining the target interface element based on the target tag in response to a determination that the confidence degree of the target tag is greater than a predetermined threshold. ### Claim 17 (depends on 12) The system of claim 12, wherein the data sent from the client computing device further comprise information indicative of a user account, and wherein the operations further comprising: determining a user preference based on the information indicative of the user account; determining an authority of the user account based on the information indicative of the user account, wherein there are a plurality of authorities comprising a first authority and a second authority, the first authority represents that a corresponding user account has a authority of launching interface elements, and the second authority represents that the corresponding user account does not have the authority of launching the interface elements; and determining whether an acquisition operation for acquiring the target interface element is performed based on the user preference or the authority of the user account. ### Claim 18 (depends on 12) The system of claim 12, wherein the target interface element comprises a picture, the picture is formed by embedding the comment content in an image, and the image is associated with the comment content. ### Claim 19 (depends on 12) The system of claim 12, wherein the target interface element comprises at least one of a hyperlink or an interactive component. ### Claim 20 A non-transitory computer-readable storage medium, storing computer-readable instructions that upon execution by a processor cause the processor to implement operations comprising: obtaining data sent from a client computing device while the client computing device is playing a multimedia file, wherein the data comprise information indicative of comment content and comment time; determining a target interface element based at least in part on the comment content; determining a timestamp of the target interface element based on the comment time; generating bullet screen data, the bullet screen data comprising the target interface element and the timestamp; and sending bullet screen data to the client computing device for displaying the target interface element in a form of a bullet screen on a playback interface of the multimedia file when the multimedia file is played to a time point that matches the timestamp of the target interface element.