# US 2022/0046291 A1 · Method and device for generating live streaming video data and method and device for playing live streaming video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0046291 A1 - Document ID: US20220046291A1 - Status: Published application - Kind: A1 - Application number: 17/392,711 - Filed: 2021-08-03 - Published: 2022-02-10 - Earliest claimed: 2020-08-04 - Assignees: Shanghai Bilibili Technology Co., Ltd. - Applicants: Shanghai Bilibili Technology Co., Ltd. - Inventors: Jun JIANG; Junwei Zhu; Yongfang Qin - Web page: https://patentgrep.com/patent/US20220046291A1 ## CPC - H04N 21/2187: Live feed - G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components - G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes} - H04N 19/46: Embedding additional information in the video signal during the compression process (H04N19/517, H04N19/68, H04N19/70 take precedence) - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/4312: {involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations} - H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window} - H04N 21/4318: {by altering the content in the rendering process, e.g. blanking, blurring or masking an image region (image enhancement or restoration in general G06T5/00)} - H04N 21/4886: {for displaying a ticker, e.g. scrolling banner for news, stock exchange, weather data} - H04N 21/8126: {involving additional data, e.g. news, sports, stocks, weather forecasts} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The present disclosure describes techniques for generating live streaming video data and playing live streaming video. The techniques comprise obtaining live broadcasting video data; identifying a target object in a video frame; determining a target area based on the target object in the video frame; encoding data indicative of the target area based on a predetermined format for supplementary enhancement information to obtain encoded target area data; and adding the encoded target area data to a data packet corresponding to the video frame, wherein the data packet is encoded based on a predetermined video data format, and wherein the predetermined video data format and the predetermined format for supplementary enhancement information are mutually compatible data encoding formats. ## Claims ### Claim 1 A method for generating live streaming video data, comprising: obtaining live broadcasting video data; identifying a target object in a first video frame, wherein the first video frame is any video frame comprised in the live broadcasting video data; determining a target area based on the target object in the first video frame, wherein the target area is an area not to be occluded by any item; encoding data indicative of the target area based on a predetermined format for supplementary enhancement information to obtain encoded target area data; and adding the encoded target area data to a data packet corresponding to the first video frame, wherein the data packet is encoded based on a predetermined video data format, and wherein the predetermined video data format and the predetermined format for supplementary enhancement information are mutually compatible data encoding formats. ### Claim 2 (depends on 1) The method of claim 1, wherein the determining a target area based on the target object in the first video frame further comprises: determining a contour of the target object in the first video frame; and identifying an area covered by the contour of the target object in the first video frame as the target area. ### Claim 3 (depends on 2) The method of claim 2, wherein when the live broadcasting video data are real-time video data captured by a camera, the method further comprises: obtaining video parameters of performing video image synthesis on the first video frame by a video software tool; and correcting the contour of the target object based on the video parameters. ### Claim 4 (depends on 1) The method of claim 1, wherein the identifying a target object in a first video frame further comprises: identifying the target object in the first video frame using a preset target object recognition model; or when the live broadcasting video data are video data captured by a virtual camera, removing other data in the first video frame than data indicative of the target object by means of green screen matting. ### Claim 5 A method for playing live streaming video, comprising: receiving live streaming video data comprising video frames and corresponding bullet screen data, wherein the corresponding bullet screen data comprise comments to be presented relative to the video frames; parsing data associated with a video frame among the video frames comprised in the live streaming video data, wherein the data associated with the video frame further comprise target area data encoded based on a predetermined format for supplementary enhancement information, the predetermined format for supplementary enhancement information is compatible with a predetermined video data format for encoding a data packet corresponding to the video frame, and the target area data comprise data indicative of a target area corresponding to a target object in the video frame; extracting the data indicative of the target area corresponding to the target object in the video frame; generating a bullet screen mask corresponding to the video frame based on the target area; and displaying at least one comment among the comments associated with and relative to the video frame on the bullet screen mask while displaying the video frame. ### Claim 6 (depends on 5) The method of claim 5, wherein the generating a bullet screen mask corresponding to the video frame based on the target area further comprises: generating the bullet screen mask corresponding to the video frame by setting the target area in a preset initial bullet screen mask of the video frame. ### Claim 7 (depends on 6) The method of claim 6, further comprising: transparently displaying the at least one comment when the at least one comment overlaps the target area in the bullet screen mask. ### Claim 8 A system of generating live streaming video data, comprising: at least a processor; and at least a memory communicatively coupled to the at least a processor to configure the at least a processor to perform operations comprising: obtaining live broadcasting video data; identifying a target object in a first video frame, wherein the first video frame is any video frame comprised in the live broadcasting video data; determining a target area based on the target object in the first video frame, wherein the target area is an area not to be occluded by any item; encoding data indicative of the target area based on a predetermined format for supplementary enhancement information to obtain encoded target area data; and adding the encoded target area data to a data packet corresponding to the first video frame, wherein the data packet is encoded based on a predetermined video data format, and wherein the predetermined video data format and the predetermined format for supplementary enhancement information are mutually compatible data encoding formats. ### Claim 9 (depends on 8) The system of claim 8, wherein the determining a target area based on the target object in the first video frame further comprises: determining a contour of the target object in the first video frame; and identifying an area covered by the contour of the target object in the first video frame as the target area. ### Claim 10 (depends on 9) The system of claim 9, wherein when the live broadcasting video data are real-time video data captured by a camera, the method further comprises: obtaining video parameters of performing video image synthesis on the first video frame by a video software tool; and correcting the contour of the target object based on the video parameters. ### Claim 11 (depends on 8) The system of claim 8, wherein the identifying a target object in a first video frame further comprises: identifying the target object in the first video frame using a preset target object recognition model; or when the live broadcasting video data are video data captured by a virtual camera, removing other data in the first video frame than data indicative of the target object by means of green screen matting. ### Claim 12 A non-transitory computer-readable storage medium bearing computer-readable instructions that upon execution on a computing device cause the computing device at least to perform operations comprising: obtaining live broadcasting video data; identifying a target object in a first video frame, wherein the first video frame is any video frame comprised in the live broadcasting video data; determining a target area based on the target object in the first video frame, wherein the target area is an area not to be occluded by any item; encoding data indicative of the target area based on a predetermined format for supplementary enhancement information to obtain encoded target area data; and adding the encoded target area data to a data packet corresponding to the first video frame, wherein the data packet is encoded based on a predetermined video data format, and wherein the predetermined video data format and the predetermined format for supplementary enhancement information are mutually compatible data encoding formats. ### Claim 13 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein the determining a target area based on the target object in the first video frame further comprises: determining a contour of the target object in the first video frame; and identifying an area covered by the contour of the target object in the first video frame as the target area. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium of claim 13, wherein when the live broadcasting video data are real-time video data captured by a camera, the method further comprises: obtaining video parameters of performing video image synthesis on the first video frame by a video software tool; and correcting the contour of the target object based on the video parameters. ### Claim 15 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein the identifying a target object in a first video frame further comprises: identifying the target object in the first video frame using a preset target object recognition model; or when the live broadcasting video data are video data captured by a virtual camera, removing other data in the first video frame than data indicative of the target object by means of green screen matting.