# US 12,120,378 B2 · Video distribution in which provided information is superimposed on video > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,120,378 B2 - Document ID: US12120378B2 - Status: Granted - Kind: B2 - Application number: 18/008,614 - Filed: 2022-06-28 - Published: 2024-07-18 - Granted: 2024-10-15 - Earliest claimed: 2022-06-28 - Assignees: Rakuten Group, Inc. - Applicants: Rakuten Group, Inc. - Inventors: Ko Kikuchi - Web page: https://patentgrep.com/patent/US12120378B2 ## CPC - H04N 21/4316: {for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window} - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 7/50: Depth or shape recovery - H04N 21/2187: Live feed - H04N 21/234: Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs - H04N 21/431: Generation of visual interfaces {for content selection or interaction}; Content or additional data rendering - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection - H04N 21/816: {involving special video data, e.g 3D video} - G06T 2207/20221: Image fusion; Image merging - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract In an information processing device to streaming-distribute a video, based on a distance image in which a streamer is captured, an acquirer (101) acquires a depth distribution in the distance image. A determiner (102) determines a threshold value, based on the acquired depth distribution. An specifier (103) specifies, in the distance image, a distant region where depth is greater than the threshold value and a near region where depth is less than the threshold value. A provider (104) provides, in conjunction with the video, to a viewer terminal (300) to which the video is to be streaming-distributed, control information to set the distant region as a background, set the near region as a foreground, superimpose provided information over the background, and superimpose the foreground over the background over which the provided information is superimposed. ## Claims ### Claim 1 An information processing device to streaming-distribute a video, based on a distance image in which a streamer is captured, comprising one or more processors, wherein the processor executes processing of: acquiring a depth distribution in the distance image; determining a threshold value, based on the acquired depth distribution; specifying, in the distance image, a distant region where depth is greater than the threshold value and a near region where depth is less than the threshold value; and providing, in conjunction with the video, to a viewer terminal to which the video is to be streaming-distributed, control information to set the distant region as a background, set the near region as a foreground, superimpose provided information over the background, and superimpose the foreground over the background over which the provided information is superimposed. ### Claim 2 (depends on 1) The information processing device according to claim 1, wherein the provided information is information provided by a viewer of the viewer terminal, and the processor executes processing of providing control information to cause the provided information to be displayed in such a way that the provided information, starting from outside of a screen of the viewer terminal, keeps moving toward an inner part of the screen and disappears out of the screen. ### Claim 3 (depends on 1) The information processing device according to claim 1, wherein the processor executes processing of: compensating the near region so as to be a region of a predetermined shape encompassing the near region; compensating the distant region so as to be a region excluding the region of the predetermined shape; and setting the compensated distant region as the background and the compensated near region as the foreground. ### Claim 4 (depends on 1) The information processing device according to claim 1, wherein the acquired depth distribution is a mixed distribution, and the processor executes processing of: separating the acquired depth distribution into a first distribution and a second distribution; and when a first representative value of the first distribution and a second representative value of the second distribution are sufficiently separated from each other, determining a middle value between the first representative value and the second representative value as the threshold value. ### Claim 5 (depends on 4) The information processing device according to claim 4, wherein the processor executes processing of, when a difference between the first representative value and the second representative value is greater than a value obtained by multiplying a sum of a standard deviation of the first distribution and a standard deviation of the second distribution by a predetermined value, determining that the first representative value of the first distribution and the second representative value of the second distribution are sufficiently separated from each other. ### Claim 6 (depends on 1) The information processing device according to claim 1, wherein the processor executes processing of providing a streamer terminal with which the streamer captures a video with control information to cause the streamer terminal to display a video or a distance image that is currently being captured and display the distant region and the near region in different display forms in the distance image. ### Claim 7 A display control method performed by a computer for streaming-distributing a video, based on a distance image in which a streamer is captured, the display control method comprising: acquiring a depth distribution in the distance image; determining a threshold value, based on the acquired depth distribution; specifying, in the distance image, a distant region where depth is greater than the threshold value and a near region where depth is less than the threshold value; and providing, in conjunction with the video, to a viewer terminal to which the video is to be streaming-distributed, control information to set the distant region as a background, set the near region as a foreground, superimpose provided information over the background, and superimpose the foreground over the background over which the provided information is superimposed. ### Claim 8 A computer-readable recording medium recording a program causing a computer for streaming-distributing a video, based on a distance image in which a streamer is captured to execute processing of: acquiring a depth distribution in the distance image; determining a threshold value, based on the acquired depth distribution; specifying, in the distance image, a distant region where depth is greater than the threshold value and a near region where depth is less than the threshold value; and providing, in conjunction with the video, to a viewer terminal to which the video is to be streaming-distributed, control information to set the distant region as a background, set the near region as a foreground, superimpose provided information over the background, and superimpose the foreground over the background over which the provided information is superimposed.