# US 12,113,968 B2 · Transmission device, transmission method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,113,968 B2 - Document ID: US12113968B2 - Status: Granted - Kind: B2 - Application number: 17/914,075 - Filed: 2021-03-24 - Published: 2023-04-20 - Granted: 2024-10-08 - Earliest claimed: 2020-03-31 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Katsushi Otsuka - Web page: https://patentgrep.com/patent/US12113968B2 ## CPC - H04N 19/115: Selection of the code volume for a coding unit prior to coding - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking - H04N 19/142: Detection of scene cut or scene change - H04N 19/146: Data rate or code amount at the encoder output - H04N 19/17: the unit being an image region, e.g. an object - H04N 19/172: the region being a picture, frame or field - H04N 19/174: the region being a slice, e.g. a line of blocks or a group of blocks - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Provided are a transmission device, a transmission method, and a program capable of performing rate control suitable for an image quality priority image. A basic parameter value decision section (28a) decides, for a frame image that is not an image quality priority image, values of parameters used for encoding an encoding unit that is a part or whole of the frame image, by applying the difference between a control amount and a target value to a predetermined control rule. An image quality priority parameter value decision section (28b) decides, for the frame image that is the image quality priority image, values by which the data size of the generated image data becomes larger than those decided by applying the difference between the control amount and the target value to the predetermined control rule as the values of the parameters used for the encoding unit of the frame image. ## Claims ### Claim 1 A transmission device comprising: an acquisition section that sequentially acquires frame images drawn in a frame buffer; a determination section that determines whether or not the frame image is an image quality priority image; a basic parameter value decision section that decides, for the frame image that is not the image quality priority image, values of parameters used for encoding an encoding unit that is a part or whole of the frame image, by applying a difference between a control amount and a target value to a predetermined control rule; an image quality priority parameter value decision section that decides, for the frame image that is the image quality priority image, values of parameters used for the encoding unit of the frame image; an encoding processing section that generates image data by executing the encoding of the encoding unit on a basis of the values of the parameters decided by the basic parameter value decision section or the image quality priority parameter value decision section; and a transmission section that transmits the image data, wherein the image quality priority parameter value decision section decides, as the values of the parameters used for the encoding unit of the frame image, values by which a data size of the generated image data becomes larger than those decided by applying the difference between the control amount and the target value to the predetermined control rule, and wherein the determination section determines that the frame image immediately after assets used for drawing the frame image are switched is the image quality priority image. ### Claim 2 (depends on 1) The transmission device according to claim 1, wherein the encoding processing section skips encoding of the frame image next to the image quality priority image. ### Claim 3 (depends on 1) The transmission device according to claim 1, wherein the encoding processing section decides the number of frame skips on a basis of the data size of the image data generated by encoding the image quality priority image, and the encoding processing section skips encoding of the frame image following the image quality priority image by the number of frame skips. ### Claim 4 (depends on 1) The transmission device according to claim 1, wherein, for the frame image next to the image quality priority image, the basic parameter value decision section decides the values of the parameters used for the encoding unit of the frame image by applying the difference between the control amount and the target value to the predetermined control rule. ### Claim 5 (depends on 1) The transmission device according to claim 1, wherein, for the frame image next to the image quality priority image, the basic parameter value decision section decides, as the values of the parameters, values by which the data size of the generated image data becomes smaller than those decided by applying the difference between the control amount and the target value to the predetermined control rule. ### Claim 6 (depends on 1) The transmission device according to claim 1, wherein the acquisition section acquires the frame image associated with image quality priority necessity data capable of specifying whether or not the frame image is the image quality priority image, and the determination section determines whether or not the frame image is the image quality priority image on a basis of the image quality priority necessity data associated with the frame image. ### Claim 7 (depends on 6) The transmission device according to claim 6, further comprising: an image quality priority necessity data generation section that generates the image quality priority necessity data and associates the image quality priority necessity data with the frame image. ### Claim 8 (depends on 7) The transmission device according to claim 7, wherein the image quality priority necessity data generation section associates a frame image immediately after a camera cut is performed after description of a continuous scene or a continuous game play is terminated with the image quality priority necessity data capable of specifying that the frame image is the image quality priority image. ### Claim 9 (depends on 7) The transmission device according to claim 7, wherein the image quality priority necessity data generation section associates a frame image immediately after a stage in a game has changed with the image quality priority necessity data capable of specifying that the frame image is the image quality priority image. ### Claim 10 (depends on 7) The transmission device according to claim 7, wherein the image quality priority necessity data generation section associates a frame image immediately after assets used for drawing the frame image are switched after the drawing of a continuous scene is terminated with the image quality priority necessity data capable of specifying that the frame image is the image quality priority image. ### Claim 11 (depends on 7) The transmission device according to claim 7, wherein the image quality priority necessity data generation section associates a frame image immediately after a new asset used for drawing the frame image is loaded into a memory with the image quality priority necessity data capable of specifying that the frame image is the image quality priority image. ### Claim 12 (depends on 1) The transmission device according to claim 1, wherein the determination section determines that the frame image in which a scene change from a last frame has occurred is the image quality priority image. ### Claim 13 (depends on 1) The transmission device according to claim 1, wherein the acquisition section acquires the frame image representing a state of viewing a virtual space from a virtual object arranged in the virtual space, and the determination section determines that the frame image immediately after the virtual object has changed from a last frame is the image quality priority image. ### Claim 14 (depends on 1) The transmission device according to claim 1, further comprising: an image generating section that generates the frame image on a basis of drawing intermediate data representing a state of viewing a virtual space from a virtual object arranged in the virtual space, wherein the determination section determines that the frame image generated by the image generation section on the basis of the drawing intermediate data immediately after the virtual object has changed from a last frame is the image quality priority image. ### Claim 15 (depends on 1) The transmission device according to claim 1, wherein the determination section determines whether or not the frame image is the image quality priority image, on a basis of a result of comparison between the frame image and a frame image of a frame immediately before the frame image. ### Claim 16 (depends on 1) The transmission device according to claim 1, wherein the encoding processing section encodes the frame image as the image quality priority image into an I frame. ### Claim 17 (depends on 1) The transmission device according to claim 1, wherein the image quality priority parameter value decision section decides, as the values of the parameters used for the encoding unit of the frame image, values by which a compression ratio of the generated image data becomes smaller than those decided by applying the difference between the control amount and the target value to the predetermined control rule. ### Claim 18 (depends on 1) The transmission device according to claim 1, wherein the encoding processing section executes encoding using a gradual decoder refresh technique for the frame image that is not the image quality priority image. ### Claim 19 (depends on 1) The transmission device according to claim 1, wherein the control amount is the data size of the image data or a length of time from a start timing of the encoding of the frame image to a transmission end timing of the image data generated on a basis of the frame image. ### Claim 20 (depends on 1) The transmission device according to claim 1, wherein the image quality priority parameter value decision section decides the values of the parameters that correspond to a case where a value smaller than the difference between the control amount and the target value is applied to the predetermined control rule and are used for the encoding unit of the frame image. ### Claim 21 (depends on 1) The transmission device according to claim 1, wherein the basic parameter value decision section and the image quality priority parameter value decision section decide the values of the parameters used for the encoding unit by performing proportional integral derivative control, and the image quality priority parameter value decision section decides the values of the parameters used for the encoding unit of the frame image by suppressing a gain of at least one of proportional control and differential control in the proportional integral derivative control more than that decided by the basic parameter value decision section. ### Claim 22 (depends on 1) The transmission device according to claim 1, wherein the image quality priority parameter value decision section decides the values of the parameters on a basis of the difference between the control amount and the target value at a time interval longer than that of the basic parameter value decision section. ### Claim 23 (depends on 1) The transmission device according to claim 1, wherein the basic parameter value decision section or the image quality priority parameter value decision section decides the values of the parameters further on a basis of at least one of the number of times the image data has been missing per unit time in the past and the number of times the I frame has been generated per unit time in the past. ### Claim 24 A transmission method comprising: a sequentially acquiring frame images drawn in a frame buffer; a determining whether or not the frame image is an image quality priority image; a basic parameter value decision step of deciding, for the frame image that is not the image quality priority image, values of parameters used for encoding an encoding unit that is a part or whole of the frame image, by applying a difference between a control amount and a target value to a predetermined control rule; an image quality priority parameter value decision step of deciding, for the frame image that is the image quality priority image, values of parameters used for the encoding unit of the frame image; an encoding processing step of generating image data by executing the encoding of the encoding unit on a basis of the values of the parameters decided in the basic parameter value decision step or the image quality priority parameter value decision step; and a transmitting the image data, wherein, in the image quality priority parameter value decision step, as the values of the parameters used for the encoding unit of the frame image, values by which a data size of the generated image data becomes larger than those decided by applying the difference between the control amount and the target value to the predetermined control rule are decided, wherein the determining includes determining that the frame image immediately after assets used for drawing the frame image are switched is the image quality priority image. ### Claim 25 A non-transitory computer storage readable medium comprising computer executable program code configured to instruct a computer to perform the following steps: by an acquisition section, sequentially acquiring frame images drawn in a frame buffer; by a determination section, determining whether or not the frame image is an image quality priority image; a basic parameter value decision procedure of deciding, for the frame image that is not the image quality priority image, values of parameters used for encoding an encoding unit that is a part or whole of the frame image, by applying a difference between a control amount and a target value to a predetermined control rule; an image quality priority parameter value decision procedure of deciding, for the frame image that is the image quality priority image, values of parameters used for the encoding unit of the frame image; an encoding processing procedure of generating image data by executing the encoding of the encoding unit on a basis of the values of the parameters decided in the basic parameter value decision procedure or the image quality priority parameter value decision procedure; and by a transmission section, transmitting the image data, wherein, in the image quality priority parameter value decision procedure, as the values of the parameters used for the encoding unit of the frame image, values by which a data size of the generated image data becomes larger than those decided by applying the difference between the control amount and the target value to the predetermined control rule are decided, and wherein the determination section determines that the frame image immediately after assets used for drawing the frame image are switched is the image quality priority image.