# US 2022/0277686 A1 · Display control apparatus, transmission apparatus, display control method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0277686 A1 - Document ID: US20220277686A1 - Status: Published application - Kind: A1 - Application number: 17/627,908 - Filed: 2020-09-08 - Published: 2022-09-01 - Earliest claimed: 2019-09-11 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Sony Interactive Entertainment Inc. - Inventors: Katsushi Otsuka - Web page: https://patentgrep.com/patent/US20220277686A1 ## CPC - A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client - G09G 3/2096: {Details of the interface to the display terminal specific for a flat panel (suitable for both CRT and flat panel G09G5/006; specific for a CRT G09G1/167)} - A63F 13/25: Output arrangements for video game devices - A63F 13/44: involving timing of operations, e.g. performing an action within a time slot - A63F 13/52: involving aspects of the displayed game scene - G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T) - G09G 5/36: characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory - H04L 7/04: Speed or phase control by synchronisation signals {(H04L7/0075 takes precedence)} - H04N 21/242: Synchronisation processes, e.g. processing of PCR [Programme Clock References] {(arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18)} - H04N 21/44: Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs - G09G 2310/08: Details of timing specific for flat panels, other than clock recovery - G09G 2320/0252: Improving the response speed - G09G 2340/0435: Change or adaptation of the frame rate of the video stream - G09G 2370/02: Networking aspects - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A display control apparatus, a transmission apparatus, a display control method, and a program that enable an image display time interval at the display control apparatus, which is connected to the transmission apparatus through a network, to properly follow an image rendering time interval at the transmission apparatus. A data reception section sequentially receives image data indicating an image associated with rendering time data capable of determining an image rendering time for the image in a cloud server. A display timing determination section determines, based on the rendering time data, a display timing of the image indicated by the image data. A display control section controls display of the image in accordance with the determined display timing. ## Claims ### Claim 1 A display control apparatus comprising: a reception section that sequentially receives image data from a transmission apparatus, the image data indicating an image associated with rendering time data, the rendering time data being capable of determining an image rendering time for the image in the transmission apparatus; a display timing determination section that determines, based on the rendering time data, a display timing of the image indicated by the image data; and a display control section that controls display of the image in accordance with the determined display timing. ### Claim 2 (depends on 1) The display control apparatus according to claim 1, wherein the display timing determination section determines the display timing based further on latency or cumulative error data, the cumulative error data indicating a cumulative error between a timing at which an image should be displayed and a timing at which the image is actually displayed. ### Claim 3 (depends on 1) The display control apparatus according to claim 1, wherein the reception section receives the image data that is further associated with at least one of a frame number, encoding time data, or transmission start timing data, the encoding time data being capable of determining a time spent on encoding, the transmission start timing data indicating a transmission timing of the image data, and the display control section controls the display of the image based further on at least one of a frame number, the encoding time data, or the transmission start timing data. ### Claim 4 (depends on 1) The display control apparatus according to claim 1, wherein, in a case where the display control apparatus does not support a variable frame rate and displays images at a constant frame rate, the display control section displays the image at a vertical synchronization timing closest to the display timing. ### Claim 5 (depends on 1) The display control apparatus according to claim 1, further comprising: a statistical data generation section that generates statistical data indicative of latency statistics on a plurality of pieces of the image data; and a tolerable latency determination section that determines tolerable latency in accordance with the statistical data, wherein the display control section exercises control, based on whether or not the tolerable latency is exceeded by the latency of a newly received piece of the image data, so as to determine whether to display or discard an image indicated by the newly received piece of image data. ### Claim 6 (depends on 5) The display control apparatus according to claim 5, further comprising: a change instruction transmission section that, in a case where a predetermined condition is satisfied by the determined tolerable latency, transmits an instruction for changing a data rate to the transmission apparatus. ### Claim 7 (depends on 6) The display control apparatus according to claim 6, wherein, in a case where a predetermined latency is exceeded by the determined tolerable latency, the change instruction transmission section transmits an instruction for decreasing the data rate to the transmission apparatus. ### Claim 8 (depends on 6) The display control apparatus according to claim 6, wherein, in a case where the determined tolerable latency is lower than a predetermined latency, the change instruction transmission section transmits an instruction for increasing the data rate to the transmission apparatus. ### Claim 9 A transmission apparatus comprising: an acquisition section that acquires an image rendered in a frame buffer; an encoding processing section that generates image data by encoding the acquired image; and a transmission section that transmits the image data associated with rendering time data to a terminal displaying the acquired image, the rendering time data being capable of determining an image rendering time. ### Claim 10 (depends on 9) The transmission apparatus according to claim 9, wherein the transmission section transmits the image data that is further associated with at least one of a frame number, encoding time data, or transmission start timing data, the encoding time data being capable of determining a time spent on encoding, the transmission start timing data indicating a transmission timing of the image data. ### Claim 11 (depends on 10) The transmission apparatus according to claim 10, wherein the transmission section transmits the image data that is associated with a history of at least one of the rendering time data, a frame number, the encoding time data, or the transmission start timing data. ### Claim 12 A display control method comprising: sequentially receiving, from a transmission apparatus, image data indicative of an image associated with rendering time data, the rendering time data being capable of determining an image rendering time for the image in the transmission apparatus; determining, based on the rendering time data, a display timing of the image indicated by the image data; and controlling display of the image in accordance with the determined display timing. ### Claim 13 A non-transitory, computer readable storage medium containing a program, which when executed by a computer, causes the computer to perform a display control method by carrying out actions, comprising: sequentially receiving, from a transmission apparatus, image data indicative of an image associated with rendering time data, the rendering time data being capable of determining an image rendering time for the image in the transmission apparatus; determining, based on the rendering time data, a display timing of the image indicated by the image data; and controlling display of the image in accordance with the determined display timing.