# US 10,482,568 B2 · Information processor and information processing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,568 B2 - Document ID: US10482568B2 - Status: Granted - Kind: B2 - Application number: 14/907,549 - Filed: 2015-02-24 - Published: 2016-06-23 - Granted: 2019-11-19 - Earliest claimed: 2014-05-28 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Kouichi Kasai; Takanori Minamino - Web page: https://patentgrep.com/patent/US10482568B2 ## CPC - G06T 1/60: Memory management - G09G 5/001: {Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor} - G09G 5/393: Arrangements for updating the contents of the bit-mapped memory - G09G 5/395: Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen (G09G5/399 takes precedence) - G09G 2340/02: Handling of images in compressed format, e.g. JPEG, MPEG - G09G 2340/0435: Change or adaptation of the frame rate of the video stream - G09G 2350/00: Solving problems of bandwidth in display systems - G09G 2370/02: Networking aspects - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract Provided is a device and a method that are capable of display control of image data, which is received through a communication unit, with a small delay. An image frame, and memory input time and transmission frame rate information that serve as metadata are stored in the memory storing the image frame. The display control unit selects the image that is output to the display unit on the basis of the elapsed time from the input time information. The waiting time, which is the elapsed time from the time of input, and the buffering time are compared with each other in each of the queues, and the image frame corresponding to the newest queue, which is a queue among the queues in which the waiting time exceeds the buffering time, is selected as the output image to the display unit. ## Claims ### Claim 1 An information processor, comprising: a communication circuit configured to receive encoded image data; a decoder configured to: decode the encoded image data received by the communication circuit; associate a first time with each of a plurality of decoded image frames of the decoded image data; and output decoded image data based on the decode of the encoded image data; a memory configured to store: the plurality of decoded image frames of the decoded image data output by the decoder; a preceding stored image of the plurality of decoded image frames as a preceding queue of a plurality of queues; a succeeding stored image of the plurality of decoded image frames as a succeeding queue of the plurality of queues; and metadata that includes the first time at which each of the plurality of decoded image frames is input to the memory, and transmission frame rate information of each of the plurality of decoded image frames, wherein each of the plurality of decoded image frames is input to the memory at the first time after the decode of the encoded image data; and a display control circuit configured to: determine an elapsed time from the first time as a waiting time based on the stored metadata; compare the waiting time and a buffering time in each of the plurality of queues based on the preceding stored image and the succeeding stored image, wherein the buffering time is a time required to store data in the memory; select one of the plurality of decoded image frames as an output image for display, wherein the selected one of the plurality of decoded image frames corresponds to a newest queue among the plurality of queues in which the waiting time exceeds the buffering time; and control a display unit to display the output image based on the transmission frame rate information. ### Claim 2 (depends on 1) The information processor according to claim 1, wherein the display control circuit is further configured to: control the display unit to continue the display of the output image, wherein the output image is displayed continuously based on non-acquisition of next data for display from the memory at an image update processing start time of the display unit; and change a device state to a display update stopped state. ### Claim 3 (depends on 2) The information processor according to claim 2, wherein the display control circuit is further configured to: compare, in the display update stopped state, the waiting time and the buffering time; select the one of the plurality of decoded image frames, as the output image, based on the waiting time that exceeds the buffering time; and change the device state to a display update executing state. ### Claim 4 (depends on 1) The information processor according to claim 1, wherein the display control circuit is further configured to switch a device state from at least one of: a display update executing state to a display update stopped state, or the display update stopped state to the display update executing state, wherein the switch of the device state is based on an acquisition state of first data for display from the memory. ### Claim 5 (depends on 4) The information processor according to claim 4, wherein the display control circuit is further configured to compare the waiting time with the buffering time based on the device state that is the display update stopped state. ### Claim 6 (depends on 4) The information processor according to claim 4, wherein the display control circuit is further configured to select the one of the plurality of decoded image frames regardless of whether the waiting time exceeds the buffering time. ### Claim 7 (depends on 1) The information processor according to claim 1, wherein the display control circuit is further configured to: determine a change in a transmission frame rate by acquisition of the transmission frame rate information that is set in the one of the plurality of decoded image frames; and change, based on the change in the transmission frame rate, a display rate of the display unit. ### Claim 8 (depends on 7) The information processor according to claim 7, wherein the display control circuit is further configured to: acquire the one of the plurality of decoded image frames, in which the waiting time exceeds the buffering time, from the memory; and change, based on a verification of the change in the transmission frame rate, the display rate of the display unit from the acquired one of the plurality of decoded image frames in addition to the change in the transmission frame rate, wherein the verification of the change in the transmission frame rate is based on the metadata associated with the one of the plurality of decoded image frames. ### Claim 9 (depends on 7) The information processor according to claim 7, wherein the display control circuit is further configured to: switch a device state from one of: a display update executing state to a display update stopped state, or the display update stopped state to the display update executing state, wherein the switch of the device state is based on an acquisition state of first data for display from the memory, select the one of the plurality of decoded image frames, in which the waiting time exceeds the buffering time as the output image to the display unit, based on the device state that is the display update stopped state; and select the one of the plurality of decoded image frames, as the output image to the display unit regardless of whether the waiting time exceeds the buffering time, based on the device state that is the display update executing state. ### Claim 10 (depends on 1) The information processor according to claim 1, wherein the memory is further configured to store the buffering time in advance. ### Claim 11 (depends on 1) The information processor according to claim 1, wherein the display control circuit is further configured to: retain the metadata of a previous image frame acquired before acquisition of the one of the plurality of decoded image frames; acquire the transmission frame rate information of the one of the plurality of decoded image frames; determine a change in a transmission frame rate based on a comparison between the transmission frame rate information of the one of the plurality of decoded image frames with transmission frame rate information of the previous image frame; and change a display rate of the display unit based on the change in the transmission frame rate. ### Claim 12 (depends on 11) The information processor according to claim 11, wherein the display rate of the display unit corresponds to a count of a vertical synchronizing signal of the display unit. ### Claim 13 An information processing method, comprising: in an information processor that includes a communication circuit, a decoder, and a memory: receiving, by the communication circuit, encoded image data; decoding, by the decoder, the encoded image data received by the communication circuit; associating, by the decoder, a first time with each of a plurality of decoded image frames of the decoded image data; outputting, by the decoder, decoded image data based on the decoding of the encoded image data; storing, in the memory, the plurality of decoded image frames of the decoded image data output by the decoder, a preceding stored image of the plurality of decoded image frames as a preceding queue of a plurality of queues, a succeeding stored image of the plurality of decoded image frames as a succeeding queue of the plurality of queues, and metadata, wherein the metadata includes the first time at which each of the plurality of decoded image frames is input to the memory, and transmission frame rate information of the each of the plurality of decoded image frames, and each of the plurality of decoded image frames is input to the memory at the first time after the decoding of the encoded image data; determining an elapsed time from the first time as a waiting time based on the stored metadata; comparing the waiting time and a buffering time in each of the plurality of queues based on the preceding stored image and the succeeding stored image, wherein the buffering time is a time required to store data in the memory; selecting one of the plurality of decoded image frames as an output image for display, wherein the selected one of the plurality of decoded image frames corresponds to a newest queue among the plurality of queues in which the waiting time exceeds the buffering time; and controlling a display unit to display the output image based on the transmission frame rate information. ### Claim 14 A non-transitory computer-readable medium having stored thereon, computer-executable instructions that when executed by a processor in a computer, cause the computer to execute operations, the operations comprising: receiving, by a communication circuit, encoded image data; decoding, by a decoder, the encoded image data received by the communication circuit; associating, by the decoder, a first time with each of a plurality of decoded image frames of the decoded image data; outputting, by the decoder, decoded image data based on the decoding of the encoded image data; storing, in a memory, the plurality of decoded image frames of the decoded image data output by the decoder, a preceding stored image of the plurality of decoded image frames as a preceding queue of a plurality of queues, a succeeding stored image of the plurality of decoded image frames as a succeeding queue of the plurality of queues, and metadata, wherein the metadata includes the first time at which each of the plurality of decoded image frames is input to the memory, and transmission frame rate information of each of the plurality of decoded image frames, and each of the plurality of decoded image frames is input to the memory at the first time after the decoding of the encoded image data; determining an elapsed time from the first time as a waiting time based on the stored metadata; comparing the waiting time and a buffering time in each of the plurality of queues based on the preceding stored image and the succeeding stored image, wherein the buffering time is a time required to store data in the memory; selecting one of the plurality of decoded image frames as an output image for display, wherein the selected one of the plurality of decoded image frames corresponds to a newest queue among the plurality of queues in which the waiting time exceeds the buffering time; and controlling a display unit to display the output image based on the transmission frame rate information.