# US 12,112,453 B2 · Image display control device, transmitting device, image display control method, and program for generating a substitute image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,453 B2
- Document ID: US12112453B2
- Status: Granted
- Kind: B2
- Application number: 17/435,864
- Filed: 2019-12-25
- Published: 2022-05-12
- Granted: 2024-10-08
- Earliest claimed: 2019-03-25
- Assignees: Sony Interactive Entertainment Inc.
- Applicants: Sony Interactive Entertainment Inc.
- Inventors: Katsushi Otsuka
- Web page: https://patentgrep.com/patent/US12112453B2

## CPC

- G06T 3/4092: Image resolution transcoding, e.g. by using client-server architectures
- H04N 21/4781: {Games}
- A63F 13/355: Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
- A63F 13/358: Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
- A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
- G09G 5/12: Synchronisation between the display unit and other units, e.g. other display units, video-disc players
- H04N 21/2187: Live feed
- H04N 21/440263: {by altering the spatial resolution, e.g. for displaying on a connected PDA}
- H04N 21/442: Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk {(arrangements for monitoring broadcast services or broadcast-related services H04H60/29; arrangements for identifying or recognising characteristics with a direct linkage to broadcast information H04H60/35; monitoring of user activities for profile generation for accessing a video database G06F16/739; monitoring in wireless networks H04W24/00)}
- H04N 21/44209: {Monitoring of downstream path of the transmission network originating from a server, e.g. bandwidth variations of a wireless network (arrangements for maintenance or administration in data switching networks involving bandwidth and capacity management H04L41/0896)}
- G09G 5/393: Arrangements for updating the contents of the bit-mapped memory
- G09G 5/395: Arrangements specially adapted for transferring the contents of the bit-mapped memory to the screen (G09G5/399 takes precedence)
- G09G 2340/0407: Resolution change, inclusive of the use of different resolutions for different screen areas
- G09G 2340/16: Determination of a pixel data signal depending on the signal applied in the previous frame
- G09G 2350/00: Solving problems of bandwidth in display systems
- G09G 2370/025: LAN communication management
- H04N 19/132: Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
- H04N 19/172: the region being a picture, frame or field
- H04N 19/85: using pre-processing or post-processing specially adapted for video compression
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A substitute image generating section generates a substitute image corresponding to second image data on the basis of first image data. According to success or failure of reception of the second image data, a display control section causes either one of the to-be-displayed image generated on the basis of the second image data and the substitute image generated on the basis of the first image data to be displayed. The substitute image generating section starts the generation of the substitute image corresponding to the second image data based on the first image data before the success or failure of the reception of the second image data is confirmed.

## Claims

### Claim 1

An image display control device comprising:
an image data receiving section that repeatedly receives image data, wherein: (i) the image data represents a downscaled image obtained by downscaling an original image, (ii) the image data is transmitted from a transmitting device, (iii) the transmitting device further transmits feature-amount data that is associated with the image data and that represents at least one of a feature amount of the original image and a feature amount that is a parameter used in encoding of the downscaled image, and (iv) the transmitting device transmits prioritizes the transmission of the feature-amount data over transmission of the image data associated with the feature-amount data;
a to-be-displayed-image generating section that generates a to-be-displayed image on a basis of the image data;
a substitute image generating section that, on a basis of first image data that is the image data received at a first timing, generates a substitute image that is an image different from the to-be-displayed image generated on a basis of the first image data and that corresponds to second image data that is the image data planned to be received at a second timing that comes after the first timing; and
a display control section that causes either one of the to-be-displayed image generated on a basis of the second image data and the substitute image generated on a basis of the first image data to be displayed according to success or failure of the reception of the second image data, wherein
the substitute image generating section starts the generation of the substitute image corresponding to the second image data based on the first image data before the success or failure of the reception of the second image data is confirmed.

### Claim 2 (depends on 1)

The image display control device according to claim 1, wherein
the image data receiving section receives the image data representing a downscaled image obtained by downscaling an original image,
the to-be-displayed-image generating section generates the to-be-displayed image having a resolution higher than a resolution of the downscaled image, and
the substitute image generating section generates the substitute image having a resolution higher than the resolution of the downscaled image.

### Claim 3 (depends on 2)

The image display control device according to claim 2, wherein
the image data receiving section further receives feature-amount data that is associated with the image data and that represents at least one of a feature amount of the original image and a feature amount that is a parameter used in encoding of the downscaled image, and
the to-be-displayed-image generating section generates the to-be-displayed image on a basis of the image data, and the feature-amount data associated with the image data.

### Claim 4 (depends on 3)

The image display control device according to claim 3, wherein the substitute image generating section generates the substitute image on a basis of the first image data, and the feature-amount data associated with the first image data.

### Claim 5 (depends on 3)

The image display control device according to claim 3, wherein the substitute image generating section generates the substitute image on a basis of the first image data, and the feature-amount data associated with the second image data.

### Claim 6 (depends on 3)

The image display control device according to claim 3, wherein, in a case where part of the second image data and the feature-amount data associated with the second image data is received, the substitute image generating section generates the substitute image further on a basis of the part.

### Claim 7 (depends on 3)

The image display control device according to claim 3, wherein the feature-amount data includes data representing at least one of a feature point of the original image, an edge strength, a depth of each pixel included in the original image, a texture type of the original image, an optical flow, and information representing in which direction and at what speed a rectangular region in an image is moving.

### Claim 8 (depends on 1)

The image display control device according to claim 1, wherein, in a case where part of the second image data is received, the substitute image generating section generates the substitute image further on a basis of the part.

### Claim 9 (depends on 1)

The image display control device according to claim 1, wherein the substitute image generating section generates the substitute image on a basis of a chronologically ordered series of the received image data.

### Claim 10

A transmitting device comprising:
an image data generating section that generates image data representing a downscaled image obtained by downscaling an original image; and
a transmitting section that transmits image data generated by the image data generating section to an image display control device,
the image display control device including
an image data receiving section that repeatedly receives the image data,
a to-be-displayed-image generating section that generates a to-be-displayed image having a resolution higher than a resolution of the downscaled image on a basis of the image data,
a substitute image generating section that, on a basis of first image data that is the image data received at a first timing, generates a substitute image that is an image different from the to-be-displayed image generated on the basis of the first image data, that corresponds to second image data that is the image data planned to be received at a second timing that comes after the first timing, and that has a resolution higher than the resolution of the downscaled image, and
a display control section that causes either one of the to-be-displayed image generated on a basis of the second image data and the substitute image generated on a basis of the first image data to be displayed according to success or failure of the reception of the second image data,
the substitute image generating section starting the generation of the substitute image corresponding to the second image data based on the first image data before the success or failure of the reception of the second image data is confirmed, wherein:
the transmitting section further transmits feature-amount data that is associated with the image data and that represents at least one of a feature amount of the original image and a feature amount that is a parameter used in encoding of the downscaled image, and
the transmitting section transmits the feature-amount data to the image display control device, prioritizing the transmission of the feature-amount data over transmission of the image data associated with the feature-amount data.

### Claim 11

An image display control method comprising:
repeatedly receiving image data, wherein: (i) the image data represents a downscaled image obtained by downscaling an original image, (ii) the image data is transmitted from a transmitting device, (iii) the transmitting device further transmits feature-amount data that is associated with the image data and that represents at least one of a feature amount of the original image and a feature amount that is a parameter used in encoding of the downscaled image, and (iv) the transmitting device transmits prioritizes the transmission of the feature-amount data over transmission of the image data associated with the feature-amount data;
generating a to-be-displayed image on a basis of the image data;
on a basis of first image data that is the image data received at a first timing, generating a substitute image that is an image different from the to-be-displayed image generated on a basis of the first image data and that corresponds to second image data that is the image data planned to be received at a second timing that comes after the first timing; and
causing either one of the to-be-displayed image generated on a basis of the second image data and the substitute image generated on a basis of the first image data to be displayed according to success or failure of the reception of the second image data,
wherein the generating the substitute image corresponding to the second image data on a basis of the first image data is started before the success or failure of the reception of the second image data is confirmed.

### Claim 12

A non-transitory, computer readable storage medium containing a computer program, which when executed by a computer, causes the computer to perform an image display control method by carrying out actions, comprising:
repeatedly receiving image data, wherein: (i) the image data represents a downscaled image obtained by downscaling an original image, (ii) the image data is transmitted from a transmitting device, (iii) the transmitting device further transmits feature-amount data that is associated with the image data and that represents at least one of a feature amount of the original image and a feature amount that is a parameter used in encoding of the downscaled image, and (iv) the transmitting device transmits prioritizes the transmission of the feature-amount data over transmission of the image data associated with the feature-amount data;
generating a to-be-displayed image on a basis of the image data;
on a basis of first image data that is the image data received at a first timing, generating a substitute image that is an image different from the to-be-displayed image generated on a basis of the first image data and that corresponds to second image data that is the image data planned to be received at a second timing that comes after the first timing; and
causing either one of the to-be-displayed image generated on a basis of the second image data and the substitute image generated on a basis of the first image data to be displayed according to success or failure of the reception of the second image data,
wherein the generating the substitute image corresponding to the second image data on a basis of the first image data is started before the success or failure of the reception of the second image data is confirmed.