# US 2017/0287444 A1 · Display control device, display control method, and display control program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0287444 A1
- Document ID: US20170287444A1
- Status: Published application
- Kind: A1
- Application number: 15/508,402
- Filed: 2015-09-04
- Published: 2017-10-05
- Earliest claimed: 2014-09-08
- Assignees: Nippon Telegraph and Telephone Corporation
- Applicants: Nippon Telegraph and Telephone Corporation
- Inventors: Takeshi MASUDA
- Web page: https://patentgrep.com/patent/US20170287444A1

## CPC

- G09G 5/377: for mixing or overlaying two or more graphic patterns (G09G5/02, G09G5/397 take precedence)
- G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34)
- G06F 3/0485: Scrolling or panning
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06V 20/635: {Overlay text, e.g. embedded captions in a TV programme}
- G09G 5/346: {for systems having a bit-mapped display memory}
- G09G 2310/04: Partial updating of the display screen
- G09G 2340/125: wherein one of the images is motion video
- G09G 2360/121: using a cache memory
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A display control device (10) detects change of a screen, and executes, depending on the detected change of the screen, any one or both of provisional update processing that generates a strip-shaped rectangular region extended in a vertical direction or a horizontal direction based on the position of an icon displayed on the screen, searches the screen for a region similar to a specified image template by using the rectangular region as a search range, and performs overlay display of an icon in the similar region, and definitive update processing that searches for a region similar to the image template by using a region larger than a search range of the provisional update processing as a search range, and performs overlay display of an icon at a position corresponding to the similar region.

## Claims

### Claim 1

A display control device comprising:
a detection unit configured to detect change of a screen; and
a display control unit configured to execute, depending on change of the screen detected by the detection unit, any one or both of provisional update processing that generates predetermined matching result information based on a result of calculation on a previous frame displayed on the screen, searches the screen for a region similar to a specified image template by using the predetermined matching result information as part of a matching condition for a next frame, and performs overlay display of an icon in the similar region, and definitive update processing that searches for a region similar to the image template by using a matching condition larger than the predetermined matching result information, and performs overlay display of an icon at a position corresponding to the similar region.

### Claim 2 (depends on 1)

The display control device according to claim 1, wherein the display control unit executes, depending on change of the screen detected by the detection unit, any one or both of provisional update processing that generates a predetermined region based on the position of the icon displayed on the screen, searches the screen for a region similar to a specified image template by using the predetermined region as a search range, and performs overlay display of an icon in the similar region, and definitive update processing that searches for a region similar to the image template by using a region larger than the predetermined region as a search range, and performs overlay display of an icon at a position corresponding to the similar region.

### Claim 3 (depends on 1)

The display control device according to claim 1, wherein the display control unit executes, depending on change of the screen detected by the detection unit, any one or both of provisional update processing that generates a predetermined substitute template image based on matching positional coordinates as a result of calculation on the previous frame displayed on the screen, sets the predetermined substitute template image as a comparison target bit map, searches the screen for a region that matches with the predetermined substitute template image, and performs overlay display of an icon in the matched region, and definitive update processing that, irrespective of the predetermined substitute template image, searches the screen for a region similar to a specified image template, and performs overlay display of an icon at a position corresponding to the similar region.

### Claim 4 (depends on 1)

The display control device according to claim 1, wherein
the detection unit detects progress of change of the screen and stop of change of the screen, and
the display control unit executes the provisional update processing when the detection unit detects the progress of change of the screen, and executes the definitive update processing when the detection unit detects the stop of change of the screen.

### Claim 5 (depends on 1)

The display control device according to claim 1, wherein
the display control unit executes the provisional update processing and the definitive update processing in parallel when the detection unit detects the change of the screen, and
in the provisional update processing, the matching result information is generated in next provisional update processing based on a position at which the icon is displayed in previous provisional update processing already executed and a result of calculation on the previous frame in the definitive update processing.

### Claim 6 (depends on 2)

The display control device according to claim 2, wherein
the display control unit executes the provisional update processing and the definitive update processing in parallel when the detection unit detects the change of the screen, and
in the provisional update processing, the predetermined region is generated in next provisional update processing based on a position at which the icon is displayed in previous provisional update processing already executed and a position at which the icon is displayed in the definitive update processing.

### Claim 7 (depends on 3)

The display control device according to claim 3, wherein
the display control unit executes the provisional update processing and the definitive update processing in parallel when the detection unit detects the change of the screen, and
in the provisional update processing, the predetermined substitute template image is generated in next provisional update processing based on matching positional coordinates as a result of calculation on the previous frame in the definitive update processing.

### Claim 8 (depends on 1)

The display control device according to claim 1, wherein the display control unit extracts a difference region of an image displayed on the screen between frames, searches the screen for a region similar to a specified image template by using the difference region as a search region, and performs overlay display of an icon in the similar region in the definitive update processing; and extracts the difference region, adds the difference region extracted in the definitive update processing to the extracted difference region, searches the screen for a region similar to a specified image template by using the added difference region as a search region, and performs overlay display of an icon in the similar region in the provisional update processing.

### Claim 9 (depends on 3)

The display control device according to claim 3, wherein the display control unit determines a predetermined comparison reference position on the screen, sequentially compares a comparison target pixel at the comparison reference position with a comparison pixel selected as a representative point among pixels on a bit map of the predetermined substitute template image, repeats the comparison at a next comparison reference position on the screen when no match is determined for at least one pixel value, performs comparison with all pixels on a bit map of the substitute template image in an optional order when a match is determined for all pixel values selected as representative points, repeats the comparison at a next comparison reference position on the screen when no match is determined for at least one pixel value as a result of the comparison with all pixels on the bit map in the optional order, and performs overlay display of the icon at a matching position as the comparison reference position on the screen when a match is determined in the comparison with all pixels.

### Claim 10 (depends on 1)

The display control device according to claim 1, wherein
the display control unit executes the provisional update processing and the definitive update processing when the detection unit detects the change of the screen,
in the definitive update processing, generates predetermined matching result information based on a result of calculation on the previous frame, and accumulates a plurality of generations of the predetermined matching result information in a cache,
in the provisional update processing, generates predetermined matching result information based on a result of calculation on the previous frame, and accumulates a plurality of generations of the predetermined matching result information in a cache, and
in the provisional update processing, generates a matching condition for the next frame in the provisional update processing by referring to any one or both of the cache used in the definitive update processing and the cache used in the provisional update processing based on a time stamp of the matching result information.

### Claim 11 (depends on 10)

The display control device according to claim 10, wherein, in the cache used in the definitive update processing and the cache used in the provisional update processing, the matching result information is stored in an information processing device that displays the screen, over an activated duration and a deactivated duration.

### Claim 12 (depends on 10)

The display control device according to claim 10, wherein, in the cache used in the definitive update processing and the cache used in the provisional update processing, the matching result information is categorized for each piece of information specifying an activated information processing device or information specifying a user, and is stored in a server capable of communicating.

### Claim 13

A display control method executed by a display control device, the display control method comprising:
a detection process of detecting change of a screen; and
a display control process of, depending on the change of the screen detected at the detection process, executing any one or both of provisional update processing that generates predetermined matching result information based on a result of calculation on a previous frame displayed on the screen, searches the screen for a region similar to a specified image template by using the predetermined matching result information as part of a matching condition for a next frame, and performs overlay display of an icon in the similar region, and definitive update processing that searches for a region similar to the image template by using a matching condition larger than the predetermined matching result information, and performs overlay display of an icon at a position corresponding to the similar region.

### Claim 14

A display control program configured to cause a computer to execute:
a detection step of detecting change of a screen; and
a display control step of executing, depending on the change of the screen detected at the detection step, any one or both of provisional update processing that generates predetermined matching result information based on a result of calculation on a previous frame displayed on the screen, searches the screen for a region similar to a specified image template by using the predetermined matching result information as part of a matching condition for a next frame, and performs overlay display of an icon in the similar region, and definitive update processing that searches for a region similar to the image template by using a matching condition larger than the predetermined matching result information, and performs overlay display of an icon at a position corresponding to the similar region.
