# US 2018/0024982 A1 · Real-time dynamic visual aid implementation based on context obtained from heterogeneous sources

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0024982 A1
- Document ID: US20180024982A1
- Status: Published application
- Kind: A1
- Application number: 15/217,849
- Filed: 2016-07-22
- Published: 2018-01-25
- Earliest claimed: 2016-07-22
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Si Bin Fan; Yu Gu; Su Liu; Cheng Xu
- Web page: https://patentgrep.com/patent/US20180024982A1

## CPC

- G06F 40/186: Templates
- G06F 9/451: Execution arrangements for user interfaces
- G06F 9/453: {Help systems}
- G06F 16/345: {Summarisation for human users}
- G06F 16/35: Clustering; Classification
- G06F 16/40: of multimedia data, e.g. slideshows comprising image and additional audio data (retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70)
- G06F 16/487: using geographical or spatial information, e.g. location
- G06F 40/258: Heading extraction; Automatic titling; Numbering
- H04L 65/611: for multicast or broadcast (systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06)
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- H04L 67/535: {Tracking the activity of the user (network monitoring arrangements H04L43/00; recording of computer activity G06F11/34)}
- H04N 21/42203: {sound input device, e.g. microphone}
- H04N 21/44226: {on social networks}
- H04N 21/4722: for requesting additional data associated with the content
- H04N 21/8133: {specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

In one embodiment, a computer-implemented method includes extracting one or more keywords from summarized content according to one or more classified topics. The method also includes searching for visual aid elements that relate to the one or more keywords in a visual aid element repository that stores a plurality of visual aid elements. In addition, the method includes selecting one or more visual aid elements from the visual aid element repository based on a type of the one or more classified topics. Also, the method includes generating at least one visual aid object using the one or more visual aid elements based on at least one predefined visual aid template. Moreover, the method includes delivering the at least one visual aid object to one or more registered devices of at least one user.

## Claims

### Claim 1

A system, comprising:
a processing circuit; and
logic integrated with the processing circuit, executable by the processing circuit, or integrated with and executable by the processing circuit, the logic being configured to cause the processing circuit to:
extract one or more keywords in accordance with a set of predefined keyword extraction rules from summarized content that is summarized according to one or more classified topics;
search for visual aid elements that relate to the one or more keywords in a visual aid element repository that stores a plurality of visual aid elements;
select one or more visual aid elements from the visual aid element repository based on a type of the one or more classified topics;
modify the one or more visual aid elements to display information relevant to the summarized content;
generate at least one visual aid object using the one or more modified visual aid elements based on at least one predefined visual aid template; and
deliver the at least one visual aid object to one or more registered devices of at least one user, the at least one visual aid object being interactive for the at least one user.

### Claim 2 (depends on 1)

The system as recited in claim 1, wherein the logic is further configured to cause the processing circuit to:
monitor outputs of at least one content source to obtain unstructured or informally structured content from one or more heterogeneous resources; and
create the set of predefined keyword extraction rules via a profile of configuration settings.

### Claim 3 (depends on 2)

The system as recited in claim 2, wherein the logic is further configured to cause the processing circuit to:
summarize key topics of the content to produce summarized key topics; and
classify the summarized key topics to produce summarized content.

### Claim 4 (depends on 2)

The system as recited in claim 2, wherein the at least one content source is selected from a group comprising: an in-person speaker, a radio broadcast, a television (TV) broadcast, and an Internet data stream.

### Claim 5 (depends on 1)

The system as recited in claim 1, wherein the logic is further configured to cause the processing circuit to:
determine the one or more registered devices associated with one or more visual aid requesters prior to delivering the at least one visual aid object to the one or more registered devices.

### Claim 6 (depends on 1)

The system as recited in claim 1, wherein the one or more visual aid elements are selected from the group consisting of:
global positioning satellite (GPS) images;
maps;
addresses; and
businesses, stores, and residents in proximity to the addresses.

### Claim 7 (depends on 1)

The system as recited in claim 1, wherein the at least one predefined visual aid template is selected from a plurality of predefined visual aid templates stored in a visual aid template database, and wherein the at least one visual aid object is selected from the group consisting of:
a two-dimensional image;
a three-dimensional image;
a listing of keywords;
a map;
a relevant photo; and
a relevant video.

### Claim 8 (depends on 1)

The system as recited in claim 1, wherein the one or more registered devices on which the at least one visual aid object is delivered are different from a device on which the summarized content is obtained, and wherein the type of the one or more classified topics is selected from the group consisting of: geographical content, content pertaining to one or more people, content pertaining to one or more places, content pertaining to one or more historical events, and content pertaining to one or more current events.

### Claim 9

A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the embodied program instructions being executable by a processing circuit to cause the processing circuit to:
extract, by the processing circuit, one or more keywords in accordance with a set of predefined keyword extraction rules from summarized content that is summarized according to one or more classified topics;
search, by the processing circuit, for visual aid elements that relate to the one or more keywords in a visual aid element repository that stores a plurality of visual aid elements;
select, by the processing circuit, one or more visual aid elements from the visual aid element repository based on a type of the one or more classified topics;
modify, by the processing circuit, the one or more visual aid elements to display information relevant to the summarized content;
generate, by the processing circuit, at least one visual aid object using the one or more modified visual aid elements based on at least one predefined visual aid template; and
deliver, by the processing circuit, the at least one visual aid object to one or more registered devices of at least one user, the at least one visual aid object being interactive for the at least one user.

### Claim 10 (depends on 9)

The computer program product as recited in claim 9, wherein the embodied program instructions are further executable by the processing circuit to cause the processing circuit to:
monitor, by the processing circuit, outputs of at least one content source to obtain unstructured or informally structured content from one or more heterogeneous resources;
create, by the processing circuit, the set of predefined keyword extraction rules via a profile of configuration settings;
summarize, by the processing circuit, key topics of the content to produce summarized key topics; and
classify, by the processing circuit, the summarized key topics to produce summarized content.

### Claim 11 (depends on 10)

The computer program product as recited in claim 10, wherein the at least one content source is selected from a group comprising: an in-person speaker, a radio broadcast, a television (TV) broadcast, and an Internet data stream.

### Claim 12 (depends on 9)

The computer program product as recited in claim 9, wherein the embodied program instructions are further executable by the processing circuit to cause the processing circuit to:
determine, by the processing circuit, the one or more registered devices associated with one or more visual aid requesters prior to delivering the at least one visual aid object to the one or more registered devices.

### Claim 13 (depends on 9)

The computer program product as recited in claim 9, wherein the one or more visual aid elements are selected from the group consisting of:
global positioning satellite (GPS) images;
maps;
addresses; and
businesses, stores, and residents in proximity to the addresses.

### Claim 14 (depends on 9)

The computer program product as recited in claim 9, wherein the at least one predefined visual aid template is selected from a plurality of predefined visual aid templates stored in a visual aid template database, and wherein the at least one visual aid object is selected from the group consisting of:
a two-dimensional image;
a three-dimensional image;
a listing of keywords;
a map;
a relevant photo; and
a relevant video.

### Claim 15 (depends on 9)

The computer program product as recited in claim 9, wherein the one or more registered devices on which the at least one visual aid object is delivered are different from a device on which the summarized content is obtained, and wherein the type of the one or more classified topics is selected from the group consisting of: geographical content, content pertaining to one or more people, content pertaining to one or more places, content pertaining to one or more historical events, and content pertaining to one or more current events.

### Claim 16

A computer-implemented method, the method comprising:
extracting one or more keywords in accordance with a set of predefined keyword extraction rules from summarized content that is summarized according to one or more classified topics;
searching for visual aid elements that relate to the one or more keywords in a visual aid element repository that stores a plurality of visual aid elements;
selecting one or more visual aid elements from the visual aid element repository based on a type of the one or more classified topics;
modifying the one or more visual aid elements to display information relevant to the summarized content;
generating at least one visual aid object using the one or more modified visual aid elements based on at least one predefined visual aid template; and
delivering the at least one visual aid object to one or more registered devices of at least one user, the at least one visual aid object being interactive for the at least one user.

### Claim 17 (depends on 16)

The method as recited in claim 16, further comprising:
monitoring outputs of at least one content source to obtain unstructured or informally structured content from one or more heterogeneous resources;
creating the set of predefined keyword extraction rules via a profile of configuration settings;
summarizing key topics of the content to produce summarized key topics;
classifying the summarized key topics to produce summarized content; and
determining the one or more registered devices associated with one or more visual aid requesters prior to delivering the at least one visual aid object to the one or more registered devices,
wherein the at least one content source is selected from the group consisting of: a radio broadcast, a television (TV) broadcast, and an Internet data stream.

### Claim 18 (depends on 16)

The method as recited in claim 16, wherein the one or more visual aid elements are selected from the group consisting of:
global positioning satellite (GPS) images;
maps;
addresses; and
businesses, stores, and residents in proximity to the addresses.

### Claim 19 (depends on 16)

The method as recited in claim 16, wherein the at least one predefined visual aid template is selected from a plurality of predefined visual aid templates stored in a visual aid template database, and wherein the at least one visual aid object is selected from the group consisting of: a two-dimensional image, a three-dimensional image, a listing of keywords, a map, a relevant photo, and a relevant video.

### Claim 20 (depends on 16)

The method as recited in claim 16, wherein the one or more registered devices on which the at least one visual aid object is delivered are different from a device on which the summarized content is obtained, and wherein the type of the one or more classified topics is selected from the group consisting of: geographical content, content pertaining to one or more people, content pertaining to one or more places, content pertaining to one or more historical events, and content pertaining to one or more current events.
