US 10,540,445 B2 · Intelligent integration of graphical elements into context for screen reader applications

Abstract

A mechanism is provided for intelligently integrating descriptions of images into surrounding text for a screen reader. A natural language understanding image description is determined for an image in a document. For each sentence of a set of sentences in the text of the document, a relatedness score between the sentence and the natural language understanding image description is determined thereby forming a set of relatedness scores. A highest relatedness score is determined from the set of relatedness scores. The natural language image description is inserted in close proximity to a sentence associated with the highest relatedness score, such that, when the text is read out by the screen reader, the natural language image description of the image is read out in close proximity to the sentence.

Claims20

Dependent claims

A method, in a data processing system, for intelligently integrating descriptions of images into text of a document for a screen reader, the method comprising:

determining a natural language understanding image description for an image in the document;
for each sentence of a set of sentences in the text of the document, determining a relatedness score between the sentence and the natural language understanding image description of the image in the document, thereby forming a set of relatedness scores;
determining a highest relatedness score from the set of relatedness scores;
and
when the text of the document is read out by the screen reader, reading out, by the screen reader, the natural language image description of the image in the document in close proximity to a sentence associated with the highest relatedness score.

2 more elements

A computer program product comprising a non-transitory computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:

determine a natural language understanding image description for an image in a document;
for each sentence of a set of sentences in the text of the document, determine a relatedness score between the sentence and the natural language understanding image description, of the image in the document, thereby forming a set of relatedness scores;
determine a highest relatedness score from the set of relatedness scores;
and
when the text of the document is read out by the screen reader, read out, by the screen reader, the natural language image description of the image in the document in close proximity to a sentence associated with the highest relatedness score.

2 more elements

An apparatus comprising:

a processor;
and
a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to:
determine a natural language understanding image description for an image in a document;
for each sentence of a set of sentences in the text of the document, determine a relatedness score between the sentence and the natural language understanding image description of the image in the document, thereby forming a set of relatedness scores;
determine a highest relatedness score from the set of relatedness scores;
and
when the text of the document is read out by the screen reader, read out, by the screen reader, the natural language image description of the image in the document in close proximity to a sentence associated with the highest relatedness score.

5 more elements

Dependencies are inferred from the claim text.

Intelligent integration of graphical elements into context for screen reader applications

Life of this patent

Intelligent integration of graphical elements into context for screen reader applications

Life of this patent

Abstract

Claims20