# US 2023/0274081 A1 · Systems and methods for annotating line charts in the wild

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274081 A1
- Document ID: US20230274081A1
- Status: Published application
- Kind: A1
- Application number: 18/106,926
- Filed: 2023-02-07
- Published: 2023-08-31
- Earliest claimed: 2022-02-07
- Assignees: Arizona Board of Regents on behalf of Arizona State University
- Inventors: Arlen Fan; Yuxin Ma; Ross Maciejewski
- Web page: https://patentgrep.com/patent/US20230274081A1

## CPC

- G06F 40/169: Annotation, e.g. comment data or footnotes
- G06F 40/279: Recognition of textual entities
- G06F 40/30: Semantic analysis
- G06F 40/56: Natural language generation
- G06V 30/1444: {Selective acquisition, locating or processing of specific regions, e.g. highlighted text, fiducial marks or predetermined fields}
- G06V 30/19173: {Classification techniques}
- G06V 30/422: Technical drawings; Geographical maps
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The present disclosure describes examples of a computer-implemented framework that helps to detect deception in charts and/or associated articles through textual and visual annotations.

## Claims

### Claim 1

A system for annotating line charts in the wild, comprising:
processor in communication with a memory, the memory including instructions, which, when executed, cause the processor to:
receive an image featuring a graphical representation;
extract a plurality of features from the image featuring the graphical representation;
compare the plurality of features to a plurality of accepted guidelines for graphical representations; and
output an assessment of the image featuring the graphical representation.

### Claim 2 (depends on 1)

The system of claim 1, wherein the memory includes instructions, which, when executed, further cause the processor to:
generate a corrected image featuring the graphical representation such that the corrected image conforms to the plurality of accepted guidelines for graphical representations

### Claim 3 (depends on 1)

The system of claim 1, wherein the plurality of accepted guidelines for graphical representations include guidelines for:
truncation of a y-axis of a graphical representation;
inversion of the y-axis of the graphical representation; and
an aspect ratio of the graphical representation.

### Claim 4 (depends on 1)

The system of claim 1, wherein the memory includes instructions, which, when executed, further cause the processor to:
extract text and location of the text from the image featuring the graphical representation.

### Claim 5 (depends on 1)

The system of claim 1, wherein the memory includes instructions, which, when executed, further cause the processor to:
classify a role of the text extracted from the image featuring the graphical representation

### Claim 6 (depends on 1)

The system of claim 1, wherein the memory includes instructions, which, when executed, further cause the processor to:
select an aspect ratio for the image featuring the graphical representation such that an average of all line segments within the graphical representation is 45°.

### Claim 7 (depends on 6)

The system of claim 6, wherein the memory includes instructions, which, when executed, further cause the processor to:
calculate an ideal aspect ratio of the image featuring the graphical representation; and
provide an alert if a current aspect ratio deviates from the ideal aspect ratio by a multiplicative factor of log 10(ARlarge/ARsmall)>0.5.

### Claim 8

A system, comprising:
a processor in communication with a memory, the memory including instructions, which, when executed, cause the processor to:
extract input information from an article that includes data visualization and text, wherein text elements of the data visualization are extracted and predetermined text parameters are extracted from the text of the article;
conduct analysis of the input information to generate a summary of the data visualization which serves as a latent text representation of the data visualization and calculate semantic similarities between the latent text representation and each sentence in the text of the article itself; and
generate, using output of the the analysis of the input information, one or more annotations for the article based on a selected annotation category.

### Claim 9 (depends on 8)

The system of claim 8, wherein the processor executes a natural language processing (NLP) model to calculate the semantic similarities.

### Claim 10 (depends on 8)

The system of claim 8, wherein the predetermined text parameters include an article headline, and a date of publication for the article.

### Claim 11 (depends on 8)

The system of claim 8, wherein the data visualization is a chart.

### Claim 12 (depends on 11)

The system of claim 11, wherein the memory includes further instructions, which, when executed, cause the processor to:
generate an embedding echart for the summary and then an embedding for all of the sentences ei, where i is the index of the sentence, and
find a most similar sentence by calculating a cosine distance between the embeddings.
