# US 2024/0394170 A1 · Systems and methods for detecting accessibility failures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0394170 A1
- Document ID: US20240394170A1
- Status: Published application
- Kind: A1
- Application number: 18/324,940
- Filed: 2023-05-26
- Published: 2024-11-28
- Earliest claimed: 2023-05-26
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Mark MORRISON; Tom VANDENBERGE
- Web page: https://patentgrep.com/patent/US20240394170A1

## CPC

- G06F 8/65: Updates (security arrangements therefor G06F21/57)
- G06F 11/3604: Analysis of software for verifying properties of programs (testing of software G06F11/3668)
- G06F 8/10: Requirements analysis; Specification techniques
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods for detecting accessibility failures. In some aspects, the system receives a request for evaluating accessibility for a user interface of an updated version of a software application, wherein the request includes the updated and previous versions of the software application, and previous textual data corresponding to the previous version. The system generates audio output by using a screen reader to process the user interface. Textual data is generated by processing the audio output using speech recognition. The system compares the textual data to previous textual data to determine whether feature differences are present between the updated and previous versions. The one or more feature differences are analyzed to identify a feature difference corresponding to an accessibility failure point. In response to identifying the accessibility failure point, the system determines actions for removing the accessibility failure point.

## Claims

### Claim 1

A system for evaluating accessibility of one or more software applications, the system comprising:
one or more processors; and
a non-transitory, computer readable medium comprising instructions that, when executed by the one or more processors, causes operations comprising:
receiving a request for evaluating accessibility for a user interface of an updated version of a software application, wherein the request comprises (1) the updated version of the software application, (2) a previous version of the software application, and (3) previous textual data corresponding to the previous version of the software application;
generating audio output using a screen reader configured to process textual or visual data of the user interface and the updated version of the software application;
generating textual data by processing the audio output of the user interface using speech recognition;
comparing the textual data to the previous textual data to determine a plurality of feature differences present between the updated version and previous version of the software application;
processing the textual data and the plurality of feature differences using a first machine learning model to identify one or more feature differences of the plurality of feature differences corresponding to unintentional design changes;
analyzing the one or more feature differences to identify a first feature difference corresponding to an accessibility failure point, wherein the accessibility failure point renders content in the user interface of the software application inaccessible to a visually impaired user;
in response to identifying the accessibility failure point, determining based on (1) the first feature difference, and (2) the updated version of the software application, one or more actions for removing the accessibility failure point; and
transmitting, to a remote device, a command for displaying the one or more actions for removing the accessibility failure point.

### Claim 2 (depends on 1)

The system of claim 1, wherein the instructions cause the one or more processors to perform operations comprising:
receiving a training dataset comprising (1) a plurality of textual data, (2) a plurality of feature differences, and (3) corresponding indications regarding whether a feature difference corresponds to unintentional or intentional design changes; and
training, based on the training dataset, the first machine learning model for identifying feature differences between versions of software applications that correspond to unintentional design changes.

### Claim 3 (depends on 1)

The system of claim 1, wherein determining one or more actions for removing the accessibility failure point comprises:
processing (1) the one or more feature differences and (2) the updated version of the software application using a second machine learning model for identifying actions for removing the accessibility failure point.

### Claim 4 (depends on 1)

The system of claim 1, wherein the textual data comprises (1) a string of text and (2) one or more timestamps and wherein generating the textual data comprises:
generating, for each of a plurality of segments of the string of text, a corresponding timestamp representing a time at which each segment was textualized.

### Claim 5 (depends on 1)

The system of claim 1, wherein the instructions cause the one or more processors to perform operations comprising:
automatically performing the one or more actions for removing the accessibility failure point; and
transmitting, to the remote device, a notification of successful completion of the one or more actions.

### Claim 6

A method for evaluating accessibility of one or more software applications, the method comprising:
receiving a request for evaluating accessibility for a user interface of an updated version of a software application, wherein the request comprises (1) the updated version of the software application, (2) a previous version of the software application, and (3) previous textual data corresponding to the previous version of the software application;
generating audio output using a screen reader configured to process textual or visual data of the user interface and the updated version of the software application;
generating textual data by processing the audio output of the user interface using speech recognition;
comparing the textual data to the previous textual data to determine a plurality of feature differences present between the updated version and previous version of the software application;
processing the textual data and the plurality of feature differences using a first machine learning model to identify one or more feature differences of the plurality of feature differences corresponding to unintentional design changes;
analyzing the one or more feature differences to identify a first feature difference corresponding to an accessibility failure point, wherein the accessibility failure point renders content in the user interface of the software application inaccessible to a visually impaired user; and
in response to identifying the accessibility failure point, determining based on (1) the first feature difference, and (2) the updated version of the software application, one or more actions for removing the accessibility failure point.

### Claim 7 (depends on 6)

The method of claim 6, further comprising transmitting, to a remote device, a command for displaying, at a remote device, the one or more actions for removing the accessibility failure point.

### Claim 8 (depends on 6)

The method of claim 6, wherein determining one or more actions for removing the accessibility failure point comprises retrieving, from a data structure, the one or more actions based on (1) the one or more feature differences and (2) the updated version of the software application.

### Claim 9 (depends on 6)

The method of claim 6, further comprising:
receiving a training dataset comprising (1) a plurality of textual data, (2) a plurality of feature differences, and (3) corresponding indications regarding whether a feature difference corresponds to unintentional or intentional design changes; and
training, based on the training dataset, the first machine learning model for identifying accessibility failure points to predict whether feature differences between versions of software applications correspond to unintentional design changes.

### Claim 10 (depends on 6)

The method of claim 6, wherein determining one or more actions for removing the accessibility failure point comprises:
processing (1) the one or more feature differences and (2) the updated version of the software application using a second machine learning model for identifying actions for removing the accessibility failure point.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
receiving a training dataset comprising (1) the one or more feature differences, (2) the updated version of the software application, and (3) corresponding actions that were performed to remove accessibility failure points; and
training, based on the training dataset, the second machine learning model for identifying actions for removing accessibility failure points.

### Claim 12 (depends on 6)

The method of claim 6, wherein the textual data comprises (1) a string of text and (2) one or more timestamps and wherein generating the textual data comprises:
generating, for each of a plurality of segments of the string of text, a corresponding timestamp representing a time at which each segment was textualized.

### Claim 13 (depends on 6)

The method of claim 6, further comprising:
automatically performing the one or more actions for removing accessibility failure points; and
transmitting, to a remote device a notification of successful completion of the one or more actions.

### Claim 14

A non-transitory, computer readable medium comprising instructions that, when executed by one or more processors, causes operations comprising:
receiving a request for evaluating accessibility for a user interface of an updated version of a software application, wherein the request comprises (1) the updated version of the software application, (2) a previous version of the software application, and (3) previous textual data corresponding to the previous version of the software application;
comparing textual data associated with the updated version to the previous textual data to determine a plurality of feature differences present between the updated version and previous version of the software application;
processing the textual data and the plurality of feature differences using a first machine learning model to identify one or more feature differences of the plurality of feature differences corresponding to unintentional design changes;
analyzing the one or more feature differences to identify a first feature difference corresponding to an accessibility failure point, wherein the accessibility failure point renders content in the user interface of the software application inaccessible to a visually impaired user; and
in response to identifying the accessibility failure point, determining one or more actions for removing the accessibility failure point.

### Claim 15 (depends on 14)

The non-transitory, computer readable medium of claim 14, further comprising:
generating audio output using a screen reader configured to process textual or visual data of the user interface and the updated version of the software application; and
generating the textual data by processing the audio output of the user interface using speech recognition.

### Claim 16 (depends on 14)

The non-transitory, computer readable medium of claim 14, wherein determining one or more actions removing the accessibility failure point comprises retrieving, from a data structure, the one or more actions based on (1) the one or more feature differences and (2) the updated version of the software application.

### Claim 17 (depends on 14)

The non-transitory, computer readable medium of claim 14, wherein the instructions cause the one or more processors to perform operations comprising: receiving a training dataset comprising (1) a plurality of textual data, (2) a plurality of feature differences, and (3) corresponding indications regarding whether a feature difference corresponds to unintentional or intentional design changes; and
training, based on the training dataset, the first machine learning model for identifying accessibility failure points to predict whether feature differences between versions of software applications correspond to unintentional design changes.

### Claim 18 (depends on 14)

The non-transitory, computer readable medium of claim 14, wherein determining the one or more actions for removing the accessibility failure point comprises:
processing (1) the one or more feature differences and (2) the updated version of the software application using a second machine learning model for identifying actions for removing the accessibility failure point.

### Claim 19 (depends on 18)

The non-transitory, computer readable medium of claim 18, wherein the instructions cause the one or more processors to perform operations comprising:
receiving a training dataset comprising (1) the one or more feature differences, (2) code of the updated version of the software application, and (3) corresponding actions that were performed to remove the accessibility failure point; and
training, based on the training dataset, the second machine learning model for identifying the actions for removing the accessibility failure point.

### Claim 20 (depends on 15)

The non-transitory, computer readable medium of claim 15, wherein the textual data comprises (1) a string of text and (2) one or more timestamps and wherein generating the textual data comprises:
generating, for each of a plurality of segments of the string of text, a corresponding timestamp representing a time at which each segment was textualized.
