# US 10,871,975 B2 · Modifying readable and focusable elements on a page during execution of automated scripts

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,871,975 B2
- Document ID: US10871975B2
- Status: Granted
- Kind: B2
- Application number: 16/850,042
- Filed: 2020-04-16
- Published: 2020-11-19
- Granted: 2020-12-22
- Earliest claimed: 2019-05-15
- Assignees: Capital One Services, LLC
- Applicants: Capital One Service, LLC
- Inventors: Vu Nguyen; Jonatan Yucra Rodriguez; Adam Vukich; Joshua Edwards; David Gabriele; Abdelkadar M'Hamed Benkreira
- Web page: https://patentgrep.com/patent/US10871975B2

## CPC

- G06F 9/451: Execution arrangements for user interfaces
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/0489: using dedicated keyboard keys or combinations thereof
- G06F 3/04895: Guidance during keyboard input operation, e.g. prompting
- G06F 9/453: {Help systems}
- G06F 16/958: Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
- G06F 16/986: {Document structures and storage, e.g. HTML extensions}
- G06F 40/14: Tree-structured documents (parsing G06F40/205; validation G06F40/226)
- G06F 40/143: Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
- G06F 40/154: Tree transformation for tree-structured or markup documents, e.g. XSLT, XSL-FO or stylesheets
- H04L 67/568: Storing data temporarily at an intermediate stage, e.g. caching
- G06F 9/45529: {Embedded in an application, e.g. JavaScript in a Web browser}
- H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A device may initiate an automated script to perform one or more interactions with a browser application and identify a first element in a page rendered by the browser application that satisfies one or more accessibility criteria, wherein the first element may include text that is readable by a screen reader application and/or an attribute that causes the first element to be navigable using a keyboard. The device may modify the first element to be inaccessible to the screen reader application and the keyboard and insert, into the page, a second element including text that is readable by the screen reader application to describe the one or more interactions that the automated script is performing. The client device may restore the page to an original state based on determining that the automated script has finished executing.

## Claims

### Claim 1

A method, comprising:
initiating, by a device, an automated script to perform one or more interactions with a browser application executing on the device;
identifying, by the device, a set of elements in a page rendered by the browser application that satisfy one or more accessibility criteria,
wherein the set of elements include at least one content element that includes text readable by a screen reader application executing on the device and at least one focusable element that is navigable using a keyboard associated with the device;
modifying, by the device, content on the page to make the text associated with the at least one content element inaccessible to the screen reader application and to make the at least one focusable element inaccessible using the keyboard associated with the device; and
inserting, by the device, at least one explanatory element into the page based on the one or more interactions that the automated script is performing,
wherein the at least one explanatory element includes text that is readable by the screen reader application to describe the one or more interactions that the automated script is performing, and
wherein the at least one explanatory element is navigable using the keyboard associated with the device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
caching an original state of the content on the page; and
restoring the cached original state of the content on the page based on determining that the automated script has finished executing.

### Claim 3 (depends on 2)

The method of claim 2, wherein caching the original state of the content on the page includes:
storing an original state of the at least one content element in association with a first unique identifier; and
storing an original state of the at least one focusable element in association with a second unique identifier,
wherein the first unique identifier and the second unique identifier are to be used to restore the original state of the at least one content element and the original state of the at least one focusable element when the automated script has finished executing.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
storing, in a cache, an original state of the content on the page; and
removing the original state of the content on the page from the cache based on determining that the automated script has caused the browser application to load a new page.

### Claim 5 (depends on 1)

The method of claim 1, wherein the text associated with the at least one content element is defined in one or more of an attribute associated with the at least one content element or a section between an opening tag and a closing tag of the at least one content element.

### Claim 6 (depends on 1)

The method of claim 1, wherein modifying the content on the page to make the at least one focusable element inaccessible using the keyboard associated with the device includes:
associating the at least one focusable element with an attribute that causes the at least one focusable element to be omitted from a sequence of elements that are navigable using the keyboard associated with the device.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
detecting multimedia on the page that is configured to automatically play,
wherein the content on the page is further modified to disable the multimedia and to make the multimedia inaccessible to the screen reader application and the keyboard.

### Claim 8 (depends on 1)

The method of claim 1, wherein:
the set of elements further include at least one interactive element, and
the content on the page is further modified to make the at least one interactive element non-interactive.

### Claim 9

A device, comprising:
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, configured to:
initiate an automated script to perform one or more interactions with a browser application executing on the device;
identify a set of elements in a page rendered by the browser application that satisfy one or more accessibility criteria,
wherein the set of elements include at least one content element that includes text readable by a screen reader application executing on the device and at least one focusable element that is navigable using a keyboard associated with the device;
modify content on the page to make the text associated with the at least one content element inaccessible to the screen reader application and to make the at least one focusable element inaccessible using the keyboard associated with the device; and
insert at least one explanatory element into the page based on the one or more interactions that the automated script is performing,
wherein the at least one explanatory element is navigable using the keyboard associated with the device.

### Claim 10 (depends on 9)

The device of claim 9, wherein the one or more processors, when modifying the content on the page, are to:
detect one or more interactive elements on the page; and
modify the content on the page to make the one or more interactive elements unselectable.

### Claim 11 (depends on 9)

The device of claim 9, wherein the one or more processors are further configured to:
store, in the one or more memories, an original state of the page prior to:
modifying the at least one content element and the at least one focusable element, and
inserting the at least one explanatory element,
wherein the original state of the page is to be obtained from the one or more memories and used to restore the page to the original state.

### Claim 12 (depends on 9)

The device of claim 9, wherein the one or more processors, when modifying the content on the page to make the text associated with the at least one content element inaccessible to the screen reader application, are to:
modify the content on the page by removing the at least one content element from the page.

### Claim 13 (depends on 9)

The device of claim 9, wherein the one or more processors are further configured to:
store, in a cache, an original state of the content on the page; and
remove the original state of the content on the page from the cache based on determining that the automated script has caused the browser application to load a new page.

### Claim 14 (depends on 9)

The device of claim 9, wherein the one or more processors are further configured to:
modify the content on the page to disable one or more multimedia elements.

### Claim 15

A non-transitory computer-readable medium storing instructions, the instructions comprising: one or more instructions that, when executed by one or more processors on a device, cause the one or more processors to:
initiate an automated script to perform one or more interactions with a browser application executing on the device;
identify a set of elements in a page rendered by the browser application that satisfy one or more accessibility criteria,
wherein the set of elements include at least one content element that includes text readable by a screen reader application executing on the device and at least one focusable element that is navigable using a keyboard associated with the device;
modify content on the page to make the text associated with the at least one content element inaccessible to the screen reader application and to make the at least one focusable element inaccessible using the keyboard associated with the device; and
insert at least one explanatory element into the page based on the one or more interactions that the automated script is performing.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the explanatory element includes one or more attributes that make the explanatory element navigable using the keyboard.

### Claim 17 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the text associated with the at least one content element is defined in a semantic header to assist in navigation of the at least one content element.

### Claim 18 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:
store an original state of the at least one content element in association with a first unique identifier; and
store an original state of the at least one focusable element in association with a second unique identifier,
wherein the first unique identifier and the second unique identifier are to be used to restore the original state of the at least one content element and the original state of the at least one focusable element when the automated script has finished executing.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:
restore one or more original elements on the page based on determining that the automated script has completed executing.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:
store, in a cache, an original state of the content on the page; and
remove the original state of the content on the page from the cache based on determining that the automated script has caused the browser application to load a new page.