# US 11,113,381 B2 · Providing an augmented reality overlay to secure input data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,381 B2 - Document ID: US11113381B2 - Status: Granted - Kind: B2 - Application number: 16/412,916 - Filed: 2019-05-15 - Published: 2019-12-12 - Granted: 2021-09-07 - Earliest claimed: 2018-06-06 - Assignees: Capital One Services, LLC - Applicants: Capital One Services, LLC - Inventors: Abdelkadar M'Hamed Benkreira; Michael Mossoba; Joshua Edwards - Web page: https://patentgrep.com/patent/US11113381B2 ## CPC - G06F 21/36: by graphic or iconic representation - G06F 3/044: by capacitive means - G06F 21/35: communicating wirelessly - G06F 21/83: input devices, e.g. keyboards, mice or controllers thereof - G06Q 20/1085: {involving automatic teller machines [ATMs]} - G06Q 20/322: {Aspects of commerce using mobile devices [M-devices]} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G07F 7/1041: {PIN input keyboard gets new key allocation at each use} - G07F 19/20: {Automatic teller machines [ATMs]} - G07F 19/201: {Accessories of ATMs} - G07F 19/2055: {Anti-skimming aspects at ATMs} - G07F 19/206: {Software aspects at ATMs} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G07F: COIN-FREED OR LIKE APPARATUS (coin sorting G07D3/00; coin testing G07D5/00; {handling coins or paper currencies apart from payment activated apparatus G07D; payment architectures, schemes or protocols G06Q20/00}) ## Abstract A device may receive an input that indicates a request to initiate a transaction at an ATM device. The device may instruct the user to capture one or more images of the ATM device. The device may determine that an image has been captured and process the image to determine first information that identifies the ATM device. The device may send the first information to a server device and receive a signal that indicates the ATM device has been validated. The device may cause an augmented reality (AR) overlay to be displayed, wherein the AR overlay includes second information related to authenticating the user to the ATM device. The device may determine whether a user action is performed with respect to the second information included in the AR overlay, and perform a device action related to the second information, the ATM device, or the AR overlay. ## Claims ### Claim 1 A device, comprising: one or more memory devices; and one or more processors, operatively coupled to the one or more memory devices, to: receive first information indicating that the device is in communication with an automated teller machine (ATM) device; cause an augmented reality (AR) overlay to be displayed, by the device, based on receiving the first information, the AR overlay including second information related to authenticating a user to the ATM device, the AR overlay being displayed to overlay an image of a touch capacitive screen of the ATM device, and the AR overlay displaying buttons, over the image of the touch capacitive screen, of one or more of varying sizes, of varying shapes, or at varying locations of the touch capacitive screen; determine whether the user interacts with the ATM device using the touch capacitive screen after causing the AR overlay to be displayed; and perform an action related to one or more of the second information, the ATM device, or the AR overlay based upon determining whether the user interacted with the ATM device. ### Claim 2 (depends on 1) The device of claim 1, wherein the second information includes at least one of: one or more characters displayed on at least one of the buttons, or one or more symbols displayed on at least one of the buttons. ### Claim 3 (depends on 1) The device of claim 1, wherein the first information includes at least one of: an ATM device identifier, information concerning transaction capabilities of the ATM device, or information concerning operation of the touch capacitive screen. ### Claim 4 (depends on 1) The device of claim 1, wherein the one or more processors, when determining whether the user interacts with the ATM device, are to: determine that the user, of the device, provided input to the ATM device using the touch capacitive screen. ### Claim 5 (depends on 1) The device of claim 1, wherein the one or more processors, when determining whether the user interacts with the ATM device, are to: monitor, using a camera of the device, for a user interaction with the touch capacitive screen. ### Claim 6 (depends on 1) The device of claim 1, wherein the one or more processors, when determining whether the user interacts with the ATM device, are to: receive, from the ATM device, data indicating a user interaction with the touch capacitive screen. ### Claim 7 (depends on 6) The device of claim 6, wherein the one or more processors, when performing the action, are further to: authorize a transaction with the ATM device based on the data indicating the user interaction and the second information. ### Claim 8 A method, comprising: receiving, by a device, first information indicating that the device is in communication with an automated teller machine (ATM) device; causing, by the device and based on receiving the first information, an augmented reality (AR) overlay to be displayed on the device, the AR overlay including second information related to authenticating a user to the ATM device, the AR overlay being displayed to overlay an image of a touch capacitive screen of the ATM device, and the AR overlay displaying buttons, over the image of the touch capacitive screen, of one or more of varying sizes, of varying shapes, or at varying locations of the touch capacitive screen; determining, by the device, whether a user action is performed with respect to the second information included in the AR overlay after causing the AR overlay to be displayed; and performing, by the device, an action related to one or more of the second information, the ATM device, or the AR overlay based upon determining whether the user action is performed with respect to the second information included in the AR overlay. ### Claim 9 (depends on 8) The method of claim 8, wherein the second information includes at least one of: one or more characters displayed on at least one of the buttons, or one or more symbols displayed on at least one of the buttons. ### Claim 10 (depends on 8) The method of claim 8, wherein the first information includes at least one of: an ATM device identifier, information concerning transaction capabilities of the ATM device, or information concerning operation of the touch capacitive screen. ### Claim 11 (depends on 8) The method of claim 8, wherein the user action comprises: providing input to the ATM device using the touch capacitive screen. ### Claim 12 (depends on 8) The method of claim 8, wherein determining whether the user action is performed comprises: monitoring, using a camera of the device, user interaction with the touch capacitive screen. ### Claim 13 (depends on 8) The method of claim 8, wherein determining whether the user action is performed comprises: receiving, from the ATM device, data indicating a user interaction with the touch capacitive screen. ### Claim 14 (depends on 13) The method of claim 13, wherein performing the action comprises: providing a supplemental overlay, based on the data indicating the user interaction and the second information, the supplemental overlay including additional instructions for display by the device. ### Claim 15 A non-transitory computer-readable medium storing instructions, the instructions comprising: one or more instructions that, when executed by one or more processors of a device, cause the one or more processors to: receive first information indicating that the device is in communication with an automated teller machine (ATM) device; cause an augmented reality (AR) overlay to be displayed, by the device, based on receiving the first information, the AR overlay including second information related to authenticating a user to the ATM device, the AR overlay being displayed to overlay an image of a touch capacitive screen of the ATM device, and the AR overlay displaying buttons, over the image of the touch capacitive screen, of one or more of varying sizes, of varying shapes, or at varying locations of the touch capacitive screen; determine whether the user interacts with the ATM device using the touch capacitive screen after causing the AR overlay to be displayed; and perform an action related to one or more of the second information, the ATM device, or the AR overlay based upon determining whether the user interacted with the ATM device. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the second information includes at least one of: one or more characters displayed on at least one of the buttons, or one or more symbols displayed on at least one of the buttons. ### Claim 17 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the first information includes at least one of: an ATM device identifier, information concerning transaction capabilities of the ATM device, or information concerning operation of the touch capacitive screen. ### Claim 18 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, that cause the one or more processors to determine whether the user interacts with the ATM device, cause the one or more processors to: determine that a user, of the device, provided input to the ATM device using the touch capacitive screen. ### Claim 19 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, that cause the one or more processors to determine whether the user interacts with the ATM device, cause the one or more processors to: monitor, using a camera of the device, for a user interaction with the touch capacitive screen. ### Claim 20 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, that cause the one or more processors to determine whether the user interacts with the ATM device, cause the one or more processors to: receive, from the ATM device, data indicating a user interaction with the touch capacitive screen.