# US 10,768,951 B2 · Providing augmented reality user interfaces and controlling automated systems based on user activity information and pre-staging information > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,768,951 B2 - Document ID: US10768951B2 - Status: Granted - Kind: B2 - Application number: 16/115,752 - Filed: 2018-08-29 - Published: 2020-03-05 - Granted: 2020-09-08 - Earliest claimed: 2018-08-29 - Assignees: Bank of America Corporation - Applicants: Bank of America Corporation - Inventors: Matthew E. Carroll - Web page: https://patentgrep.com/patent/US10768951B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 9/451: Execution arrangements for user interfaces - H04L 67/131: Protocols for games, networked simulations or virtual reality - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Aspects of the disclosure relate to providing augmented reality user interfaces and controlling automated systems based on user activity information and pre-staging information. A computing platform may receive, from a client user device, a trip start notification indicating that a user of the client user device is initiating a trip to an enterprise center. In response to receiving the trip start notification, the computing platform may generate a pre-staging augmented reality user interface for a client augmented reality device linked to the client user device. Thereafter, the computing platform may receive pre-staging information identifying one or more events to be performed at the enterprise center when the user of the client user device arrives at the enterprise center. The computing platform may generate one or more pre-staging commands based on the pre-staging information and may send these commands to one or more systems associated with the enterprise center. ## Claims ### Claim 1 A computing platform, comprising: at least one processor; a communication interface communicatively coupled to the at least one processor; and memory storing computer-readable instructions that, when executed by the at least one processor, cause the computing platform to: receive, via the communication interface, from a client user device, a trip start notification indicating that a user of the client user device is initiating a trip to an enterprise center; in response to receiving the trip start notification from the client user device, generate a pre-staging augmented reality user interface for a client augmented reality device linked to the client user device; send, via the communication interface, to the client augmented reality device linked to the client user device, the pre-staging augmented reality user interface generated for the client augmented reality device linked to the client user device, wherein sending the pre-staging augmented reality user interface generated for the client augmented reality device linked to the client user device causes the client augmented reality device linked to the client user device to display the pre-staging augmented reality user interface and prompt the user of the client user device to provide pre-staging input; receive, via the communication interface, from the client augmented reality device linked to the client user device, pre-staging information identifying one or more events to be performed at the enterprise center when the user of the client user device arrives at the enterprise center; generate one or more pre-staging commands based on the pre-staging information identifying the one or more events to be performed at the enterprise center; send, via the communication interface, to one or more systems associated with the enterprise center, the one or more pre-staging commands generated based on the pre-staging information identifying the one or more events to be performed at the enterprise center; after sending the one or more pre-staging commands to the one or more systems associated with the enterprise center, generate a navigation augmented reality user interface for the client augmented reality device linked to the client user device; and send, via the communication interface, to the client augmented reality device linked to the client user device, the navigation augmented reality user interface generated for the client augmented reality device linked to the client user device. ### Claim 2 (depends on 1) The computing platform of claim 1, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, cause the computing platform to: prior to receiving the trip start notification from the client user device: receive, via the communication interface, from enterprise institution server infrastructure, a notification generated by an artificial-intelligence engine executing on the enterprise institution server infrastructure, the notification identifying at least one predicted event for the user of the client user device; and send, via the communication interface, to the client user device, the notification received from the enterprise institution server infrastructure. ### Claim 3 (depends on 1) The computing platform of claim 1, wherein receiving the trip start notification from the client user device comprises receiving information indicating an estimated time of arrival determined by an autonomous vehicle being used by the user of the client user device to travel to the enterprise center. ### Claim 4 (depends on 1) The computing platform of claim 1, wherein receiving the trip start notification from the client user device comprises receiving information indicating an estimated time of arrival determined by a ride sharing application associated with a ride sharing service being used by the user of the client user device to travel to the enterprise center. ### Claim 5 (depends on 1) The computing platform of claim 1, wherein sending the pre-staging augmented reality user interface to the client augmented reality device linked to the client user device comprises sending the pre-staging augmented reality user interface to an augmented reality headset linked to the client user device. ### Claim 6 (depends on 1) The computing platform of claim 1, wherein sending the pre-staging augmented reality user interface to the client augmented reality device linked to the client user device comprises sending the pre-staging augmented reality user interface to an augmented reality windshield display system linked to the client user device. ### Claim 7 (depends on 1) The computing platform of claim 1, wherein receiving the pre-staging information identifying the one or more events to be performed at the enterprise center comprises receiving information identifying a request to pick up at least one physical item at the enterprise center. ### Claim 8 (depends on 1) The computing platform of claim 1, wherein receiving the pre-staging information identifying the one or more events to be performed at the enterprise center comprises receiving information identifying a request to drop off at least one physical item at the enterprise center. ### Claim 9 (depends on 1) The computing platform of claim 1, wherein receiving the pre-staging information identifying the one or more events to be performed at the enterprise center comprises receiving information identifying a request for a specific encounter at the enterprise center. ### Claim 10 (depends on 1) The computing platform of claim 1, wherein sending the one or more pre-staging commands to the one or more systems associated with the enterprise center comprises sending at least one command directing an enterprise center automation platform associated with the enterprise center to execute one or more automated physical item loading actions. ### Claim 11 (depends on 1) The computing platform of claim 1, wherein sending the one or more pre-staging commands to the one or more systems associated with the enterprise center comprises sending at least one command directing an enterprise center user device to present a notification instructing a user of the enterprise center user device to perform one or more manual physical item loading actions. ### Claim 12 (depends on 1) The computing platform of claim 1, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, cause the computing platform to: receive, via the communication interface, from the client user device, a trip arrival notification indicating that the user of the client user device has completed the trip to the enterprise center; in response to receiving the trip arrival notification from the client user device, generate an arrival-assistance augmented reality user interface for the client augmented reality device linked to the client user device; and send, via the communication interface, to the client augmented reality device linked to the client user device, the arrival-assistance augmented reality user interface generated for the client augmented reality device linked to the client user device, wherein sending the arrival-assistance augmented reality user interface generated for the client augmented reality device linked to the client user device causes the client augmented reality device linked to the client user device to display the arrival-assistance augmented reality user interface and notify the user of the client user device of one or more locations at the enterprise center to visit to complete the one or more events to be performed at the enterprise center. ### Claim 13 (depends on 1) The computing platform of claim 1, wherein the memory stores additional computer-readable instructions that, when executed by the at least one processor, cause the computing platform to: generate a destination activity notification for the client augmented reality device linked to the client user device, the destination activity notification generated for the client augmented reality device linked to the client user device comprising information associated with the one or more events to be performed at the enterprise center; and send, via the communication interface, to the client augmented reality device linked to the client user device, the destination activity notification generated for the client augmented reality device linked to the client user device. ### Claim 14 A method, comprising: at a computing platform comprising at least one processor, a communication interface, and memory: receiving, by the at least one processor, via the communication interface, from a client user device, a trip start notification indicating that a user of the client user device is initiating a trip to an enterprise center; in response to receiving the trip start notification from the client user device, generating, by the at least one processor, a pre-staging augmented reality user interface for a client augmented reality device linked to the client user device; sending, by the at least one processor, via the communication interface, to the client augmented reality device linked to the client user device, the pre-staging augmented reality user interface generated for the client augmented reality device linked to the client user device, wherein sending the pre-staging augmented reality user interface generated for the client augmented reality device linked to the client user device causes the client augmented reality device linked to the client user device to display the pre-staging augmented reality user interface and prompt the user of the client user device to provide pre-staging input; receiving, by the at least one processor, via the communication interface, from the client augmented reality device linked to the client user device, pre-staging information identifying one or more events to be performed at the enterprise center when the user of the client user device arrives at the enterprise center; generating, by the at least one processor, one or more pre-staging commands based on the pre-staging information identifying the one or more events to be performed at the enterprise center; sending, by the at least one processor, via the communication interface, to one or more systems associated with the enterprise center, the one or more pre-staging commands generated based on the pre-staging information identifying the one or more events to be performed at the enterprise center; after sending the one or more pre-staging commands to the one or more systems associated with the enterprise center, generating, by the at least one processor, a navigation augmented reality user interface for the client augmented reality device linked to the client user device; and sending, by the at least one processor, via the communication interface, to the client augmented reality device linked to the client user device, the navigation augmented reality user interface generated for the client augmented reality device linked to the client user device. ### Claim 15 (depends on 14) The method of claim 14, comprising: prior to receiving the trip start notification from the client user device: receiving, by the at least one processor, via the communication interface, from enterprise institution server infrastructure, a notification generated by an artificial-intelligence engine executing on the enterprise institution server infrastructure, the notification identifying at least one predicted event for the user of the client user device; and sending, by the at least one processor, via the communication interface, to the client user device, the notification received from the enterprise institution server infrastructure. ### Claim 16 (depends on 14) The method of claim 14, wherein receiving the trip start notification from the client user device comprises receiving information indicating an estimated time of arrival determined by an autonomous vehicle being used by the user of the client user device to travel to the enterprise center. ### Claim 17 (depends on 14) The method of claim 14, wherein receiving the trip start notification from the client user device comprises receiving information indicating an estimated time of arrival determined by a ride sharing application associated with a ride sharing service being used by the user of the client user device to travel to the enterprise center. ### Claim 18 (depends on 14) The method of claim 14, wherein sending the pre-staging augmented reality user interface to the client augmented reality device linked to the client user device comprises sending the pre-staging augmented reality user interface to an augmented reality headset linked to the client user device. ### Claim 19 (depends on 14) The method of claim 14, comprising: receiving, by the at least one processor, via the communication interface, from the client user device, a trip arrival notification indicating that the user of the client user device has completed the trip to the enterprise center; in response to receiving the trip arrival notification from the client user device, generating, by the at least one processor, an arrival-assistance augmented reality user interface for the client augmented reality device linked to the client user device; and sending, by the at least one processor, via the communication interface, to the client augmented reality device linked to the client user device, the arrival-assistance augmented reality user interface generated for the client augmented reality device linked to the client user device, wherein sending the arrival-assistance augmented reality user interface generated for the client augmented reality device linked to the client user device causes the client augmented reality device linked to the client user device to display the arrival-assistance augmented reality user interface and notify the user of the client user device of one or more locations at the enterprise center to visit to complete the one or more events to be performed at the enterprise center. ### Claim 20 One or more non-transitory computer-readable media storing instructions that, when executed by a computing platform comprising at least one processor, a communication interface, and memory, cause the computing platform to: receive, via the communication interface, from a client user device, a trip start notification indicating that a user of the client user device is initiating a trip to an enterprise center; in response to receiving the trip start notification from the client user device, generate a pre-staging augmented reality user interface for a client augmented reality device linked to the client user device; send, via the communication interface, to the client augmented reality device linked to the client user device, the pre-staging augmented reality user interface generated for the client augmented reality device linked to the client user device, wherein sending the pre-staging augmented reality user interface generated for the client augmented reality device linked to the client user device causes the client augmented reality device linked to the client user device to display the pre-staging augmented reality user interface and prompt the user of the client user device to provide pre-staging input; receive, via the communication interface, from the client augmented reality device linked to the client user device, pre-staging information identifying one or more events to be performed at the enterprise center when the user of the client user device arrives at the enterprise center; generate one or more pre-staging commands based on the pre-staging information identifying the one or more events to be performed at the enterprise center; send, via the communication interface, to one or more systems associated with the enterprise center, the one or more pre-staging commands generated based on the pre-staging information identifying the one or more events to be performed at the enterprise center; after sending the one or more pre-staging commands to the one or more systems associated with the enterprise center, generate a navigation augmented reality user interface for the client augmented reality device linked to the client user device; and send, via the communication interface, to the client augmented reality device linked to the client user device, the navigation augmented reality user interface generated for the client augmented reality device linked to the client user device.