US 2025/0085784 A1Published application

Multi-dimensional extrasensory deciphering of human gestures for digital authentication and related events

AssigneeBank of America Corporation

InventorsSubburathinam Krishnan, Abhishek Arya, Prabhakaran Balasubramanian, Durga Prasad P. Khuttumolu

Life of this patent

Estimating the term…

Loading patent…

Abstract

Virtual assistant devices and related methods are described that allow users having special needs to communicate with the virtual assistant device. The virtual assistant devices are augmented with additional intelligent sensors, including, but not limited to, three-dimensional optical sensors that are used to capture optical signals of user's non-verbal communication (e.g., sign language, lip movements, gestures or the like). Additionally, the virtual assistant device includes an Artificial Intelligence (AI)-based engine that includes one or more Machine Learning (ML) models trained on user-specific data and used to determine the non-verbal communication of the user (i.e., sign language, lip movements or other gestures) based on inputs derived from the sensors. Moreover, the virtual assistant devices may be configured to generate and display visual information, such as three-dimensional floating avatars of the user and/or the virtual assistant.

Claims18

Dependent claims

A system for multi-dimensional extrasensory deciphering of non-verbal communication of a user, the system comprising:

a virtual assistant device including a memory, one or more computing processing devices in communication with the memory and a plurality of sensors in communication with at least one of the one or more computing processing devices, wherein the plurality of sensors include at least one three-dimensional optical sensor configured for capturing, at least, optical signals that detect motion of the user;
an artificial intelligence (AI)-based engine stored in the memory, executable by at least one of the one or more computing processing devices, and including at least one first machine-learning (ML) model trained at least on user-specific data including historical resource exchange event data for purposes of determining non-verbal communication of the user, wherein the AI-based engine is configured to:
receive, at one or more of the first ML models, first inputs derived from signals captured from the plurality of sensors including the optical signals captured from the at least one three-dimensional optical sensor,
based on the first inputs, execute the one or more of the first ML models to determine a first transcript of the non-verbal communication of the user, and
communicate the first transcript of the non-verbal communication to a response system associated with the virtual assistant device, wherein the response system processes the non-verbal communication to determine a responsive communication to provide to the user;
and
a floating image-generating and communication engine stored in the memory, executable by at least one of the one or more computing processing devices and configured to:
generate an avatar that is a three-dimensional floating image of a virtual assistant,
display the avatar in a three-dimensional-enabled display system, wherein the avatar is configured to perform movements that provide the responsive communication to the user in a non-verbal communication format, and
adjust the movements performed by the avatar based on the detected movements of the avatar in comparison to the responsive communication.

7 more elements

A computer-implemented method for multi-dimensional extrasensory deciphering of non-verbal communication of a user, at least a portion of the method being executable by one or more computing device processors and comprising:

capturing optical signals from at least one three-dimensional optical sensor disposed within a virtual assistant device;
receiving, at one or more first ML models, first inputs derived from the optical signals captured from the at least one three-dimensional optical sensor;
based on the first inputs, executing the one or more first ML models to determine a first transcript of the non-verbal communication of the user;
communicating first transcript of the non-verbal communication to a response system associated with the virtual assistant device, wherein the response system processes the non-verbal communication to determine a responsive communication to provide to the user;
generating an avatar that is a three-dimensional floating image of a virtual assistant;
displaying the avatar in a three-dimensional-enabled display system, wherein the avatar is configured to perform movements that provide the responsive communication to the user in a non-verbal communication format;
and
adjusting the movements performed by the avatar based on the detected movements of the avatar in comparison to the responsive communication.

5 more elements

A computer program product comprising:

a non-transitory computer-readable medium comprising sets of codes for causing one or more computing devices to:
capture optical signals from at least one three-dimensional optical sensor disposed within a virtual assistant device, wherein the optical signals detect motion of a user;
receive, at one or more first ML models, first inputs derived from the optical signals captured from the at least one three-dimensional optical sensor;
based on the first inputs, (i) execute the one or more first ML models to determine a first transcript of the non-verbal communication of the user based on the detected motion;
communicate the first transcript of the non-verbal communication to a response system associated with the virtual assistant device, wherein the response system processes the non-verbal communication to determine a responsive communication to provide to the user;
generate an avatar that is a three-dimensional floating image of a virtual assistant;
display the avatar in a three-dimensional-enabled display system, wherein the avatar is configured to perform movements that provide the responsive communication to the user in a non-verbal communication format;
and
adjust the movements performed by the avatar based on the detected movements of the avatar in comparison to the responsive communication.

6 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/755,536Filed 26 Jun 2024
Priority: 7 Sep 2023
Published: 13 Mar 2025
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

G06F 3/017
Input arrangements or combined input and output arrangements for interaction between user and computer: gesture based interaction, e.g. based on a…
G06F 3/011
Input arrangements or combined input and output arrangements for interaction between user and computer: arrangements for interaction with the…
G06F 21/31
Authentication, i.e. establishing the identity or authorisation of security principals: user authentication
G06T 13/40
Three-dimensional [3D] animation: of characters, e.g. humans, animals or virtual beings

Family

Application18/243,292Continuation · filed 7 Sep 2023
Application18/755,536Continuation
US 12,099,661 B1Related patent
US 12,393,281 B2Related patent
US 2025/0085784 A1This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key