Input Commands via Visual Cues
AssigneeCapital One Services, LLC
InventorsHannes Jouhikainen, Timur Sherif, Steve Blanchet
Life of this patent
Estimating the term…
Abstract
Embodiments disclosed herein generally relate to a method and system of generating text input via facial recognition. A computing system receives a video stream of a user operating an application on a client device. The video stream includes a time series of images of the user. The computing system parses the video stream to identify one or more visual cues of the user. The computing system identifies a current page of the application. The computing system maps the identified on or more visual cues to an operation available on the current page of the application. The computing system executes the mapped operation.
Claims20
A method, comprising:
- displaying, by a computing device, a portion of an application to a user operating the computing device;
- requesting, by the computing device, permission to capture video data associated with the user interacting with the portion of the application,
- receiving, by the computing device, permission to capture video data of the user;
- receiving, by the computing device, a video stream of the user interacting with the portion of the application, the video stream comprising a time series of images of the user;
- parsing, by the computing device, the video stream to identify one or more visual cues of the user;
- identifying, by the computing system, a current portion of the application accessed by the user;
- mapping, by the computing system, the identified visual cues of the user to an operation available on the current portion of the application;
- and
- executing, by the computing system, the mapped operation.
A system, comprising:
- a processor;
- and
- memory storing program code, which, when executed by the processor, performs an operation, comprising:
- displaying a portion of an application to a user operating the computing device;
- requesting permission to capture video data associated with the user interacting with the portion of the application,
- receiving permission to capture video data of the user;
- receiving a video stream of the user interacting with the portion of the application, the video stream comprising a time series of images of the user;
- parsing the video stream to identify one or more visual cues of the user;
- identifying a current portion of the application accessed by the user;
- mapping the identified visual cue of the user to an operation available on the current portion of the application;
- and
- executing the mapped operation.
A method, comprising:
- requesting, by a client device, a web page of a website, the request sent via an application executing on the client device;
- receiving, by the client device, the web page associated with the request;
- capturing, by a camera associated with the client device, a video stream of a user while the user is operating the application on the client device;
- parsing, by the client device, the video stream to identify one or more visual cues to identify a command visually communicated by the user;
- mapping, by the client device, the one or more visual cues performed by the user to one or more stored commands;
- identifying, by the client device, an operation that maps to an operation available on a current page of the website by identifying a uniform resource locator associated with the request for the web page;
- and
- updating, by the client device, an interface of the client device to prompt the user with a text-based description of the mapped operation.
Dependencies are inferred from the claim text.