Back to results
US 12,087,301 B2Granted
Systems, methods, and non-transitory computer readable media for human interface device accessibility
AssigneeCephable Inc.
InventorAlexander Dunn
Life of this patent
Estimating the term…
Abstract
A method, system, and non-transitory computer readable media for converting input from a user into a human interface device (HID) output to cause a corresponding action at an mapped device includes receiving one or more user input from a user at an input device, analyzing the user input selecting a command from a command profile that maps at least one of the received user inputs to one or more mapped tasks, executing the one or more mapped tasks associated with the selected command, and causing one or more corresponding actions at one or more mapped devices associated with the one or more mapped tasks.
Claims19
A method comprising:
- receiving, by a computing device, a video stream, wherein the video stream comprises one or more movement inputs corresponding to at least one of a facial movement or a body movement of a user;
- identifying, by the computing device, a first body part of the user in the video stream;
- assigning, by the computing device, a plurality of baseline landmark points and a plurality of baseline pose positions to the first body part;
- monitoring, by the computing device, the video stream for changes in a position of the first body part based on a change in position of a plurality of landmark points and a plurality of pose positions relative to the plurality of baseline landmark points and the plurality of baseline pose positions;
- determining, by the computing device, that at least one of a change in position of the plurality of landmark points and a change in position of the plurality of pose positions exceeds a threshold value, wherein the threshold value corresponds to a user input of a command mapping the user input to one or more mapped tasks, the one or more mapped tasks comprising one or more remote tasks at a third-party system;
- selecting, by the computing device, the command mapping the user input comprising the threshold value to the one or more mapped tasks;
- executing, by the computing device, the one or more mapped tasks of the selected command, wherein executing the one or more mapped tasks comprises at least in part executing the one or more remote tasks at the third party system, wherein executing the one or more remote tasks at the third-party system comprises:
- identifying, by the computing device, one or more remote task access tokens associated with the selected command;
- communicating, by the computing device, a remote task request and the remote task access token to the third-party system;
- and
- receiving, by the computing device, access to the third-party system;
- and
- causing, by the computing device, one or more corresponding actions at a mapped device associated with the one or more mapped tasks of the selected command.
10 more elements
One or more non-transitory computer-readable media comprise instructions that, when executed by the computing system, cause the computing system to:
- receive a video stream, wherein the video stream comprises one or more movement inputs corresponding to at least one of a facial movement and a body movement of a user;
- identify a first body part of the user in the video stream;
- assign a plurality of baseline landmark points and a plurality of baseline pose positions to the first body part;
- monitor the video stream for changes in a position of the first body part based on a change in position of at least one of a plurality of landmark points and at least one of a plurality of pose positions relative to the plurality of baseline landmark points and the plurality of baseline pose positions;
- determine that at least one of a change in position of at least one landmark of the plurality of landmark points and a change in position of the plurality of pose positions exceeds a threshold value, wherein the threshold value corresponds to a user input of a command mapping the user input to one or more mapped tasks, the one or more mapped tasks comprising one or more remote tasks at a third-party system;
- select the command mapping the user input to the one or more mapped tasks in a command profile comprising the threshold value;
- execute the one or more mapped tasks of the selected command, wherein the instructions that cause the computing system to execute the one or more mapped tasks of the selected command cause the computing system to execute the one or more remote tasks of the one or more mapped tasks at the third party system, wherein executing the one or more remote tasks at the third-party system comprises:
- identifying, by the computing device, one or more remote task access tokens associated with the selected command;
- communicating, by the computing device, a remote task request and the remote task access token to the third-party system;
- and
- receiving, by the computing device, access to the third-party system;
- and
- cause one or more corresponding actions at a mapped device associated with the one or more mapped tasks of the selected command.
10 more elements
A method comprising:
- receiving, by a computing device, a video stream, wherein the video stream comprises one or more movement inputs corresponding to at least one of a facial movement or a body movement of a user;
- identifying, by the computing device, a first body part of the user in the video stream, wherein identifying the first body part of the user in the video feed comprises:
- comparing, by the computing device, the video stream to an identification model comprising one or more pre-defined body parts of a user;
- and
- identifying, by the computing device, a match between at least one of the one or more pre-defined body parts of the user in the identification model and a body part in the video stream;
- assigning, by the computing device, a plurality of baseline landmark points and a plurality of baseline pose positions to the first body part, wherein each baseline landmark position comprises coordinates of the first body party on a two-dimensional plane, and each baseline pose position comprises a pitch, roll, and yaw of the first body part;
- monitoring, by the computing device, the video stream for changes in a position of the first body part based on a change in position of a plurality of landmark points and a plurality of pose positions relative to the plurality of baseline landmark points and the plurality of baseline pose positions;
- determining, by the computing device, that at least one of a change in position of the plurality of landmark points and a change in position of the plurality of pose positions exceeds a threshold value, wherein the threshold value corresponds to a user input of a command mapping the user input to one or more mapped tasks;
- selecting, by the computing device, the command mapping the user input comprising the threshold value to the one or more mapped tasks;
- executing, by the computing device, the one or more mapped tasks of the selected command, wherein executing the one or more mapped tasks of the first command comprises:
- storing, by the computing device, the one or more mapped tasks of the selected command in a HID state and a HID queue, wherein the HID state comprises one or more mapped tasks being executed by the computing device, and wherein the HID queue comprises one or more mapped tasks to be executed by the computing device after the one or more mapped tasks in the HID state are executed;
- storing, by the computing device, one or more HID packets in a HID report after executing a mapped task of the one or more mapped tasks of the selected command;
- and
- communicating, by the computing device, the HID report to the mapped device;
- and
- causing, by the computing device, one or more corresponding actions at a mapped device associated with the one or more mapped tasks of the selected command.
13 more elements
A method comprising:
- receiving, by a computing device, a video stream, wherein the video stream comprises one or more movement inputs corresponding to at least one of a facial movement or a body movement of a user;
- identifying, by the computing device, a first body part of the user in the video stream;
- assigning, by the computing device, a plurality of baseline landmark points and a plurality of baseline pose positions to the first body part;
- monitoring, by the computing device, the video stream for changes in a position of the first body part based on a change in position of a plurality of landmark points and a plurality of pose positions relative to the plurality of baseline landmark points and the plurality of baseline pose positions;
- determining, by the computing device, that at least one of a change in position of the plurality of landmark points and a change in position of the plurality of pose positions exceeds a threshold value, wherein the threshold value corresponds to a user input of a command mapping the user input to one or more mapped tasks;
- selecting, by the computing device, the command mapping the user input comprising the threshold value to the one or more mapped tasks;
- executing, by the computing device, the one or more mapped tasks of the selected command, wherein executing the one or more mapped tasks of the selected command comprises:
- storing, by the computing device, the one or more mapped tasks of the selected command in a HID state and a HID queue, wherein the HID state comprises one or more mapped tasks being executed by the computing device, and wherein the HID queue comprises one or more mapped tasks to be executed by the computing device after the one or more mapped tasks in the HID state are executed;
- storing, by the computing device, one or more HID packets in a HID report after executing a mapped task of the one or more mapped tasks of the selected command;
- and
- communicating, by the computing device, the HID report to the mapped device;
- and
- causing, by the computing device, one or more corresponding actions at a mapped device associated with the one or more mapped tasks of the command.
10 more elements
One or more non-transitory computer-readable media comprise instructions that, when executed by the computing system, cause the computing system to:
- receive a video stream, wherein the video stream comprises one or more movement inputs corresponding to at least one of a facial movement and a body movement of a user;
- identify a first body part of the user in the video stream;
- assign a plurality of baseline landmark points and a plurality of baseline pose positions to the first body part;
- monitor the video stream for changes in a position of the first body part based on a change in position of at least one of a plurality of landmark points and at least one of a plurality of pose positions relative to the plurality of baseline landmark points and the plurality of baseline pose positions;
- determine that at least one of a change in position of at least one landmark of the plurality of landmark points and a change in position of the plurality of pose positions exceeds a threshold value, wherein the threshold value corresponds to a user input of a command mapping the user input to one or more mapped tasks;
- select a first command in a command profile comprising the threshold value;
- execute the one or more mapped tasks of the selected command, wherein the instruction hat cause the computing system to execute the one or more mapped tasks of the selected command cause the computing system to:
- store the one or more mapped tasks of the selected command in a HID state and a HID queue, wherein the HID state comprises one or more mapped tasks being executed by the computing device, and wherein the HID queue comprises one or more mapped tasks to be executed by the computing device after the one or more mapped tasks in the HID state are executed;
- store one or more HID packets in a HID report after executing a mapped task of the one or more mapped tasks of the selected command;
- and
- communicate the HID report to the mapped device;
- and
- cause one or more corresponding actions at a mapped device associated with the one or more mapped tasks of the command.
10 more elements
Dependencies are inferred from the claim text.