Controlling augmented reality effects through multi-modal human interaction
AssigneeSnap Inc.
InventorsJonathan Solichin, Xinyao Wang
Life of this patent
Estimating the term…
Abstract
Systems and methods herein describe a multi-modal interaction system. The multi-modal interaction system, receives a selection of an augmented reality (AR) experience within an application on a computer device, displays a set of AR objects associated with the AR experience on a graphical user interface (GUI) of the computer device, display textual cues associated with the set of augmented reality objects on the GUI, receives a hand gesture and a voice command, modifies a subset of augmented reality objects of the set of augmented reality objects based on the hand gesture and the voice command, and displays the modified subset of augmented reality objects on the GUI.
Claims20
A method comprising:
- causing display, by at least one processor of a set of augmented reality objects on a graphical user interface (GUI) of a computer device;
- causing display of textual cues associated with the set of augmented reality objects on the GUI;
- in response to the displayed textual cues:
- receiving, from the computer device, a hand gesture;
- generating a first set of modification data based on the hand gesture;
- receiving, from the computer device, a voice command;
- and
- generating a second set of modification data based on keywords in the voice command;
- identifying a subset of augmented reality objects of the set of set of augmented reality objects using the first set of modification data;
- generating a modified subset of augmented reality objects by applying the second set of modification data to the identified subset of augmented reality objects;
- and
- causing display of the modified subset of augmented reality objects on the GUI.
A computing system comprising:
- a processor;
- and
- a memory storing instructions that, when executed by the processor, configure the system to:
- cause display of a set of augmented reality objects on a graphical user interface (GUI) of a computer device;
- cause display of textual cues associated with the set of augmented reality objects on the GUI;
- in response to the displayed textual cues:
- receive, from the computer device, a hand gesture;
- generate a first set of modification data based on the hand gesture;
- receive, from the computer device, a voice command;
- and
- generate a second set of modification data based on keywords in the voice command;
- identify a subset of augmented reality objects of the set of set of augmented reality objects using the first set of modification data;
- generate a modified subset of augmented reality objects by applying the second set of modification data to the identified subset of augmented reality objects;
- and
- cause display of the modified subset of augmented reality objects on the GUI.
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:
- cause display of a set of augmented reality objects on a graphical user interface (GUI) of a computer device;
- cause display of textual cues associated with the set of augmented reality objects on the GUI;
- in response to the displayed textual cues:
- receive, from the computer device, a hand gesture;
- generate a first set of modification data based on the hand gesture;
- receive, from the computer device, a voice command;
- generate a second set of modification data based on keywords in the voice command;
- identify a subset of augmented reality objects of the set of set of augmented reality objects using the first set of modification data;
- generate a modified subset of augmented reality objects by applying the second set of modification data to the identified subset of augmented reality objects;
- and
- cause display of the modified subset of augmented reality objects on the GUI.
Dependencies are inferred from the claim text.