# US 11,310,462 B2 · User engagement computer system and method of using same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,310,462 B2 - Document ID: US11310462B2 - Status: Granted - Kind: B2 - Application number: 16/948,552 - Filed: 2020-09-23 - Published: 2021-01-14 - Granted: 2022-04-19 - Earliest claimed: 2018-02-02 - Assignees: Mado Labs, Inc. - Applicants: Mado Labs, Inc. - Inventors: William Rocklin; Rembert Reynolds; Rei Inamoto; Tucker Charles Schoos; Nathalie Gina Torres; Robert Cecil Dickerson; William Turnage; Fumiaki Yoshimatsu; Chelsea Garber - Web page: https://patentgrep.com/patent/US11310462B2 ## CPC - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06Q 10/40: Business processes related to social networking or social networking services - G06Q 30/0201: Market modelling; Market analysis; Collecting market data - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - H04L 65/1069: Session establishment or de-establishment - H04L 65/1089: by adding media; by removing media - H04L 65/4015: {where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference} - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A user engagement computer system for enhancing user engagement via video chats and methods of using same are disclosed. The system includes one or more modules that provide enhanced video chat features such as one of the following: detecting user presence at the user devices, automating the establishment and/or scheduling of video chat sessions based on user presence, controlling the available functionality of the video chat application during a video chat session based on user presence, providing a kids mode with safeguards for children who video chat, providing visualization of storytelling, providing shared digital interactions via a shared interactive digital screen overlaid onto the video chat display, providing user controls via unspoken words, motions and gestures, actively measuring user engagement during a video chat session, and utilizing state machines to dynamically change between system states. ## Claims ### Claim 1 A computer-implemented method for interactive engagement by two or more user devices, each associated with a different user, with a user engagement computer system that is in communication with the two or more user devices, wherein the two or more user devices each has a respective audio sensor, a respective video sensor, and a respective motion sensor, the method comprising: (a) detecting, by the user engagement computer system during a video chat session comprising substantially simultaneous presentation of audio and video between the two or more user devices using outputs from at least one of the respective audio sensors, the respective video sensors, and the respective motion sensors, a gestural input by one or more of the respective users of the two or more user devices; (b) in response to detecting the gestural input, determining, during the video chat session by the user engagement system, that the detected gestural input matches gestural content stored in a content database, wherein the match is based on a confidence score that reflects how closely the detected gestural input matches the stored gestural content; and (c) transmitting, by the user engagement system to the one or more of the respective users of the two or more user devices, a command corresponding to the matched stored gestural content, wherein the command initiates an automated action to further engage at least one of the respective users. ### Claim 2 (depends on 1) The method of claim 1, wherein the gestural input comprises a facial gestural input and the at least one of the respective audio sensors, the respective video sensors, and the respective motion sensors comprises the respective motion sensors. ### Claim 3 (depends on 1) The method of claim 1, wherein the gestural input comprises blowing of air by a respective user of one of the two or more user devices into a microphone at the respective user device and the at least one of the respective audio sensors, the respective video sensors and the respective motion sensors comprises the respective audio sensors. ### Claim 4 (depends on 1) The method of claim 1, wherein the automated action that is performed during the video chat session comprises activating a shared interactive digital screen that is displayable as overlaid onto the video of the video chat session between the two or more user devices.