# US 2018/0322800 A1 · Communication and skills training using interactive virtual humans > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0322800 A1 - Document ID: US20180322800A1 - Status: Published application - Kind: A1 - Application number: 15/967,205 - Filed: 2018-04-30 - Published: 2018-11-08 - Earliest claimed: 2009-02-13 - Assignees: University of Florida Research Foundation, Inc.; Augusta University Research Institute, Inc. - Applicants: University of Florida Research Foundation, Incorporated; Augusta University Research Institute, Inc. - Inventors: Benjamin Chak Lum Lok; David Scott Lind; Juan Carlos Cendan; Andrew Brian Raij; Brent H. Rossen; Aaron Andrew Kotranza; Kyle John Johnsen - Web page: https://patentgrep.com/patent/US20180322800A1 ## CPC - G09B 7/00: Electrically-operated teaching apparatus or devices working with questions and answers (mechanically operated G09B3/00; computing arrangements G06F) - G09B 23/28: for medicine - G16Z 99/00: Subject matter not provided for in other main groups of this subclass - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - G16Z: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS, NOT OTHERWISE PROVIDED FOR ## Abstract A system for providing interaction between a virtual human and a user, the system comprising: a tangible interface providing a physical interface between the user and the virtual human, an imaging system directed towards the physical interface to provide images of the user interacting with the tangible interface; a tracking system tracking at least one position or the user; a microphone capturing speech from the user; a simulation system receiving inputs from the tangible interface, the imaging system, the tracking system and the microphone, the simulation system generating output signals corresponding to the virtual human; and a display presenting the output signals to the user. ## Claims ### Claim 1 A system for providing interaction between a virtual human and a user, the system comprising: a tangible interface providing a physical interface between the user and the virtual human, an imaging system directed towards the physical interface to provide images of the user interacting with the tangible interface; a tracking system tracking at least one position or the user; a microphone capturing speech from the user; a simulation system receiving inputs from the tangible interface, the imaging system, the tracking system and the microphone, the simulation system generating output signals corresponding to the virtual human; and a display presenting the output signals to the user. ### Claim 2 (depends on 1) The system of claim 1 wherein: the output signals include video and audio signals. ### Claim 3 (depends on 1) The system of claim 1 wherein: the display is at least one of a head mounted display, data projector and large screen television. ### Claim 4 (depends on 1) The system of claim 1 wherein: the tangible interface includes a plurality of force sensors providing force signals to the simulation system. ### Claim 5 (depends on 1) The system of claim 1 wherein: wherein the tracking system includes cameras for tracking a location of markers worn by the user. ### Claim 6 (depends on 5) The system of claim 5 wherein: the camera is an infrared camera and the markers are infrared markers. ### Claim 7 (depends on 5) The system of claim 5 wherein: the tracking system tracks the user's head. ### Claim 8 (depends on 1) The system of claim 1 wherein: the virtual human is developed using a virtual people factory (VPF) system in communication with an educator system and user system over a network. ### Claim 9 (depends on 8) The system of claim 8 wherein: the educator system interacts with the VPF system to generate an initial script for the virtual human. ### Claim 10 (depends on 9) The system of claim 9 wherein: the VPF system gathers inputs from the user system interacting with the virtual human, the inputs indicating when the virtual human responds incorrectly to a user response. ### Claim 11 (depends on 10) The system of claim 10 wherein: the educator system updates the VPF system in response to the inputs indicating when the virtual human responds incorrectly to the user response. ### Claim 12 (depends on 8) The system of claim 8 wherein: the VPF system generates a response to a user input stimulus based on the stimulus matching a trigger stored in the VPF system, the trigger corresponding to a prior user stimulus. ### Claim 13 (depends on 12) The system of claim 12 wherein: wherein the VPF system ranks triggers based on a keyword relevance-weighting scheme to determine which trigger of a plurality of stored triggers best matches the stimulus, the VPF system outputting the response linked to the trigger having the highest relevance. ### Claim 14 (depends on 1) The system of claim 1 further comprising: an interpersonal scenario visualizer (IPSViz) implementing an after action review (AAR) process for reviewing user interaction with the virtual human. ### Claim 15 (depends on 14) The system of claim 14 wherein: the IPSViz generates a visualization screen depicting the interaction between the user and the virtual person in a virtual environment. ### Claim 16 (depends on 15) The system of claim 15 wherein: the IPSViz generates a transcript screen depicting a text transcript of the interaction between the user and the virtual human, the text transcript including a timeline. ### Claim 17 (depends on 16) The system of claim 16 wherein: the IPSViz generates a raw video screen depicting the interaction between the user and the virtual person in a real world environment. ### Claim 18 (depends on 16) The system of claim 16 wherein: the IPSViz generates a graph of a factor from the interaction between the user and the virtual plotted against time. ### Claim 19 (depends on 18) The system of claim 18 wherein: the factor plotted versus time is the user's body lean. ### Claim 20 (depends on 18) The system of claim 18 wherein: the factor plotted versus time is topic of discussion between the user and the virtual human.