# US 2021/0241513 A1 · Activity Surface Detection, Display and Enhancement of a Virtual Scene

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241513 A1
- Document ID: US20210241513A1
- Status: Published application
- Kind: A1
- Application number: 17/234,798
- Filed: 2021-04-20
- Published: 2021-08-05
- Earliest claimed: 2012-10-15
- Assignees: Tangible Play, Inc.
- Applicants: Tangible Play, Inc.
- Inventors: Pramod Kumar Sharma; Jerome Scholler; Conrad Verkler
- Web page: https://patentgrep.com/patent/US20210241513A1

## CPC

- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/42: by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- G06F 3/005: {Input arrangements through a video camera}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0425: {using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386)}
- G06T 7/254: involving subtraction of images
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G09B 19/0038: {Sports}
- H04N 7/183: {for receiving images from a single remote source}
- G06T 2207/10024: Color image
- G06T 2207/20116: Active contour; Active surface; Snakes
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Activity scene detection, display, and enhancement implementations are described. In an example implementation, a method includes displaying an animated character on a display of a computing device, detecting a tangible interface object on a physical activity scene proximate to the computing device, rendering a virtual interface object based on the tangible interface object, determining an interaction routine between the animated character and the virtual interface object, and executing the animation routine to animate on the display, an interaction between the animated character and the virtual interface object.

## Claims

### Claim 1

A method for monitoring user activity in a physical activity scene, the method comprising:
displaying, on a display of a computing device, a graphical user interface embodying a virtual scene and including an animated character;
capturing, using a video capture device coupled to the computing device, a video stream of a physical activity scene proximate to the computing device, the video stream including a plurality of sets of points;
determining, using a processor of the computing device, a first set of points and a second set of points from the plurality of sets of points;
comparing, using the processor of the computing device, the first set of points and the second set of points based on a criteria;
determining, using the processor of the computing device, a state of a work in the physical activity scene based on the comparison;
determining, using the processor of the computing device, an interaction routine executable to animate an interaction between the animated character and a visualization of the work in the virtual scene in response to determining that the state of the work is complete; and
executing, using the processor of the computing device, the interaction routine to animate, on the display of the computing device, the interaction between the animated character and the visualization.