# US 2014/0208272 A1 · User-controlled 3d simulation for providing realistic and enhanced digital object viewing and interaction experience

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0208272 A1
- Document ID: US20140208272A1
- Status: Published application
- Kind: A1
- Application number: 13/946,364
- Filed: 2013-07-19
- Published: 2014-07-24
- Earliest claimed: 2012-07-19
- Inventors: NITIN VATS; GAURAV VATS
- Web page: https://patentgrep.com/patent/US20140208272A1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2219/2024: Style variation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Method, technology and system of user-controlled realistic 3D simulation and interaction are disclosed for providing realistic and enhanced digital object viewing and interaction experience with improved three dimensional (3D) visualization effects. A solution is provided to make available 3D-model/s carrying similar properties of real object, where performing user-controlled realistic interactions selected from extrusive interaction, intrusive interactions, time-bound changes based interaction and real environment mapping based interactions are made possible as per user choice.

## Claims

### Claim 1

A method of user-controlled realistic 3D simulation for enhanced object viewing and interaction experience, the method comprising:
receiving request by at least one input mode for display of an object (1101);
displaying image of the said object or object containing consolidated-view category (1102);
receiving second request by at least one input mode for display of 3D-model of the said object (1103);
loading and simulating 3D-model of the said object in real-time, wherein a virtual operating sub-system is optionally installed in the loaded 3D-model based on characteristics, state and nature of the said object, and where loading and simulating 3D-model of the said object comprises:
i. using image associated data of said object, and auto-linking with real object associated data, polygon data and texturing data of the said object in a simulative manner;
ii. transforming the linked polygon data, texturing data, image associated data and real object associated data into 3D-model of the said object (1104);
displaying 3D-model of the said object in 3D-computer graphic environment, where the displayed 3D-model of the said object comprises at least one realistic 3D-view (1105);
making available user-controlled realistic interactions with the displayed 3D-model to an user, where the user-controlled realistic interactions include extrusive interaction (FIG. 4) and/or intrusive interactions (1e-1h,3b-3f,5b-5e,6a-6c,9a-9c,10a-10c,FIG. 11,FIG. 13,FIG. 14) and/or time bound changes based interaction (FIG. 7), and/or real environment mapping based interactions (FIG. 15,FIG. 16) as per user choice and as per characteristics, state and nature of the said object (1106);
performing user-controlled realistic interactions with the displayed 3D-model and simultaneous displaying in at least one realistic 3D-view with at least one input (1107).

### Claim 2 (depends on 1)

The method as in claim 1, wherein loading of 3D model of the said object is routed either directly, or through a consolidated-view category (FIG. 17,FIG. 18,FIG. 19) or via live telecast category (FIG. 20) as per user choice or in pre-determined manner, where the consolidated-view category emulates real showroom view containing real products, and is selected from an interactive video view category or an interactive panoramic view category, where during interacting in panoramic view, a virtual assistant (1901,1901′) remains intact in same position over the panoramic view while panoramic image or panoramic model moves in interactive and synchronized manner, and where the virtual assistant while remaining intact is capable of moving mouth, carrying out hand movements and portraying facial expressions while replying to user query in interactive manner, and where during loading of the 3D-model, texturing pattern and/or polygons of 3D-model is either dynamically customized or entire 3D-model is loaded without texturing pattern customization.

### Claim 3 (depends on 1)

The method as in claim 1, wherein realistic 3D-view is first realistic 3D-view, a pressure view for judgment of pressure required to operate the said displayed object, a taste view to judge sense of taste, a temperature view for judging heat generated during operation of the said displayed object after certain time intervals, a touch view for judging the sense of softness touch when applied on the displayed object, where the said first realistic 3D-view is preferably displayed initially, and where the pressure view, the taste view, the temperature view, the touch view are available, and displayed on request as per characteristics, state and nature of displayed object, and where properties of heating, cooling, softness, hardness and pressure applied to open or operate a movable sub-part of multi-part 3D-model is represented by texturing the 3D-model in different color or texture pattern, where different pressure, temperature, softness or hardness is distinguished at different sub-parts of said 3D-model or entire 3D-model in different colors or texture patterns.

### Claim 4 (depends on 1)

The method as in claim 1, wherein input mode is selected from placing a search query to search said object; through a pointing device such as mouse; via a keyboard; a gesture guided input of hand or eye movement captured by a sensor of an system; a touch input; a command to a virtual assistant sub-system, where command to the said virtual assistant system can be a voice command or via chat.

### Claim 5 (depends on 1)

The method as in claim 1, wherein extrusive interaction includes rotating 3D-model of object in 360 degree in different planes, lighting effect (1103,1104) for light-emitting parts of 3D-model of object, interacting with 3D-models having electronic display parts for understanding electronic display functioning, sound effects, and displaying said object as per input, with precision, where polygons along with associated texture of said 3D-model moves as per user command, and movement of simulated 3D-model or its parts is achieved and displayed with precision based on user input commands, and where user desiring to operate a section or part of 3d-model provides input on a section or part, which is capable of taking the input as in real set-up for the said object, where the provided input generates same interaction response emulating functioning of said section in real object.

### Claim 6 (depends on 1)

The method as in claim 1, wherein the intrusive interactions includes viewing and interacting with internal parts, opening and closing of sub-parts of said 3d-model, sub-parts change (1101,1102) where the simulated 3D-model is multi-part object, disintegrating parts of the simulated 3D-model one by one to interact with interior and individual parts of the said 3d-model, interacting for exploded view, and where polygons along with associated texture of said 3D-model moves as per user command, and movement of 3D-model or its parts is achieved and displayed with precision based on user input commands as per characteristics, state and nature of displayed object.

### Claim 7 (depends on 1)

The method as in claim 1, wherein the time bound changes based interactions comprises monitoring or visualizing time-bound changes observed on using or operating the said 3D-model of object, where object behavior can be ascertained after a desired duration.

### Claim 8 (depends on 1)

The method as in claim 1, wherein real environment mapping based interactions comprises interactions in which area in vicinity of user is captured, mapped and simulated in real-time such that simulated 3D-model or virtual object displayed on electronic screen of user can be interacted with the mapped and simulated environment, where environment mapping based interactions also include mirror effect of producing reflection of user body or body part in mirror part of simulated 3D-model.

### Claim 9

A system of user-controlled realistic 3D simulation for enhanced object viewing and interaction experience comprising:
a graphical user interface (GUI) (2302) connected to a central search component (2301) configured for accepting user inputs;
a consolidated view displayer (2303) for displaying 3D graphics environment, containing one or more 3D-models in an organized manner using a 3D consolidated view generating engine (2306);
a 3D-model displayer (2305) for displaying 3D-model of an object simulated using a 3D-model generating engine (2307), where the 3D-model displayer comprises at least one display space for displaying the virtual interactive 3D-model;
a virtual operating sub-system (2308) for providing functionality of operation of displayed 3D-model, where the virtual operating sub-system is installed during loading of said 3D-model as per characteristics, state and nature of displayed object, where the virtual operating sub-system is in direct connection to the 3D-model displayer and the 3D objects generating engine;
optionally a virtual assistant sub-system (2304) as one input mode for two way communication, where the virtual assistant sub-system comprises another graphical user interface, a natural language processing component for processing of user input in form of words or sentences and providing output as per the received input, where the natural language processing component is integrated to a central database (2309);
optionally a live telecast displayer (2312) for displaying live telecast of a place containing plurality of objects, where a dynamic link is built over each identified object, where each dynamic link invokes the 3D-model displayer for displaying 3D-model of the said identified object;
optionally a camera (2311) for capturing video for background mapping based interaction, where the video captured from the camera is layered beneath the 3D-model displayer;
where the GUI is direct connection with the consolidated view displayer, the virtual assistant sub-system, the 3D-model displayer, and the central database in addition to the central search component, and where the 3D-model displayer and the 3D objects generating engine are in direct connection to each other, and are also connected to the virtual operating sub-system, and where the central database is also connected to a central data storage (2310) for storing at least image associated data.

### Claim 10 (depends on 9)

The system as in claim 9, wherein the 3D-model displayer is an interactive platform for carrying out extrusive interaction and/or intrusive interactions and/or time bound changes based interaction and/or real environment mapping based interactions as per user choice and as per characteristics, state and nature of the said object.

### Claim 11 (depends on 9)

The system as in claim 9, wherein the 3D objects generating engine uses image associated data, real object associated data, polygon data and texturing data of the said object for simulating said 3D-model, where the 3D-model comprises plurality of polygons.

### Claim 12 (depends on 9)

The system as in claim 9, wherein enhanced object viewing and interaction experience can be provided over a web-page via hypertext transfer protocol in a wearable or non-wearable display, or as offline content in stand-alone system or as content in system connected to network.

### Claim 13 (depends on 9)

The system as in claim 9, wherein virtual assistant sub-system further includes a microphone for receiving voice command, and a sound output device, where interaction with the said virtual assistant system is two-way communication emulating a user interacting with real person to gain object related information and receiving precise replies, either in the form of text output or sound output, as per the query asked.

### Claim 14 (depends on 9)

The system as in claim 9, wherein the link built over each identified object in the live telecast displayer is a dynamic link built in real time during live video telecast of a remote place or a link built with a lag time.

### Claim 15 (depends on 9)

The system as in claim 9, wherein real object associated data, polygon data and texturing data of the said object is stored in the central database or partially in the central data storage.