An arrangement for facilitating viewing and real-time user-controlled interactions with a 3D computer graphics model representing a physical object, and for facilitating real-time two-way voice communication between a human user and a virtual product assistant for receiving product information, the arrangement comprising:
- a first soft-copy display device for displaying at least one 3D model of real object;
- one or more processors;
- a camera for capturing video or image input;
- instructions stored in a non-transitory computer readable storage medium executable by the one or more processors that upon such execution cause the one or more processors to perform operations comprising:
- (a) providing a first view of at least one 3D model by a Graphical User Interface (GUI) with one or more user-controlled interactions, using one or more processors, where the GUI comprises one or more interactive 3D space adapted to display one or more 3D models, where a first message related to the presented first view is simultaneously passed to a virtual product assistant unit of the virtual product assistant sub-system;
- (b) receiving a user input, the user input are one or more interaction commands provided using one or more input device that is integrated with the electronic panel system, where one or more interaction commands are provided for performing each user-controlled interaction in real-time, and where an updated message related to provided input is passed to the virtual product assistant unit on providing each interaction command by the user;
- (c) identifying one or more interaction commands, using one or more processors, by user-controlled interaction unit;
- (d) in response to the identification, invoking a corresponding user-controlled interaction functionality of the user-controlled interaction unit, using one or more processors;
- (e) in response to the invoking, producing real-time rendering of user-controlled interaction response using 3D model data, and texture data and a 3D rendering engine, using one or more processors, by the user-controlled interaction unit, where the user-controlled interaction response comprises corresponding interactive view of the 3D model with synchronized sound or without sound as per the input;
- (f) displaying the rendered user-controlled interaction response in one interactive 3D space of the GUI through the soft-copy display device in real-time, using one or more processors, in response to the user input;
- wherein the operations (b)-(f) are repeated to perform each user-controlled interaction, and sending consequent updated messages by the user-controlled interaction unit to the virtual product assistant unit on providing each interaction command by the user;
- wherein at least one user-controlled interaction is performed selected from user-controlled interactions which includes extrusive and intrusive interactions, liquid and fumes flow interactions, time bound changes based interactions, environment mapping based interactions, deletion interaction, addition interaction, immersive interactions, linked movement interactions, interaction for getting un-interrupted view of internal parts using transparency-opacity effect, inter-interactions, and engineering disintegration interactions;
- wherein the displayed rendered user-controlled interaction response is output result displaying effect of each performed user-controlled interaction according to the interaction command provided in the form of input;
- wherein the virtual product assistant sub-system comprises:
- a second soft-copy display device for displaying virtual product assistant;
- one or more processors;
- instructions stored in a non-transitory computer readable storage system executable by the one or more processors that upon such execution cause the one or more processors to perform operations comprising:
- receiving messages of one or more 3D model representing physical object displayed by the GUI on the first soft-copy display device of the electronic panel system, using one or more processors, by a virtual product assistant unit, where the first message is received on providing the first view of the 3D model, and consequent updated messages are received on providing each interaction command by the user;
- activating corresponding product information data set in a database on receiving the first message, and continually updating the product information data set in accordance to the current 3D model display state on the GUI of the electronic panel system in response to received messages;
- receiving a user input, the input is in the form of at least one natural language speech provided using a hardware voice input device that is in communication with the virtual product assistant sub-system, where the user voice input is either a product information query for gaining product information in real-time or an introduction related speech for introduction and salutation;
- processing voice-based input to retrieve relevant information as per received product information query or introduction related speech;
- outputting reply in the form of natural language speech with lip-synchronization of spoken words in accordance to the current 3D model display state displayed on the electronic panel system, wherein the lip-synchronization occurs dynamically in image, video of displayed virtual product assistant, using one or more processors, by an image processor;
- and
- wherein the arrangement is arranged to provide real-time information related to displayed 3D model of real object based on user voice input using the voice input device.