# US 2019/0318659 A1 · Hands-On Laboratory and Demonstration Equipment with a Hybrid Virtual/Augmented Environment, Along With Their Methods of Use > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318659 A1 - Document ID: US20190318659A1 - Status: Published application - Kind: A1 - Application number: 16/384,134 - Filed: 2019-04-15 - Published: 2019-10-17 - Earliest claimed: 2018-04-14 - Assignees: The California State University - San Marcos - Applicants: The California State University - San Marcos - Inventors: Kambiz Hamadani; Ali Ahmadinia; Xin Ye; Yuanyuan Jiang - Web page: https://patentgrep.com/patent/US20190318659A1 ## CPC - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G09B 23/28: for medicine - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G09B 19/00: Teaching not covered by other main groups of this subclass (teaching or practice apparatus for gun-aiming or gun-laying F41G3/26) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems and methods for utilizing laboratory equipment in a hybrid reality environment or augmented virtual reality environment, are contemplated herein and include: at least one piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked; a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment; a virtual model, stored in a memory, comprising at least one 3-D virtual representation of the at least one feature of the at least one piece of laboratory equipment; and an experimentation module. A piece of laboratory equipment for use in a hybrid reality environment or augmented virtual reality environment is also included that comprises at least one piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked; at least one marker that is coupled with the at least one piece of laboratory equipment; and a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment, wherein the tracking module accesses the at least one marker. ## Claims ### Claim 1 A system for utilizing laboratory equipment in a hybrid reality environment, comprising: at least one piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked; a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment; a virtual model, stored in a memory, comprising at least one 3-D virtual representation of the at least one feature of the at least one piece of laboratory equipment; an experimentation module that: accesses the virtual model stored in the memory; receives input from the tracking module regarding the position and the orientation of the at least one piece of tracked laboratory equipment; determines, based on the input from the tracking module, a corresponding interaction between the at least one piece of laboratory equipment and at least one additional piece of laboratory equipment, at least a portion or part of a user, or a combination thereof; determines a consequence of the corresponding interaction; and renders, to a display, a hybrid representation comprising a particular set of the at least one 3-D virtual representation, at least a part or portion of the user, the corresponding interaction, the consequence of the corresponding interaction, or a combination thereof. ### Claim 2 (depends on 1) The system of claim 1, further comprising at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a physical actual model representation, a virtual model representation or a combination thereof. ### Claim 3 (depends on 1) The system of claim 1, further comprising at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof, wherein the at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a physical actual model representation, a virtual model representation or a combination thereof. ### Claim 4 (depends on 2) The system of claim 2, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation. ### Claim 5 (depends on 3) The system of claim 3, wherein the at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation. ### Claim 6 (depends on 1) The system of claim 1, wherein the hybrid reality environment includes an augmented virtual reality environment. ### Claim 7 (depends on 1) The system of claim 1, further comprising at least one marker that is coupled with the at least one piece of laboratory equipment; and wherein the tracking module accesses the at least one marker. ### Claim 8 A system for utilizing laboratory equipment in a hybrid reality environment, comprising: at least one first piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked; a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment; a virtual model, stored in a memory, comprising at least one 3-D virtual representation of at least one feature of an additional piece of laboratory equipment, at least a portion of a part of a user, or a combination thereof; an experimentation module that: accesses the virtual model stored in the memory; receives input from the tracking module regarding the position and the orientation of the at least one first piece of tracked laboratory equipment; determines, based on the input from the tracking system, a corresponding interaction between the at least one piece of laboratory equipment and at least one additional piece of laboratory equipment, at least a portion or part of a user, or a combination thereof; determines a consequence of the corresponding interaction; and renders, to a display, a hybrid representation comprising the physical visual representation of the at least one piece of tracked laboratory equipment, at least one 3-D virtual representation, at least a part or portion of the user, the corresponding interaction, the consequence of the corresponding interaction, or a combination thereof. ### Claim 9 (depends on 8) The system of claim 8, further comprising at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a physical actual model representation, a virtual model representation or a combination thereof. ### Claim 10 (depends on 8) The system of claim 8, further comprising at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof, wherein the at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a physical actual model representation, a virtual model representation or a combination thereof. ### Claim 11 (depends on 9) The system of claim 9, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation. ### Claim 12 (depends on 10) The system of claim 10, wherein the at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation. ### Claim 13 (depends on 8) The system of claim 8, wherein the hybrid reality environment includes an augmented virtual reality environment. ### Claim 14 (depends on 8) The system of claim 8, further comprising at least one marker that is coupled with the at least one piece of laboratory equipment; and wherein the tracking module accesses the at least one marker. ### Claim 15 A piece of laboratory equipment for use in a hybrid reality environment, comprising: at least one piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked; at least one marker that is coupled with the at least one piece of laboratory equipment; and a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment, wherein the tracking module accesses the at least one marker. ### Claim 16 (depends on 15) The piece of laboratory equipment for use in a hybrid reality environment of claim 15, wherein the piece of laboratory equipment comprises at least one laboratory solid material, at least one laboratory liquid material, at least one laboratory gaseous material, or a combination thereof, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation. ### Claim 17 (depends on 15) The piece of laboratory equipment for use in a hybrid reality environment of claim 15, wherein the piece of laboratory equipment comprises a pipetteman, a beaker, a sample tube, a graduated cylinder, a scale, or a combination thereof.