Grasping virtual objects in augmented reality

A computer-implemented method of calculating a virtual object in a virtual reality environment comprising:

• receiving, at a processor, a plurality of images of at least part of a real object in an interaction volume associated with the virtual reality environment;
• tracking 3D motion and/or deformation of a surface of the real object in the interaction volume using the received images and approximate the surface of the real object by filling the contour of the real object with a plurality of non-intersecting passive particles, wherein each passive particle is linked to one of a plurality of kinematic particles;
• and
• calculating a position of the virtual object in the virtual reality environment according to simulated forces between the virtual object and the real object, the simulated forces being determined at least using the tracked 3D motion and/or deformation, the plurality of passive particles, and the plurality of kinematic particles, wherein the simulated forces are determined such that the passive particles cannot penetrate the virtual object or the other passive particles.

A computer-implemented method of simulating grasping of a virtual object comprising:

• receiving, at a processor, a plurality of images of at least part of a user's hand in an interaction volume having an associated virtual reality environment visible to the user;
• tracking 3D motion and/or deformation of a surface of the user's hand in the interaction volume using the received images and approximate the surface of the user's hand by filling the contour of the user's hand with a plurality of non-intersecting passive particles, wherein each passive particle is linked to one of a plurality of kinematic particles;
• and
• calculating a position of the virtual object in the virtual reality environment according to simulated forces between the virtual object and the user's hand, the simulated forces being determined at least using the tracked 3D motion and/or deformation, the plurality of passive particles, and the plurality of kinematic particles in order that when the user grasps the virtual object corresponding simulated grasping in the virtual reality environment is calculated, wherein the simulated forces are determined such that the passive particles cannot penetrate the virtual object or the other passive particles.