System and method of image augmentation

A non-transitory computer program product comprising computer readable instructions that when implemented by a computer cause it to perform a method comprising the steps of:

• generating a disparity map from the pair of images of the first stereoscopic image, the disparity map being indicative of distances in the first stereoscopic image;
• generating a virtual three-dimensional model comprising a mesh based upon the distances indicated by the disparity map;
• modelling an interaction of a virtual object with the three dimensional model, in which the virtual object has a physical attribute associated with it, and the interaction of the virtual object with the three dimensional model utilises the physical attribute;
• and
• outputting, for display, an image corresponding to the first stereoscopic image that comprises a visible effect of the interaction of the virtual object with the three dimensional model;
• in which the step of generating the virtual three-dimensional model comprises in turn the steps of:
• defining a series of value ranges corresponding to disparity values of the disparity map, each value range in the series having an end point corresponding to a greater disparity than an end point of preceding value ranges in the series;
• selecting points in the disparity map falling within the respective value range;
• generating a respective mesh responsive to those selected points;
• and
• merging the resulting series of generated meshes to form the 3D model of the scene;
• wherein:
• each respective mesh is within a depth range;
• and
• merging the resulting series of generated meshes to form the 3D model of the scene is done across a depth spanned by the depth ranges of the respective meshes wherein the merged 3D model of the scene is made up of a continuous geometry.

An entertainment device for augmenting a first stereoscopic image comprising a pair of images, the entertainment device comprising:

• input means for receiving the first stereoscopic image data;
• disparity processing means for generating a disparity map from the pair of images of the first stereoscopic image, the disparity map being indicative of distances in the first stereoscopic image;
• virtual modelling means for generating a virtual three-dimensional model comprising a mesh based upon the distances indicated by the disparity map;
• interaction modelling means for modeling an interaction of a virtual object with the three dimensional model, in which the virtual object has a physical attribute associated with it, and the interaction of the virtual object with the three dimensional model is utilises to the physical attribute;
• and
• output means operable for outputting, for display, an image corresponding to the first stereoscopic image that comprises a visible effect of the interaction of the virtual object with the three dimensional model;
• in which the virtual modelling means is configured to generate the virtual three-dimensional model by:
• defining a series of value ranges corresponding to disparity values of the disparity map, each value range in the series having an end point corresponding to a greater disparity than an end point of preceding value ranges in the series;
• selecting points in the disparity map falling within the respective value range;
• generating a respective mesh responsive to those selected points;
• and
• merging the resulting series of generated meshes to form the 3D model of the scene;
• wherein:
• each respective mesh is within a depth range;
• and
• merging the resulting series of generated meshes to form the 3D model of the scene is done across a depth spanned by the depth ranges of the respective meshes wherein the merged 3D model of the scene is made up of a continuous geometry.