Augmented reality method applied to the integration of a pair of spectacles into an image of a face

A method for creating a real-time photorealistic image of a virtual object, arranged in a realistic orientation related to a position of said user, the method comprising the following steps:

• selecting a texture to be used for the virtual object, in accordance with an angle-of-view, and generating a view of the virtual object in a 3D (φ, ψ)/2D (Θ, s) position in question, the virtual object corresponding to a real object, and the angles φ and ψ respectively correspond to an angle of rotation around a y-axis and an angle of rotation around an x-axis, wherein Θ corresponds to a parameter of rotation in the image plane, and s corresponds to a parameter of scale;
• creating a first rendering by establishing a layered rendering in a correct position consistent with the position of the placement area for the virtual object in the first photo;
• and
• obtaining the photorealistic rendering by adding overlays, referred to as semantic overlays, so as to obtain a final image,
• wherein said step of creating a first rendering comprises generating an oriented textured model, oriented according to the angles φ and ψ and according to a scale and orientation of the first photo, from a textured reference model, oriented according to the reference orientation closest to the angles φ and ψ, and parameters of a similarity β, by the following sub-steps:
• using a bilinear affine interpolation to orient an interpolated textured model according to the angles φ and ψ based on the textured reference model oriented according to the reference orientation closest to the angles φ and ψ, and
• using the similarity β to be applied, so as to obtain a same scale, a same image orientation and a same centering as the first photo, thus producing an oriented textured model.

A method for creating a real-time photorealistic image of a virtual object, arranged in a realistic orientation related to a position of said user, the method comprising the following steps:

• selecting a texture to be used for the virtual object, in accordance with an angle-of-view, and generating a view of the virtual object in a 3D (φ, ψ)/2D (Θ, s) position in question, the virtual object corresponding to a real object, and the angles φ and ψ respectively correspond to an angle of rotation around a y-axis and an angle of rotation around an x-axis, wherein Θ corresponds to a parameter of rotation in the image plane, and s corresponds to a parameter of scale;
• creating a first rendering by establishing a layered rendering in a correct position consistent with the position of the placement area for the virtual object in the first photo;
• and
• obtaining the photorealistic rendering by adding overlays, referred to as semantic overlays, so as to obtain a final image,
• wherein said method comprises in addition a phase of creating a database of models of eyes DBmodels_eyes Comprising a plurality of photographs of faces referred to as learning photographs Appeyesk, and
• wherein said step of creating the database comprises the following sub-steps:
• a step of defining a reference face shape and orientation by setting a reference interpupillary distance di0, by centering an interpupillary segment on a center of the image and orienting the interpupillary segment parallel to the image's horizontal axis,
• then, for each kth learning photograph Appeyesk not yet processed:
• a step of determining a precise position of characteristic points:
• exterior point Blk, Blk, and interior point Alk, Ark of each eye and determining a respective geometric center Glk, Glk of the eyes and the interpupillary distance dik,
• a step of transforming the kth learning photograph Appeyesk into a gray-scale image Appeyes-grayk, and normalizing the gray-scale image by applying a similarity Sk(tx, ty, s, Θ) so as to establish an orientation and scale of the reference face to obtain a kth gray-scale normalized learning photograph Appeyes_gray_normk,
• a step of defining a window of fixed dimensions for each of the two eyes, in the kth gray-scale normalized learning photograph Appeyes_gray_normk:
• left patch Plk and right patch Prk;
• where a position of a patch P is defined by a fixed distance Δ between the exterior point of the eye B and the edge of the patch P le closest to this exterior point of the eye B,
• a step, for each of the two patches Plk, Prk associated to the kth gray-scale normalized learning photograph Appeyes_gray_normk, of normalizing the gray-scales,
• a step, for the first learning photograph Appeyes1, of storing each of the patches Pl1, Pr1 called descriptor patches, in the eyes database DBmodels_eyes,
• a step, for each of the patches P associated to the kth gray-scale normalized learning photograph Appeyes_gray_normk, of correlating a corresponding normalized texture column-vector To with each normalized texture column-vectors T0i of the corresponding descriptor patches, and
• a step of comparing, for each of the patches Plk, Plk, the correlation measurement with a previously defined correlation threshold, and, when the correlation is less than the correlation threshold, of storing the patch P as a descriptor patch in the eyes database DBmodels_eyes.