Annotation of 3d models with signs of use visible in 2d images

A method performed by data processing apparatus, the method comprising:

• estimating a relative pose of an instance of an object in a relatively smaller field of view image based on matched signs of use in a relatively larger field of view image and the relatively smaller field of view images;
• identifying, using the relative pose, matched signs of use that satisfy a threshold likelihood that the matched signs of use are correct matches;
• and
• projecting the matched signs of use that satisfy the threshold likelihood onto a 3D model of the object.

At least one computer-readable storage medium encoded with executable instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

• estimating a relative pose of an instance of an object in a relatively smaller field of view image based on matched signs of use in a relatively larger field of view image and the relatively smaller field of view images;
• identifying, using the relative pose, matched signs of use that satisfy a threshold likelihood that the matched signs of use are correct matches;
• and
• projecting the matched signs of use that satisfy the threshold likelihood onto a 3D model of the object.

A method performed by a data processing apparatus, the method comprising:

• generating a 3D model of an object;
• projecting first signs of use onto the 3D model of the object, wherein the first signs of use comprise signs of use that are visible in a relatively larger field of view image of an instance of the object and a relatively smaller field of view image of the instance of object;
• and
• eliminating improperly projected signs of use from the projections onto the 3D model of the object, wherein the improperly projected signs of use comprise signs of use with actual positions in the relatively smaller field of view image of the instance of object that differ from hypothetical positions of the signs of use in the relatively smaller field of view image of the instance of object, the hypothetical positions computed using an estimated pose of the instance of the object in the relatively smaller field of view image of the instance of object.