Correlating image annotations with foreground features

A system comprising:

• one or more hardware processors configured to perform operations comprising:
• accessing a reference furniture image of a reference item and a set of item annotations that is an existing set of item annotations applied to the reference furniture image and is descriptive of the reference item depicted in the reference furniture image;
• calculating a plurality of reference feature descriptors from the reference furniture image, wherein the calculating of the plurality of reference feature descriptors comprises:
• defining a first portion of the reference furniture image;
• defining a second portion of the reference furniture image, the first portion and the second portion at least partially overlapping;
• determining the first portion represents a foreground and a third portion of the reference furniture image represents a background, the third portion being outside of the second portion;
• determining whether an overlapping portion represents the background, the overlapping portion extending between the first portion and the third portion;
• determining dense scale-invariant feature transform (SIFT) features within the reference furniture image on a regular grid at three different scales, the determining constrained to the foreground of the reference furniture image, and
• assigning visual words using locality-constrained linear coding sampled from only the foreground of the reference furniture image, wherein the dense SIFT features are determined using a first global level with respect to the reference furniture image and a second level comprised of a plurality of grids;
• correlating the plurality of reference feature descriptors with one or more item annotations in the set of item annotations;
• and
• generating a data structure that comprises the plurality of reference feature descriptors with the correlated one or more item annotations.

A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

• accessing a reference furniture image of a reference item and a set of item annotations that is an existing set of item annotations applied to the reference item and is descriptive of the reference item depicted in the reference furniture image;
• calculating a plurality of reference feature descriptors from the reference furniture image, wherein the calculating of the plurality of reference feature descriptors comprises:
• defining a first portion of the reference furniture image;
• defining a second portion of the reference furniture image, the first portion and the second portion at least partially overlapping;
• determining the first portion represents a foreground and a third portion of the reference furniture image represents a background, the third portion being outside of the second portion;
• and
• determining whether an overlapping portion represents the background, the overlapping portion extending between the first portion and the third portion;
• correlating the plurality of reference feature descriptors with one or more item annotations in the set of item annotations;
• generating a data structure that comprises the plurality of reference feature descriptors with the correlated one or more item annotations;
• receiving a query from a user;
• and
• providing the user with a response to the query based on the generated data structure.