Visual search using multiple visual input modalities

One or more computer-readable storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform a method comprising:

• receiving a search query;
• providing a plurality of image results, each of the plurality of image results having one or more associated shape-descriptors, including a first edgel-based representation thereof;
• receiving a visual query refinement;
• converting the visual query refinement into at least one shape-descriptor, including a second edgel-based representation;
• and
• comparing the one or more shape-descriptors, including the first edgel-based representations of the plurality of images results with the at least one shape-descriptor, including the second edgel-based representation of the visual query refinement to identify at least one refinement-matching image of the plurality of image results.

A method comprising:

• accessing a plurality of images;
• segmenting each of the plurality of images into a respective plurality of segments;
• performing a multi-phase contour detection on each respective plurality of segments to create one or more shape-descriptors, including a first edgel-based representation, of each of the plurality of images;
• and
• generating, by a processor, an edgel index comprising image data that describes the respective one or more shape-descriptors, including the first edgel-based representations of each of the plurality of images.