Methods and apparatus for detecting, filtering, and identifying objects in streaming video

A system comprising:

• a server to:
• implement a neural network trained on a plurality of triplets of training vectors, each triplet in the plurality of triplets comprising a reference data point, a positive data point, and a negative data point, the reference data point representing a first object, the positive data point representing a second object similar to the first object, and the negative data point representing a third object dissimilar to the first object;
• generate a feature embedding of an object appearing in a video with the neural network;
• perform a comparison of the feature embedding to feature embeddings of other objects generated by the neural network;
• and
• return, based on the comparison, a representation of an object similar to the object appearing in the video.

A method of training and using a neural network, the method comprising:

• generating a plurality of triplets of training vectors, each triplet in the plurality of triplets comprising a reference image showing a first product, a positive image showing a second product having extrinsic and intrinsic similarity to the first product, and a negative image showing a third product having at least one of extrinsic dissimilarity or intrinsic dissimilarity to the first product;
• training the neural network on the plurality of triplets;
• receiving, from a person, an inquiry about a product appearing in a video;
• generating, with the neural network, a feature embedding representing the product appearing in the video;
• identifying, based on the feature embedding and feature embeddings generated by the neural network for other products, a product similar the product appearing in the video;
• and
• retrieving, from a database, a representation of the product similar to the product appearing in the video for display to the person.