Using voice and visual signatures to identify objects

A system for identifying sound-emanating objects in an environment of a user, the system comprising:

• at least one memory device configured to store a database of reference visual characteristics and reference voiceprints corresponding to a plurality of objects;
• and
• at least one processor programmed to:
• receive a plurality of images captured by a wearable camera, wherein at least one of the plurality of images depicts at least one sound-emanating object in an environment of a user;
• analyze the received at least one of the plurality of images to determine one or more visual characteristics associated with the at least one sound-emanating object;
• identify within the database in view of the one or more visual characteristics, the at least one sound-emanating object and determine a degree of certainty of identification;
• receive audio signals acquired by a wearable microphone, wherein the audio signals are representative of one or more sounds emanating from the at least one sound-emanating object;
• analyze the received audio signals to determine a voiceprint of the at least one sound emanating object;
• when the degree of certainty of identification falls below a predetermined level, further identify the at least one sound-emanating object based on the determined voiceprint;
• and
• initiate at least one action based on an identity of the at least one sound-emanating object.

A method for identifying sound-emanating objects in an environment of a user, the method comprising:

• accessing a database of reference visual signatures and reference voice signatures corresponding to a plurality of objects;
• receiving a plurality of images captured by a wearable camera, wherein at least one of the plurality of images depicts at least one sound-emanating object in an environment of a user;
• analyzing the received at least one of the plurality of images to determine one or more visual characteristics associated with the at least one sound-emanating object;
• identifying, based on review of the database in view of the one or more visual characteristics, the at least one sound-emanating object and determine a degree of certainty of identification;
• receiving audio signals acquired by a wearable microphone, wherein the audio signals are representative of one or more sounds emanating from the at least one sound-emanating object;
• analyzing the received audio signals to determine voiceprint of the at least one sound-emanating object;
• when the degree of certainty of identification falls below a predetermined level, further identifying the at least one sound-emanating object based on the determined voiceprint;
• and
• initiating at least one action based on an identity of the at least one sound-emanating object.

A software product stored on a non-transitory computer readable medium and comprising computer implementable instructions for a method for identifying sound-emanating objects, the method comprising:

• accessing a database of reference visual signatures and reference voice signatures corresponding to a plurality of objects;
• receiving a plurality of images captured by a wearable camera, wherein at least one of the plurality of images depicts at least one sound-emanating object in an environment of a user;
• analyzing the received at least one of the plurality of images to determine one or more visual characteristics associated with the at least one sound-emanating object;
• identifying, based on review of the database in view of the one or more visual characteristics, the at least one sound-emanating object and determine a degree of certainty of identification;
• receiving audio signals acquired by a wearable microphone, wherein the audio signals are representative of one or more sounds emanating from the at least one sound-emanating object;
• analyzing the received audio signals to determine voiceprint of the at least one sound-emanating object;
• when the degree of certainty of identification falls below a predetermined level, further identifying the at least one sound-emanating object based on the determined voiceprint;
• and
• initiating at least one action based on an identity of the at least one sound-emanating object.