Acoustic Eavesdropping Using A Smartphone Camera

A method for acoustic eavesdropping on a speaker using a camera in a mobile device, comprising:

• capturing, by the camera, a series of images of a scene, where a lens of the camera is free to move in relation to an image sensor of the camera;
• designating, by an image processor, one of the images in the series of images as a reference image;
• computing, by the image processor, displacement values of pixels forming images in the series of images, where the displacement values are computed in relation to corresponding pixels in the reference image;
• constructing, by the image processor, a motion signature for the lens of the camera from the computed displacement values;
• and
• classifying, by the image processor, the motion signature using machine learning.

A non-transitory computer-readable medium having computer-executable instructions that, upon execution of the instructions by a processor of a device, cause the device to:

• capture a series of images of a scene by a camera, where a lens of the camera is free to move in relation to an image sensor of the camera;
• extracting data indicative of sound spoken by a speaker, where the sound impinges on the camera and the data is extracted from the series of images;
• and
• identify an attribute of the speaker using the extracted data.