# US 11,030,292 B2 · Authentication using sound based monitor detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,292 B2 - Document ID: US11030292B2 - Status: Granted - Kind: B2 - Application number: 16/216,811 - Filed: 2018-12-11 - Published: 2020-06-11 - Granted: 2021-06-08 - Earliest claimed: 2018-12-11 - Assignees: Advanced New Technologies Co., Ltd. - Applicants: Advanced New Technologies Co., Ltd. - Inventors: Gregory Lee Storm; Reza R. Derakhshani - Web page: https://patentgrep.com/patent/US11030292B2 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 21/32: using biometric data, e.g. fingerprints, iris scans or voiceprints - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 21/31: User authentication - G06F 21/44: Program or device authentication - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 5/04: Inference or reasoning models - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Techniques are described for authentication using sound-based monitor detection. In some implementations, the sound (e.g., ultrasound) generated by a monitor is used to detect its presence in the vicinity of a (e.g., portable) computing device that is being used to capture image(s) to be provided to an image-based authentication system. While the computing device is being used to capture image(s) to be provided to the image-based authentication system, the sound data in proximity to the computing device can be captured and analyzed (e.g., using machine learning or other techniques) to determine whether the sound data exhibits a particular pattern and/or characteristics (e.g., signature) that indicates the presence of a monitor in proximity to the computing device. The presence (or absence) of a monitor can be used in determining whether to authenticate the user. ## Claims ### Claim 1 A computer-implemented method performed by at least one processor, the method comprising: obtaining, by the at least one processor, an image of a user of a computing device as part of an image-based authentication process of the computing device; accessing, by the at least one processor, sound data that is generated by at least one microphone of the computing device, the sound data describing sound that is detected by the at least one microphone; generating, by the at least one processor, based on frequency characteristics of the detected sound indicating presence of sound emitted by a display monitor, prediction information that describes a probability that the display monitor is operating in proximity to the computing device during the image-based authentication process; determining, by the at least one processor, based on the prediction information, that the image of the user is a spoofed image displayed by the display monitor; and determining, by the at least one processor, based on determining that the image of the user is a spoofed image, that the image-based authentication process is to be aborted. ### Claim 2 (depends on 1) The method of claim 1, wherein the sound data is in an ultrasonic frequency range. ### Claim 3 (depends on 1) The method of claim 1, comprising adjusting, by the at least one processor, a sampling rate of the at least one microphone to generate the sound data. ### Claim 4 (depends on 1) The method of claim 1, further comprising: activating, by the at least one processor, the at least one microphone in response to the image-based authentication process. ### Claim 5 (depends on 1) The method of claim 1, wherein the prediction information describes a probability that the display monitor is operating within a threshold distance of the computing device during the image-based authentication process. ### Claim 6 (depends on 1) The method of claim 1, wherein the prediction information includes a binary indication of whether the display monitor is operating in proximity to the computing device. ### Claim 7 (depends on 1) The method of claim 1, wherein analyzing the sound data to generate the prediction information includes providing at least a portion of the sound data as input to at least one classifier that is trained, using at least one machine learning (ML) algorithm, to output the prediction information based on the input of at least the portion of the sound data. ### Claim 8 (depends on 7) The method of claim 7, wherein the at least one classifier includes one or more of a neural network, a recurrent neural network (RNN), a long short-term memory (LSTM) network, or a support vector machine (SVM). ### Claim 9 (depends on 7) The method of claim 7, wherein the at least one classifier includes multiple classifiers that are each trained to predict proximity of a respective type of monitor. ### Claim 10 A system comprising: at least one hardware processor, and a physical memory communicatively coupled to the at least one processor, the memory storing instructions which, when executed by the at least one processor, cause the at least one processor to perform operations comprising: obtaining an image of a user of a computing device as part of an image-based authentication process of the computing device; accessing sound data that is generated by at least one microphone of the computing device, the sound data describing sound that is detected by the at least one microphone; generating, based on frequency characteristics of the detected sound indicating presence of sound emitted by a display monitor, prediction information that describes a probability that the display monitor is operating in proximity to the computing device during the image-based authentication process; determining, based on the prediction information, that the image of the user is a spoofed image displayed by the display monitor; and determining, based on determining that the image of the user is a spoofed image, that the image-based authentication process is to be aborted. ### Claim 11 (depends on 10) The system of claim 10, wherein the sound data is in an ultrasonic frequency range. ### Claim 12 (depends on 10) The system of claim 10, wherein the operations comprise adjusting a sampling rate of the at least one microphone to generate the sound data. ### Claim 13 (depends on 10) The system of claim 10, the operations further comprising: activating, by the at least one processor, the at least one microphone in response to the image-based authentication process. ### Claim 14 (depends on 10) The system of claim 10, wherein the prediction information describes a probability that the display monitor is operating within a threshold distance of the computing device during the image-based authentication process. ### Claim 15 (depends on 10) The system of claim 10, wherein the prediction information includes a binary indication of whether the display monitor is operating in proximity to the computing device. ### Claim 16 (depends on 10) The system of claim 10, wherein analyzing the sound data to generate the prediction information includes providing at least a portion of the sound data as input to at least one classifier that is trained, using at least one machine learning (ML) algorithm, to output the prediction information based on the input of at least the portion of the sound data. ### Claim 17 (depends on 16) The system of claim 16, wherein the at least one classifier includes multiple classifiers that are each trained to predict proximity of a respective type of monitor. ### Claim 18 One or more non-transitory, computer-readable storage media storing instructions which, when executed by at least one processor, cause the at least one processor to perform operations comprising: obtaining an image of a user of a computing device as part of an image-based authentication process of the computing device; accessing sound data that is generated by at least one microphone of the computing device, the sound data describing sound that is detected by the at least one microphone; generating, based on frequency characteristics of the detected sound indicating presence of sound emitted by a display monitor, prediction information that describes a probability that the display monitor is operating in proximity to the computing device during the image-based authentication process; determining, based on the prediction information, that the image of the user is a spoofed image displayed by the display monitor; and determining, based on determining that the image of the user is a spoofed image, that the image-based authentication process is to be aborted.