# US 11,307,657 B2 · User detection by a computing device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,657 B2 - Document ID: US11307657B2 - Status: Granted - Kind: B2 - Application number: 16/881,599 - Filed: 2020-05-22 - Published: 2020-09-10 - Granted: 2022-04-19 - Earliest claimed: 2014-06-19 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Avi E. Cieplinski; Jeffrey Traer Bernstein; Julian Missig; May-Li Khoe; Bianca Cheng Costanzo; Myra Mary Haggerty; Duncan Robert Kerr; Bas Ording; Elbert D. Chen - Web page: https://patentgrep.com/patent/US11307657B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/0485: Scrolling or panning - G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display} - H04N 21/4126: {The peripheral being portable, e.g. PDAs or mobile phones} - H04N 21/41265: {having a remote control device for bidirectional communication between the remote control device and client device (unidirectional remote control device H04N21/42204)} - H04N 21/422: Input-only peripherals {, i.e. input devices connected to specially adapted client devices}, e.g. global positioning system [GPS] {(input devices also receiving signals from specially adapted client devices H04N21/4104)} - H04N 21/42201: {biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user (input arrangements for interaction with the human body based on nervous system activity detection G06F3/015)} - H04N 21/42203: {sound input device, e.g. microphone} - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/485: End-user interface for client configuration - H04N 21/4852: {for modifying audio parameters, e.g. switching between mono and stereo} - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract In some embodiments, an electronic device optionally identifies a person's face, and optionally performs an action in accordance with the identification. In some embodiments, an electronic device optionally determines a gaze location in a user interface, and optionally performs an action in accordance with the determination. In some embodiments, an electronic device optionally designates a user as being present at a sound-playback device in accordance with a determination that sound-detection criteria and verification criteria have been satisfied. In some embodiments, an electronic device optionally determines whether a person is further or closer than a threshold distance from a display device, and optionally provides a first or second user interface for display on the display device in accordance with the determination. In some embodiments, an electronic device optionally modifies the playing of media content in accordance with a determination that one or more presence criteria are not satisfied. ## Claims ### Claim 1 A method comprising: at an electronic device with one or more processors and memory: generating a first sound for playback on a sound-playback device; receiving an output from a microphone on a sound-receiving device; determining whether the sound-receiving device is present at the sound-playback device based on a plurality of presence criteria, including: sound-detection criteria that is satisfied when the output from the microphone on the sound-receiving device corresponds to the first sound for playback on the sound-playback device; and verification criteria that is satisfied when a presence of the sound-receiving device at the sound-playback device is verified using a quantity other than the first sound, wherein the verification criteria include a criterion that is satisfied when the sound-receiving device has sensed motion during a most recent time period; and in accordance with a determination that the plurality of presence criteria are satisfied, designating the sound-receiving device as being present at the sound-playback device. ### Claim 2 (depends on 1) The method of claim 1, wherein designating the sound-receiving device as being present at the sound-playback device comprises determining a user associated with the sound-receiving device, and designating the user as being present at the sound-playback device. ### Claim 3 (depends on 1) The method of claim 1, further comprising prior to generating the first sound for playback on the sound-playback device, transmitting a wake command to the sound-receiving device for turning on the microphone on the sound-receiving device. ### Claim 4 (depends on 1) The method of claim 1, wherein the verification criteria include a criterion that is satisfied when a face of a user of the sound-receiving device is recognized at the sound-playback device. ### Claim 5 (depends on 1) The method of claim 1, wherein the verification criteria include a criterion that is satisfied when the sound-receiving device resides on a same wireless network as the sound-playback device. ### Claim 6 (depends on 1) The method of claim 1, further comprising: detecting motion of the sound-receiving device; and in accordance with a determination that the motion exceeds a predefined threshold, re-evaluating the plurality of presence criteria to determine whether the sound-receiving device remains at the sound-playback device. ### Claim 7 (depends on 1) The method of claim 1, wherein: the sound-playback device is configured to transition from a first state to a second state in response to one or more transition criteria being satisfied at the sound-playback device, the method further comprising: in accordance with the determination that the plurality of presence criteria are satisfied, forgoing transitioning from the first state of the sound-playback device to the second state of the sound-playback device in response to the one or more transition criteria being satisfied at the sound-playback device. ### Claim 8 An electronic device, comprising: one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for: generating a first sound for playback on a sound-playback device; receiving an output from a microphone on a sound-receiving device; determining whether the sound-receiving device is present at the sound-playback device based on a plurality of presence criteria, including: sound-detection criteria that is satisfied when the output from the microphone on the sound-receiving device corresponds to the first sound for playback on the sound-playback device; and verification criteria that is satisfied when a presence of the sound-receiving device at the sound-playback device is verified using a quantity other than the first sound, wherein the verification criteria include a criterion that is satisfied when the sound-receiving device has sensed motion during a most recent time period; and in accordance with a determination that the plurality of presence criteria are satisfied, designating the sound-receiving device as being present at the sound-playback device. ### Claim 9 (depends on 8) The electronic device of claim 8, wherein designating the sound-receiving device as being present at the sound-playback device comprises determining a user associated with the sound-receiving device, and designating the user as being present at the sound-playback device. ### Claim 10 (depends on 8) The electronic device of claim 8, wherein the one or more programs further include instructions for, prior to generating the first sound for playback on the sound-playback device, transmitting a wake command to the sound-receiving device for turning on the microphone on the sound-receiving device. ### Claim 11 (depends on 8) The electronic device of claim 8, wherein the verification criteria include a criterion that is satisfied when a face of a user of the sound-receiving device is recognized at the sound-playback device. ### Claim 12 (depends on 8) The electronic device of claim 8, wherein the verification criteria include a criterion that is satisfied when the sound-receiving device resides on a same wireless network as the sound-playback device. ### Claim 13 (depends on 8) The electronic device of claim 8, wherein the one or more programs further include instructions for: detecting motion of the sound-receiving device; and in accordance with a determination that the motion exceeds a predefined threshold, re-evaluating the plurality of presence criteria to determine whether the sound-receiving device remains at the sound-playback device. ### Claim 14 (depends on 8) The electronic device of claim 8, wherein: the sound-playback device is configured to transition from a first state to a second state in response to one or more transition criteria being satisfied at the sound-playback device, the one or more programs further including instructions for: in accordance with the determination that the plurality of presence criteria are satisfied, forgoing transitioning from the first state of the sound-playback device to the second state of the sound-playback device in response to the one or more transition criteria being satisfied at the sound-playback device. ### Claim 15 A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with one or more processors and memory, cause the electronic device to: generate a first sound for playback on a sound-playback device; receive an output from a microphone on a sound-receiving device; determine whether the sound-receiving device is present at the sound-playback device based on a plurality of presence criteria, including: sound-detection criteria that is satisfied when the output from the microphone on the sound-receiving device corresponds to the first sound for playback on the sound-playback device; and verification criteria that is satisfied when a presence of the sound-receiving device at the sound-playback device is verified using a quantity other than the first sound, wherein the verification criteria include a criterion that is satisfied when the sound-receiving device has sensed motion during a most recent time period; and in accordance with a determination that the plurality of presence criteria are satisfied, designate the sound-receiving device as being present at the sound-playback device. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein designating the sound-receiving device as being present at the sound-playback device comprises determining a user associated with the sound-receiving device, and designating the user as being present at the sound-playback device. ### Claim 17 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the one or more programs further cause the electronic device to, prior to generating the first sound for playback on the sound-playback device, transmit a wake command to the sound-receiving device for turning on the microphone on the sound-receiving device. ### Claim 18 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the verification criteria include a criterion that is satisfied when a face of a user of the sound-receiving device is recognized at the sound-playback device. ### Claim 19 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the verification criteria include a criterion that is satisfied when the sound-receiving device resides on a same wireless network as the sound-playback device. ### Claim 20 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the one or more programs further cause the electronic device to: detect motion of the sound-receiving device; and in accordance with a determination that the motion exceeds a predefined threshold, re-evaluate the plurality of presence criteria to determine whether the sound-receiving device remains at the sound-playback device. ### Claim 21 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein: the sound-playback device is configured to transition from a first state to a second state in response to one or more transition criteria being satisfied at the sound-playback device, wherein the one or more programs further cause the electronic device to: in accordance with the determination that the plurality of presence criteria are satisfied, forgo transitioning from the first state of the sound-playback device to the second state of the sound-playback device in response to the one or more transition criteria being satisfied at the sound-playback device.