# US 2023/0410348 A1 · Object Detection Outside a User Field-of-View > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0410348 A1 - Document ID: US20230410348A1 - Status: Published application - Kind: A1 - Application number: 18/211,513 - Filed: 2023-06-19 - Published: 2023-12-21 - Earliest claimed: 2022-06-21 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Brett D. Miller; Daniel K. Boothe; Martin E. Johnson - Web page: https://patentgrep.com/patent/US20230410348A1 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)} - H04S 7/304: {For headphones} - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - H04S: STEREOPHONIC SYSTEMS ## Abstract In one implementation, a method of playing an audio notification is performed at a device including one or more image sensors, one or more speakers, one or more processors, and non-transitory memory. The method includes receiving, from the one or more image sensors, an image of a physical environment having a device field-of-view different than a user field-of-view. The method includes detecting, in the image of the physical environment, an object at a location in the physical environment. The method includes determining that the location in the physical environment is outside an area of the user field-of-view. The method includes, in response to determining that the location in the physical environment is outside the area of the user field-of-view, playing, via the speaker, an audio notification of the detection. ## Claims ### Claim 1 A method comprising: at a device including one or more image sensors, one or more speakers, one or more processors, and non-transitory memory: receiving, from the one or more image sensors, an image of a physical environment having a device field-of-view different than a user field-of-view; detecting, in the image of the physical environment, an object at a location in the physical environment; determining that the location in the physical environment is outside an area of the user field-of-view; and in response to determining that the location in the physical environment is outside the area of the user field-of-view, playing, via the one or more speakers, an audio notification of the detection. ### Claim 2 (depends on 1) The method of claim 1, further comprising, in response to determining that the location is within the area of the user field-of-view, forgoing playing the audio notification. ### Claim 3 (depends on 1) The method of claim 1, wherein detecting the object at the location in the physical environment includes transmitting, to a peripheral device, the image of the physical environment, and receiving, from the peripheral device, an indication of the detection. ### Claim 4 (depends on 1) The method of claim 1, wherein determining that the location in the physical environment is outside the area of the user field-of-view includes determining that the location in the physical environment is outside the user field-of-view. ### Claim 5 (depends on 1) The method of claim 1, wherein determining that the location in the physical environment is outside the area of the user field-of-view includes determining that the location in the physical environment is outside a portion of the user field-of-view. ### Claim 6 (depends on 1) The method of claim 1, wherein determining that the location in the physical environment is outside the area of the user field-of-view includes estimating the area of the user field-of-view. ### Claim 7 (depends on 6) The method of claim 6, wherein estimating the area of the user field-of-view includes determining an area around a gaze location of the user. ### Claim 8 (depends on 1) The method of claim 1, wherein playing the audio notification includes playing the audio notification spatially from the location. ### Claim 9 (depends on 8) The method of claim 8, further comprising: receiving, from the one or more image sensors, a second image of the physical environment having a device field-of-view different than the user field-of-view; detecting, in the second image of the physical environment, the object at a second location in the physical environment; and playing, via the one or more speakers, a second audio notification of the detection spatially from the second location. ### Claim 10 (depends on 8) The method of claim 8, further comprising: receiving, from the one or more image sensors, a second image of the physical environment having a device field-of-view different than the user field-of-view; detecting, in the second image of the physical environment, the object at a second location in the physical environment within the area of the user field-of-view; and forgoing playing, via the one or more speakers, a second audio notification of the detection. ### Claim 11 (depends on 1) The method of claim 1, further comprising: detecting, in the image of the physical environment, a second object at a second location in the physical environment; and playing, via the one or more speakers, a second audio notification of the detection of the second object spatially from the second location. ### Claim 12 (depends on 1) The method of claim 1, wherein detecting the object at the location includes determining an object type of the object and wherein playing the audio notification is based on the object type. ### Claim 13 (depends on 1) The method of claim 1, wherein detecting the object at the location includes determining a motion of the object and wherein playing the audio notification is based on the motion of the object. ### Claim 14 (depends on 1) The method of claim 1, wherein playing the audio notification includes generating an audio signal indicative of the detection and playing, via the one or more speakers, the audio signal. ### Claim 15 (depends on 1) The method of claim 1, wherein playing the audio notification includes altering playback, via the one or more speakers, of an audio stream. ### Claim 16 A device comprising: one or more image sensors; one or more speakers; a non-transitory memory; and one or more processors to: receive, from the one or more image sensors, an image of a physical environment having a device field-of-view different than a user field-of-view; detect, in the image of the physical environment, an object at a location in the physical environment; determine that the location in the physical environment is outside an area of the user field-of-view; and in response to determining that the location in the physical environment is outside the area of the user field-of-view, play, via the one or more speakers, an audio notification of the detection. ### Claim 17 (depends on 16) The device of claim 16, wherein the one or more processors are further to, in response to determining that the location is within the area of the user field-of-view, forgo playing the audio notification. ### Claim 18 (depends on 16) The device of claim 16, wherein the one or more processors are to determine an object type of the object and play the audio notification based on the object type. ### Claim 19 (depends on 16) The device of claim 16, wherein the one or more processors are to determine a motion of the object and play the audio notification based on the motion of the object. ### Claim 20 A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device including one or more image sensors and one or more speakers cause the device to: receive, from the one or more image sensors, an image of a physical environment having a device field-of-view different than a user field-of-view; detect, in the image of the physical environment, an object at a location in the physical environment; determine that the location in the physical environment is outside an area of the user field-of-view; and in response to determining that the location in the physical environment is outside the area of the user field-of-view, play, via the one or more speakers, an audio notification of the detection.