# US 11,442,693 B2 · Metadata-free audio-object interactions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,693 B2 - Document ID: US11442693B2 - Status: Granted - Kind: B2 - Application number: 17/350,197 - Filed: 2021-06-17 - Published: 2021-10-07 - Granted: 2022-09-13 - Earliest claimed: 2017-05-05 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Lasse Juhani Laaksonen - Web page: https://patentgrep.com/patent/US11442693B2 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method including, detecting interaction between the user and an audio-object via local tracking, determining an audio-object state modification based on the local tracking, and performing an audio-object interaction based on the audio-object state modification. ## Claims ### Claim 1 A method comprising: determining an amount of received metadata; enabling detection of interaction between a user and an audio-object at an interaction area based, at least partially, on the determined amount of received metadata, wherein the detecting of the interaction between the user and the audio-object is based on the received metadata when the determined amount of the received metadata comprises a first amount, wherein the detecting of the interaction between the user and the audio-object is based on tracking of a position of the user in relation to the audio-object when the determined amount of the received metadata comprises a second amount, where the first amount is different from the second amount; determining a state of the audio-object based, at least partially, on a determination of whether the interaction is detected, wherein the determined state is one of a plurality of audio-object states; and rendering the audio-object based on the determined state of the audio-object. ### Claim 2 (depends on 1) The method of claim 1, wherein one or more of the plurality of audio-object states are associated with the first amount, wherein one of the plurality of audio-object states is associated with the second amount. ### Claim 3 (depends on 1) The method of claim 1, wherein the first amount is not zero, wherein the second amount is zero. ### Claim 4 (depends on 1) The method of claim 1, further comprising: determining an audio-object state modification based on the detected interaction; transitioning between a first interaction state and a second interaction state based on the determined audio-object state modification; and performing an audio-object interaction response based on the transitioning. ### Claim 5 (depends on 1) The method of claim 1, wherein the detecting of the interaction between the user and the audio- object based on tracking of the position of the user in relation to the audio-object further comprises: detecting the user entering a vicinity of the audio-object; initializing the tracking; and performing the tracking and updating a local tracking area. ### Claim 6 (depends on 1) The method of claim 1, further comprising: determining the interaction area based on a center of mass associated with the user and a center of mass associated with the audio-object. ### Claim 7 An apparatus comprising: at least one processor; and at least one non-transitory memory including computer program code; the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform: determine an amount of received metadata; enable detection of interaction between a user and an audio-object at an interaction area based, at least partially, on the determined amount of received metadata, wherein detecting the interaction between the user and the audio-object is based on the received metadata when the determined amount of the received metadata comprises a first amount, wherein detecting the interaction between the user and the audio-object is based on tracking of a position of the user in relation to the audio-object when the determined amount of the received metadata comprises a second amount, where the first amount is different from the second amount; determine a state of the audio-object based, at least partially, on a determination of whether the interaction is detected, wherein the determined state is one of a plurality of audio-object states; and cause rendering of the audio-object based on the determined state of the audio-object. ### Claim 8 (depends on 7) The apparatus of claim 7, wherein one or more of the plurality of audio-object states are associated with the first amount, wherein one of the plurality of audio-object states is associated with the second amount. ### Claim 9 (depends on 7) The apparatus of claim 7, wherein the first amount is not zero, wherein the second amount is zero. ### Claim 10 (depends on 7) The apparatus of claim 7, wherein the at least one memory and the computer program code further configured to, with the at least one processor, cause the apparatus at least to perform: determine an audio-object state modification based on the detected interaction; transition between a first interaction state and a second interaction state based on the determined audio-object state modification; and perform an audio-object interaction response based on the transitioning. ### Claim 11 (depends on 7) The apparatus of claim 7, wherein detecting the interaction between the user and the audio-object based on tracking of the position of the user in relation to the audio-object further comprises the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to: detect the user entering a vicinity of the audio-object; initialize the tracking; and perform the tracking and update a local tracking area. ### Claim 12 (depends on 7) The apparatus of claim 7, wherein the at least one memory and the computer program code further configured to, with the at least one processor, cause the apparatus at least to perform: determine the interaction area based on a center of mass associated with the user and a center of mass associated with the audio-object. ### Claim 13 A non-transitory computer-readable medium comprising program instructions stored thereon which, when executed with at least one processor, cause the at least one processor to: determine an amount of received metadata; enable detection of interaction between a user and an audio-object at an interaction area based, at least partially, on the determined amount of received metadata, wherein detecting the interaction between the user and the audio-object is based on the received metadata when the determined amount of the received metadata comprises a first amount, wherein detecting the interaction between the user and the audio-object is based on tracking of a position of the user in relation to the audio-object when the determined amount of the received metadata comprises a second amount, where the first amount is different from the second amount; determine a state of the audio-object based, at least partially, on a determination of whether the interaction is detected, wherein the determined state is one of a plurality of audio-object states; and cause rendering of the audio-object based on the determined state of the audio-object. ### Claim 14 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein one or more of the plurality of audio-object states are associated with the first amount, wherein one of the plurality of audio-object states is associated with the second amount. ### Claim 15 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the first amount is not zero, wherein the second amount is zero. ### Claim 16 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the program instructions are further configured to, when executed with at least one processor, cause the at least one processor to: determine an audio-object state modification based on the detected interaction; transition between a first interaction state and a second interaction state based on the determined audio-object state modification; and perform an audio-object interaction response based on the transitioning. ### Claim 17 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein detecting the interaction between the user and the audio-object based on tracking of the position of the user in relation to the audio-object further comprises the program instructions are configured to, when executed with at least one processor, cause the at least one processor to: detect the user entering a vicinity of the audio-object; initialize the tracking; and perform the tracking and update a local tracking area. ### Claim 18 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the program instructions are further configured to, when executed with at least one processor, cause the at least one processor to: determine the interaction area based on a center of mass associated with the user and a center of mass associated with the audio-object.