# US 2020/0401372 A1 · Method and apparatus for an interactive user interface > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401372 A1 - Document ID: US20200401372A1 - Status: Published application - Kind: A1 - Application number: 17/013,993 - Filed: 2020-09-08 - Published: 2020-12-24 - Earliest claimed: 2014-07-17 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Miikka Tapani VILERMO; Arto Juhani LEHTINIEMI; Koray OZCAN - Web page: https://patentgrep.com/patent/US20200401372A1 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 3/04817: using icons (graphical or visual programming using iconic symbols G06F8/34) - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - H04N 21/43072: {of multiple content streams on the same device} - G10L 21/0272: Voice signal separating - H04R 1/406: {microphones} - H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)} - H04R 2430/01: Aspects of volume control, not necessarily automatic, in sound systems - H04R 2499/11: Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's - H04S 7/30: {Control circuits for electronic adaptation of the sound field} - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - H04S 2400/13: Aspects of volume control, not necessarily automatic, in stereophonic sound systems - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - H04S: STEREOPHONIC SYSTEMS ## Abstract A method, apparatus and computer program product are provided to facilitate user interaction with, such as modification of, respective audio objects. An example method may include causing a multimedia file to be presented that includes at least two images. The images are configured to provide animation associated with respective audio objects and representative of a direction of the respective audio objects. The method may also include receiving user input in relation to an animation associated with an audio object or the direction of the audio object represented by an animation. The method may further include causing replay of the audio object for which the user input was received to be modified. ## Claims ### Claim 1 A method comprising: causing a multimedia file to be presented, the multimedia file comprising at least two images and at least one audio object, wherein the at least two images identify a direction from which the at least one audio object originates; causing a visual indicator associated with the at least one audio object to be provided, wherein the visual indicator is configured to be controlled; processing the at least one audio object in dependence upon a determined setting of the visual indicator to adjust audibility of the at least one audio object while maintaining the direction from which the at least one audio object originates; forming a modified multimedia file based on the processed at least one audio object; and causing the modified multimedia file to be outputted, the modified multimedia file comprising the at least two images and the processed at least one audio object. ### Claim 2 (depends on 1) A method according to claim 1 wherein causing the visual indicator to be provided comprises causing a visual indicator of volume associated with the at least one audio object to be provided. ### Claim 3 (depends on 1) A method according to claim 1 further comprising receiving user input adjusting the setting of the visual indicator of volume in order to cause the volume associated with the at least one audio object to be correspondingly adjusted. ### Claim 4 (depends on 1) A method according to claim 1 further comprising receiving user input indicating the direction from which the at least one audio object originates, wherein causing the visual indicator to be provided comprises causing the visual indicator associated with the at least one audio object to be provided in response to the user input indicating the direction from which the at least one audio object originates. ### Claim 5 (depends on 4) A method according to claim 4 wherein causing the modified multimedia file to be outputted comprises causing the at least one audio object that originates in the direction indicated by the user input to be emphasized relative to audio objects having other directions. ### Claim 6 (depends on 4) A method according to claim 4 wherein causing the modified multimedia file to be outputted comprises causing the at least one audio object that originates in the direction indicated by the user input to be deemphasized relative to audio objects having other directions. ### Claim 7 (depends on 1) A method according to claim 1 wherein the at least one audio object originates from a location that is not depicted by the at least two images. ### Claim 8 An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processor, cause the apparatus to at least: cause a multimedia file to be presented, the multimedia file comprising at least two images and at least one audio object, wherein the at least two images identify a direction from which the at least one audio object originates; cause a visual indicator associated with the at least one audio object to be provided, wherein the visual indicator is configured to be controlled; process the at least one audio object in dependence upon a determined setting of the visual indicator to adjust audibility of the at least one audio object while maintaining the direction from which the at least one audio object originates; form a modified multimedia file based on the processed at least one audio object; and cause the modified multimedia file to be outputted, the modified multimedia file comprising the at least two images and the processed at least one audio object. ### Claim 9 (depends on 8) An apparatus according to claim 8 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the visual indicator to be provided by causing a visual indicator of volume associated with the at least one audio object to be provided. ### Claim 10 (depends on 9) An apparatus according to claim 9 wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to receive user input adjusting the setting of the visual indicator of volume in order to cause the volume associated with the at least one audio object to be correspondingly adjusted. ### Claim 11 (depends on 8) An apparatus according to claim 8 wherein the at least one memory and the computer program code are further configured to, with the processor, cause the apparatus to receive user input indicating the direction from which the at least one audio object originates, wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the visual indicator to be provided by causing the visual indicator associated with the at least one audio object to be provided in response to the user input indicating the direction from which the at least one audio object originates. ### Claim 12 (depends on 11) An apparatus according to claim 11 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the modified multimedia file to be outputted by causing the at least one audio object that originates in the direction indicated by the user input to be emphasized relative to audio objects having other directions. ### Claim 13 (depends on 11) An apparatus according to claim 11 wherein the at least one memory and the computer program code are configured to, with the processor, cause the apparatus to cause the modified multimedia file to be outputted by causing the at least one audio object that originates in the direction indicated by the user input to be deemphasized relative to audio objects having other directions. ### Claim 14 (depends on 8) An apparatus according to claim 8 wherein the at least one audio object originates from a location that is not depicted by the at least two images. ### Claim 15 A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions, when executed by the processor, configured to: cause a multimedia file to be presented, the multimedia file comprising at least two images and at least one audio object, wherein the at least two images identify a direction from which the at least one audio object originates; cause a visual indicator associated with the at least one audio object to be provided, wherein the visual indicator is configured to be controlled; process the at least one audio object in dependence upon a determined setting of the visual indicator to adjust audibility of the at least one audio object while maintaining the direction from which the at least one audio object originates; form a modified multimedia file based on the processed at least one audio object; and cause the modified multimedia file to be outputted, the modified multimedia file comprising the at least two images and the processed at least one audio object. ### Claim 16 (depends on 15) A computer program product according to claim 15 wherein the program code instructions configured to cause the visual indicator to be provided comprise program code instructions configured to cause a visual indicator of volume associated with the at least one audio object to be provided, and wherein the computer-executable program code instructions further comprise program code instructions configured to receive user input adjusting the setting of the visual indicator of volume. ### Claim 17 (depends on 15) A computer program product according to claim 15 wherein the computer-executable program code instructions further comprise program code instructions configured to receive user input indicating the direction from which the at least one audio object originates, wherein the program code instructions configured to cause the visual indicator to be provided comprise program code instructions configured to cause the visual indicator associated with the at least one audio object to be provided in response to the user input indicating the direction from which the at least one audio object originates. ### Claim 18 (depends on 17) A computer program product according to claim 17 wherein the program code instructions configured to cause the modified multimedia file to be outputted comprise program code instructions configured to cause the at least one audio object that originates in the direction indicated by the user input to be emphasized relative to audio objects having other directions. ### Claim 19 (depends on 17) A computer program product according to claim 17 wherein the program code instructions configured to cause the modified multimedia file to be outputted comprise program code instructions configured to cause the at least one audio object that originates in the direction indicated by the user input to be deemphasized relative to audio objects having other directions. ### Claim 20 (depends on 15) A computer program product according to claim 15 wherein the at least one audio object originates from a location that is not depicted by the at least two images.