# US 12,192,738 B2 · Electronic apparatus for audio signal processing and operating method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,192,738 B2
- Document ID: US12192738B2
- Status: Granted
- Kind: B2
- Application number: 17/727,202
- Filed: 2022-04-22
- Published: 2022-10-27
- Granted: 2025-01-07
- Earliest claimed: 2021-04-23
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Anant Baijal; Cheulhee Hahm; Jayoon Koo
- Web page: https://patentgrep.com/patent/US12192738B2

## CPC

- H04S 7/303: {Tracking of listener position or orientation}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
- G10L 25/57: for processing of video signals
- H04N 21/81: Monomedia components thereof
- H04S 7/00: Indicating arrangements; Control arrangements, e.g. balance control
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30196: Human being; Person
- G06T 2207/30242: Counting objects in image
- H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)}
- H04R 2499/15: Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04S 2420/07: Synergistic effects of band splitting and sub-band processing
- H04S: STEREOPHONIC SYSTEMS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

A method of an electronic apparatus for audio signal processing includes obtaining a parameter related to spatialization of an audio object, obtaining rendering information based on the parameter related to spatialization, and rendering the audio object based on the rendering information. The parameter related to spatialization includes at least one of an object parameter of a feature of at least one of the audio object or a video object associated with the audio object, an electronic apparatus parameter of a feature of the electronic apparatus, or a user parameter of a feature of a user.

## Claims

### Claim 1

A method of an electronic apparatus for audio signal processing, the method comprising:
obtaining a parameter related to spatialization of an audio object;
obtaining rendering information based on the parameter related to spatialization; and
rendering the audio object based on the rendering information,
wherein the parameter related to spatialization comprises an object parameter of a feature of a video object associated with the audio object,
wherein the parameter related to spatialization further comprises a plurality of parameters, each parameter of the plurality of parameters having a priority, and
wherein the obtaining of the rendering information comprises obtaining rendering information corresponding to a parameter of the plurality of parameters in an order of a parameter having a higher priority.

### Claim 2 (depends on 1)

The method of claim 1, wherein the obtaining of the rendering information comprises:
obtaining a lookup table; and
retrieving the rendering information from the obtained lookup table.

### Claim 3 (depends on 2)

The method of claim 2, further comprising, based on the rendering information not being included in the lookup table, estimating the rendering information based on the lookup table using at least one of a statistical model, a machine learning model, or a regression model.

### Claim 4 (depends on 2)

The method of claim 2,
wherein the obtaining of the rendering information further comprises:
based on a parameter value with Nth priority, wherein N comprises a natural number, being greater than or equal to a reference value, sequentially considering a parameter value with (N+1) th priority as a next priority, and
obtaining the rendering information from the lookup table based on a first combination of parameters greater than or equal to the reference value.

### Claim 5 (depends on 4)

The method of claim 4, wherein the obtaining of the rendering information further comprises obtaining the rendering information from the lookup table based on a second combination of parameters with up to (N−1) th priority as a previous priority based on the parameter value with the Nth priority being less than the reference value, and
wherein the method further comprising deactivating the obtaining of the rendering information and the rendering of the audio object based on N being 1.

### Claim 6 (depends on 2)

The method of claim 2, wherein the obtaining of the lookup table comprises:
changing at least one parameter value and audio rendering information related to the spatialization of the audio object in response to an output of a video signal and an audio signal from the electronic apparatus,
obtaining a first assessment score from a plurality of assessors;
obtaining second assessment score from the plurality of assessors with respect to a combination of a plurality of parameters,
obtaining a first parameter value and rendering information corresponding to the first parameter value based on a highest average assessor score being obtained; and
generating and storing the lookup table including the first parameter value and the rendering information corresponding to the first parameter value.

### Claim 7 (depends on 1)

The method of claim 1, wherein the parameter related to spatialization further comprises an object parameter of an audio object type of the audio object, and
wherein the audio object type comprises at least one of information indicating whether the audio object is a voice or information indicating whether the audio object is a background sound.

### Claim 8 (depends on 7)

The method of claim 7, wherein the object parameter of the feature of the video object associated with the audio object comprises a video object type, and
wherein the video object type comprises at least one of information indicating whether the video object is a human face or information indicating whether the video object is a foreground or a background.

### Claim 9 (depends on 1)

The method of claim 1, wherein the parameter related to spatialization further comprises an electronic apparatus parameter of a feature of the electronic apparatus,
wherein the electronic apparatus parameter comprises at least one of unique information or setting information of the electronic apparatus,
wherein the unique information comprises at least one of a sound feature, a screen size, or speaker layout information depending on a model of the electronic apparatus, and
wherein the setting information comprises at least one of an audio volume of the electronic apparatus, a direction of a screen, or an operation mode of the electronic apparatus.

### Claim 10 (depends on 1)

The method of claim 1, wherein the parameter related to spatialization further comprises a user parameter of a feature of a user, and
wherein the user parameter comprises information indicating at least one of a number of users or a position of the user.

### Claim 11

An electronic apparatus for audio signal processing, the electronic apparatus comprising:
a memory storing instructions; and
a processor configured to execute the instructions to:
obtain a parameter related to spatialization of an audio object,
obtain rendering information based on the parameter related to spatialization, and
render the audio object based on the rendering information,
wherein the parameter related to spatialization comprises an object parameter of a feature of a video object associated with the audio object,
wherein the parameter related to spatialization further comprises a plurality of parameters, each parameter of the plurality of parameters having a priority, and
wherein the processor is further configured to execute the instructions to obtain the rendering information by obtaining rendering information corresponding to a parameter of the plurality of parameters in an order of a parameter having a higher priority.

### Claim 12 (depends on 11)

The electronic apparatus of claim 11, wherein the processor is further configured to execute the instructions to:
obtain a lookup table; and
retrieve the rendering information from the obtained lookup table.

### Claim 13 (depends on 12)

The electronic apparatus of claim 12, wherein the processor is further configured to execute the instructions to, based on the rendering information not being included in the lookup table:
estimate the rendering information based on the lookup table using at least one of a statistical model, a machine learning model, or a regression model.

### Claim 14 (depends on 12)

The electronic apparatus of claim 12,
wherein the processor is further configured to execute the instructions to, based a parameter value with Nth priority, wherein N comprises a natural number, being greater than or equal to a reference value, sequentially consider a parameter value with (N+1) th priority as a next priority, and
obtain the rendering information from the lookup table based on a first combination of parameters greater than or equal to the reference value.

### Claim 15 (depends on 14)

The electronic apparatus of claim 14, wherein the processor is further configured to execute the instructions to
obtain the rendering information from the lookup table based on a second combination of parameters with up to (N−1) th priority as a previous priority based on the parameter value with the Nth priority being less than the reference value, and
deactivate the obtaining of the rendering information and the rendering of the audio object based on N being 1.

### Claim 16 (depends on 12)

The electronic apparatus of claim 12, wherein the lookup table comprises a value for each of types of parameters and rendering information corresponding to the value for each of the types, and
wherein the processor is further configured to execute the instructions to obtain the lookup table by:
changing at least one parameter value and audio rendering information related to spatialization of an audio object in response to a video signal and an audio signal output from the electronic apparatus,
obtaining a first assessment score from a plurality of assessors,
obtaining second assessment score from the plurality of assessors with respect to a combination of a plurality of parameters,
obtaining a first parameter value and rendering information corresponding to the first parameter value based on a highest average assessor score being obtained, and
generating and storing the lookup table including the first parameter value and the rendering information corresponding to the first parameter value.

### Claim 17 (depends on 11)

The electronic apparatus of claim 11, wherein the parameter related to spatialization further comprises an object parameter of an audio object type of the audio object, and
wherein the audio object type comprises at least one of information indicating whether the audio object is a voice or information indicating whether the audio object is a background sound.

### Claim 18 (depends on 17)

The electronic apparatus of claim 17, wherein the object parameter of the feature of the video object associated with the audio object comprises a video object type, and
wherein the video object type comprises at least one of information indicating whether the video object is a human face or information indicating whether the video object is a foreground or a background.

### Claim 19 (depends on 11)

The electronic apparatus of claim 11, wherein the parameter related to spatialization further comprises an electronic apparatus parameter of a feature of the electronic apparatus,
wherein the electronic apparatus parameter comprises at least one of unique information or setting information of the electronic apparatus,
wherein the unique information comprises at least one of a sound feature, a screen size, or speaker layout information depending on a model of the electronic apparatus, and
wherein the setting information comprises at least one of an audio volume of the electronic apparatus, a direction of a screen, or an operation mode of the electronic apparatus.

### Claim 20

A non-transitory computer-readable recording medium storing instructions that, when executed by at least one processor, cause the at least one processor to:
obtain a parameter related to spatialization of an audio object;
obtain rendering information based on the parameter related to spatialization; and
render the audio object based on the rendering information,
wherein the parameter related to spatialization comprises an object parameter of a feature of a video object associated with the audio object,
wherein the parameter related to spatialization further comprises a plurality of parameters, each parameter of the plurality of parameters having a priority, and
wherein the instructions, when executed by the at least one processor, cause the at least one processor to obtain the rendering information by obtaining rendering information corresponding to a parameter of the plurality of parameters in an order of a parameter having a higher priority.