# US 10,375,499 B2 · Sound signal processing apparatus, sound signal processing method, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,375,499 B2 - Document ID: US10375499B2 - Status: Granted - Kind: B2 - Application number: 15/631,934 - Filed: 2017-06-23 - Published: 2018-01-04 - Granted: 2019-08-06 - Earliest claimed: 2016-06-29 - Assignees: Canon Kabushiki Kaisha - Applicants: Canon Kabushiki Kaisha - Inventors: Kyohei Kitazawa - Web page: https://patentgrep.com/patent/US10375499B2 ## CPC - H04S 7/301: {Automatic calibration of stereophonic sound system, e.g. with test microphone} - G10L 21/0272: Voice signal separating - H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)} - H04R 25/407: {Circuits for combining signals of a plurality of transducers} - G10L 2021/02166: {Microphone arrays; Beamforming} - G11B 2020/10546: {specifically adapted for audio data} - H04R 29/005: {Microphone arrays} - H04R 2410/05: Noise reduction with a separate noise microphone - H04S: STEREOPHONIC SYSTEMS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract A sound signal processing apparatus includes a sound acquisition unit, a processing unit, an output unit, and a control unit. The sound acquisition unit is configured to acquire sound data generated by collecting a sound from a sound-collecting target space. The processing unit is configured to execute, on the sound data acquired by the sound acquisition unit, signal processing corresponding to each of a plurality of areas included in the sound-collecting target space. The output unit is configured to output processed data generated based on the signal processing executed by the processing unit. The control unit configured to perform control such that an amount of processing performed by the processing unit does not exceed a predetermined amount of processing. ## Claims ### Claim 1 A sound signal processing apparatus comprising: one or more processors; and a memory which stores instructions executable by the one or more processors to cause the sound signal processing apparatus to perform at least: acquiring sound data generated by collecting a sound in a sound-collecting target space; detecting an amount of processing by a processor included in the one or more processors; causing the processor to execute signal processing for a sound in an area of a plurality of areas included in the sound-collecting target space by using the acquired sound data, wherein the signal processing is controlled based on a result of the detecting of the amount of processing by the processor, and outputting processed data generated based on the executed signal processing. ### Claim 2 (depends on 1) The sound signal processing apparatus according to claim 1, wherein the signal processing is controlled such that a period between a first time point when a sound is collected and a second time point when a sound based on the output processed data generated based on the signal processing using the sound collected at the first time point is reproduced is equal to a threshold value or less. ### Claim 3 (depends on 1) The sound signal processing apparatus according to claim 1, wherein the instructions further cause the sound signal processing apparatus to perform: after outputting the processed data generated by based on the executing, the signal processing using the acquired sound data according to control to reduce the amount of processing of the processor, outputting different processed data generated based on executing signal processing which requires a larger amount of processing than an amount of processing required by the signal processing according to the control. ### Claim 4 (depends on 1) The sound signal processing apparatus according to claim 1, wherein the instructions further cause the sound signal processing apparatus to perform: determining the plurality of areas by dividing the sound-collecting target space, wherein the processor executes the signal processing for a sound in the area of the plurality of determined areas, and wherein a size of the area is determined such that the amount of processing by the processor does not exceed a predetermined amount of processing. ### Claim 5 (depends on 4) The sound signal processing apparatus according to claim 4, wherein the size of each of the plurality of areas is determined in the determining. ### Claim 6 (depends on 1) The sound signal processing apparatus according to claim 1, wherein the signal processing is sound source separation processing which extracts the sound in the area of the plurality of areas from the acquired sound data. ### Claim 7 (depends on 6) The sound signal processing apparatus according to claim 6, wherein the number of sound channels to be used in the sound source separation processing such that the amount of processing performed by the processing unit does not exceed the predetermined amount of processing. ### Claim 8 (depends on 6) The sound signal processing apparatus according to claim 6, wherein the control unit controls the number of taps of a filter used in the sound source separation processing among a plurality of channels of the acquired sound data is controlled based on the result of the detecting such that the amount of processing by the processor does not exceed a predetermined amount of processing. ### Claim 9 (depends on 1) The sound signal processing apparatus according to claim 1, wherein a quantization bit rate relating to the signal processing is controlled such that the amount of processing by the processor does not exceed a predetermined amount of processing. ### Claim 10 (depends on 1) The sound signal processing apparatus according to claim 1, wherein a sampling frequency relating to the signal processing is controlled such that the amount of processing by the processor does not exceed a predetermined amount of processing. ### Claim 11 (depends on 1) The sound signal processing apparatus according to claim 1, wherein the instructions further cause the signal processing apparatus to perform: acquiring information about a listening point corresponding to a sound for reproduction which is based on the sound data acquired, wherein the amount of processing for the signal processing is controlled based on the information about the listening point acquired. ### Claim 12 A sound signal processing apparatus comprising: one or more processors; and a memory which stores instructions executable by the one or more processors to cause the sound signal processing apparatus to perform at least: acquiring sound data generated by collecting a sound in a sound-collecting target space; executing signal processing for a sound in an area of a plurality of areas included in the sound-collecting target space by using the acquired sound data; detecting data amount of processed data generated based on the executed signal processing, and outputting data based on the processed data generated based on the executed signal processing executed by the processing unit, wherein the output data is controlled based on a result of the detecting of the amount of the processed data. ### Claim 13 (depends on 12) The sound signal processing apparatus according to claim 12, wherein the output data is controlled such that a period between a first time point when a sound is collected and a second time point when a sound based on the output data based on the signal processing using the sound collected at the first time point is reproduced is equal to a threshold value or less. ### Claim 14 (depends on 12) The sound signal processing apparatus according to claim 12, wherein the instructions further cause the sound signal processing apparatus to perform: after outputting the data based on the processed data according to control to reduce the data amount of the processed data, outputting different processed data based on the processed data and having a larger data amount than the data amount of the output data according to the control. ### Claim 15 (depends on 12) The sound signal processing apparatus according to claim 12, wherein a plurality of pieces of processed data which is generated based on the executed signal processing is combined together such that the amount of the combined data to be output does not exceed predetermined data amount, wherein the plurality of pieces of processed data respectively corresponds to different areas. ### Claim 16 (depends on 15) The sound signal processing apparatus according to claim 15, wherein the instructions further cause the signal processing apparatus to perform: acquiring information about a listening point corresponding to a sound for reproduction which is based on the sound data acquired, wherein a plurality of pieces of data corresponding to the plurality of areas determined according to the acquired information about the listening point is combined together. ### Claim 17 (depends on 12) The sound signal processing apparatus according to claim 12, wherein the signal processing is sound source separation processing which extracts the sound in the area of the plurality of areas from the acquired sound data. ### Claim 18 (depends on 12) The sound signal processing apparatus according to claim 12, wherein a quantization bit rate relating to the processed data is controlled such that the data amount of the output data based on the processed data does not exceed a predetermined data amount. ### Claim 19 (depends on 12) The sound signal processing apparatus according to claim 12, wherein a sampling frequency relating to the processed data is controlled such that the data amount of the output data of the processed data does not exceed predetermined data amount. ### Claim 20 (depends on 12) The sound signal processing apparatus according to claim 12, wherein the instructions further cause the signal processing apparatus to perform: acquiring information about a listening point corresponding to a sound for reproduction which is based on the sound data acquired, wherein the data amount of the output data is controlled based on the information about the listening point acquired. ### Claim 21 A sound signal processing method executed by a sound signal processing apparatus comprising a processor, the method comprising: acquiring sound data generated by collecting a sound from a sound-collecting target space; detecting an amount of processing by the processor; causing the processor to execute signal processing for a sound in an area of a plurality of areas included in the sound-collecting target space by using the acquired sound data, wherein the signal processing is controlled based on a result of the detecting of the amount of processing by the processor; and outputting processed data generated based on the executed signal processing by the processor. ### Claim 22 (depends on 21) The sound signal processing method according to claim 21, wherein the signal processing is controlled such that a period between a first time point when a sound is collected and a second time point when a sound based on the output processed data generated based on the signal processing using the sound collected at the first time point is reproduced is equal to a threshold value or less. ### Claim 23 (depends on 21) The sound signal processing method according to claim 21, wherein the signal processing is sound source separation processing which extracts the sound in the area of the plurality of areas from the acquired sound data. ### Claim 24 A non-transitory storage medium storing a program for causing a computer to execute a sound signal processing method, the sound signal processing method comprising: acquiring sound data generated by collecting a sound from a sound-collecting target space; detecting an amount of processing by a processor; causing the processor to execute signal processing for a sound in an area of a plurality of areas included in the sound-collecting target space by using the acquired sound data, wherein the signal processing is controlled based on a result of the detecting of the amount of processing by the processor; and outputting processed data generated based on the executed signal processing by the processor.