# US 10,873,824 B2 · Apparatus, system, and method of processing data, and recording medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,873,824 B2 - Document ID: US10873824B2 - Status: Granted - Kind: B2 - Application number: 16/509,670 - Filed: 2019-07-12 - Published: 2019-11-07 - Granted: 2020-12-22 - Earliest claimed: 2017-03-07 - Assignees: Ricoh Company, Ltd. - Applicants: Takafumi Okuma - Inventors: Takafumi Okuma - Web page: https://patentgrep.com/patent/US10873824B2 ## CPC - H04S 7/303: {Tracking of listener position or orientation} - H04R 1/406: {microphones} - H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)} - H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)} - H04R 5/027: Spatial or constructional arrangements of microphones, e.g. in dummy heads - H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction - H04S 2420/11: Application of ambisonics in stereophonic audio systems - H04S: STEREOPHONIC SYSTEMS - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract An apparatus, method, and system each of which obtains sound data based on a plurality of sound signals respectively output from a plurality of microphones, receives a user instruction for enhancing directivity of sensitivity characteristics of at least one of the plurality of microphones in a specific direction, and generates sound data having the directivity in the specific direction, based on the obtained sound data. ## Claims ### Claim 1 A data processing apparatus comprising: processing circuitry configured to, obtain sound data based on a plurality of sound signals respectively output from a plurality of microphones, generate, based on the sound data, enhanced sound data having a directivity in a specific direction such that the directivity in the enhanced sound data is maintained in the specified direction in response to movement of one or more of the plurality of microphones while capturing the plurality of sound signals, combine a plurality of images captured with an image capturing device and information indicating an inclination angle of the plurality of microphones relative to a reference direction, and store, in a memory, the enhanced sound data having the directivity in the specific direction in association with at least the plurality of images and the information indicating the inclination angle of the plurality of microphones relative to the reference direction. ### Claim 2 (depends on 1) The data processing apparatus of claim 1, wherein the processing circuitry is further configured to, control a display to display a polar pattern that reflects the directivity of sensitivity characteristics of at least one of the plurality of the microphones in the specific direction, set directivity selection information for enhancing the directivity in the specific direction in response to a change in a shape of the polar pattern, and convert the sound data of the plurality of sound signals according to the directivity selection information, to generate the enhanced sound data having the directivity in the specific direction. ### Claim 3 (depends on 2) The data processing apparatus of claim 2, wherein the enhanced sound data having the directivity in the specific direction is stereophonic sound data. ### Claim 4 (depends on 2) The data processing apparatus of claim 2, wherein the processing circuitry is further configured to, receive a user instruction for enhancing the directivity of the sensitivity characteristics of at least one of the plurality of microphones in the specific direction, the user instruction being a user operation to change the shape of the polar pattern displayed on the display. ### Claim 5 (depends on 1) The data processing apparatus of claim 1, wherein the plurality of microphones includes one or more built-in microphones incorporated in the data processing apparatus, and one or more external microphones connected to the data processing apparatus. ### Claim 6 (depends on 5) The data processing apparatus of claim 5, wherein the enhanced sound data having the directivity includes sound data generated based on the built-in microphones, and sound data generated based on the external microphones. ### Claim 7 (depends on 5) The data processing apparatus of claim 5, wherein the processing circuitry is configured to, combine sound data generated based on the built-in microphones and sound data generated based on the external microphones into combined sound data, and generate the enhanced sound data having the directivity based on the combined sound data. ### Claim 8 (depends on 1) A system comprising: the data processing apparatus of claim 1; and an information processing apparatus communicably connected with the data processing apparatus, the information processing apparatus including, a display configured to display a polar pattern that reflects the directivity of sensitivity characteristics of the plurality of microphones; and processing circuitry configured to transmit a change in a shape of the polar pattern to the data processing apparatus for enhancing the directivity. ### Claim 9 (depends on 8) The system of claim 8, further comprising: a reproduction apparatus communicably connected with the data processing apparatus, the reproduction apparatus including a speaker configured to output sounds based on the enhanced sound data having the directivity in the specific direction. ### Claim 10 (depends on 1) The data processing apparatus of claim 1, wherein the image capturing device includes the one or more of the plurality of microphones such that the processing circuitry is configured to maintain the directivity in the enhanced sound data in the specified direction irrespective of a change in the inclination angle of the image capturing device. ### Claim 11 A system comprising: processing circuitry configured to, obtain sound data based on a plurality of sound signals respectively output from a plurality of microphones, generate, based on the sound data, enhanced sound data having a directivity in a specific direction such that the directivity in the enhanced sound data is maintained in the specified direction in response to movement of one or more of the plurality of microphones while capturing the plurality of sound signals, combine a plurality of images captured with an image capturing device and information indicating an inclination angle of the plurality of microphones relative to a reference direction, and store, in a memory, the enhanced sound data having the directivity in the specific direction in association with at least the plurality of images and the information indicating the inclination angle of the plurality of microphones relative to the reference direction. ### Claim 12 (depends on 11) The system of claim 11, wherein the processing circuitry is further configured to, control a display to display a polar pattern that reflects the directivity of sensitivity characteristics of at least one of the plurality of the microphones in the specific direction, and set directivity selection information for enhancing the directivity in the specific direction in response to a change in a shape of the polar pattern. ### Claim 13 (depends on 12) The system of claim 12, wherein the processing circuitry is further configured to output sounds based on the enhanced sound data having the directivity in the specific direction. ### Claim 14 (depends on 11) The system of claim 11, wherein the image capturing device includes the one or more of the plurality of microphones such that the processing circuitry is configured to maintain the directivity in the enhanced sound data in the specified direction irrespective of a change in the inclination angle of the image capturing device. ### Claim 15 A method of controlling data processing, comprising: obtaining sound data based on a plurality of sound signals respectively output from a plurality of microphones; generating, based on the sound data, enhanced sound data having a directivity in a specific direction such that the directivity in the enhanced sound data is maintained in the specified direction in response to movement of one or more of the plurality of microphones while capturing the plurality of sound signals; combining a plurality of images captured with an image capturing device and information indicating an inclination angle of the plurality of microphones relative to a reference direction; and storing, in a memory, the enhanced sound data having the directivity in the specific direction in association with at least the plurality of images and the information indicating the inclination angle of the plurality of microphones relative to the reference direction. ### Claim 16 (depends on 15) The method of claim 15, further comprising: displaying a polar pattern that reflects the directivity of sensitivity characteristics of the plurality of microphones; and setting directivity selection information for enhancing the directivity in the specific direction in response to a change in a shape of the polar pattern. ### Claim 17 (depends on 15) The method of claim 15, wherein the image capturing device includes the one or more of the plurality of microphones such that the generating generates the enhanced sound data maintains the directivity in the enhanced sound data in the specified direction irrespective of a change in the inclination angle of the image capturing device.