# US 2004/0001137 A1 · Integrated design for omni-directional camera and microphone array

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2004/0001137 A1
- Document ID: US20040001137A1
- Status: Published application
- Kind: A1
- Application number: 10/184,499
- Filed: 2002-06-27
- Published: 2004-01-01
- Earliest claimed: 2002-06-27
- Assignees: Microsoft Corporation
- Inventors: Ross Cutler; Ivan Tashev; Yong Rui; Anoop Gupta
- Web page: https://patentgrep.com/patent/US20040001137A1

## CPC

- H04R 3/02: for preventing acoustic reaction {, i.e. acoustic oscillatory feedback (specially adapted for hearing aids H04R25/453)}
- H04N 7/142: {Constructional details of the terminal equipment, e.g. arrangements of the camera and the display}
- H04N 23/58: Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
- H04N 23/66: Remote control of cameras or camera parts, e.g. by remote control devices
- H04R 2201/401: 2D or 3D arrays of transducers
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An omni-directional camera (a 360 degree camera) is proposed with an integrated microphone array. The primary application for such a camera is videoconferencing and meeting recording, and the device is designed to be placed on a meeting room table. The microphone array is in a planar configuration, and the microphones are located as close to the desktop as possible to eliminate sound reflections from the table. The camera is connected to the microphone array base with a thin cylindrical rod, which is acoustically invisible to the microphone array for the frequency range [50-4000] Hz. This provides a direct path from the person talking to all of the microphones in the array, and can therefore be used for sound source localization (determining the location of the talker) and beam-forming (improving the sound quality of the talker by filtering only sound from a particular direction). The camera array is elevated from the table to provide a near frontal viewpoint of the meeting participants.

## Claims

### Claim 1

A system for capturing audio and video data comprising:
one or more cameras that capture video image data;
an array of microphones, that captures audio signals; and
a cylinder connecting said one or more cameras to said microphone array, said cylinder being acoustically transparent in the frequency range of the human voice.

### Claim 2 (depends on 1)

The system of claim 1 wherein said one or more cameras capture images over 360 degrees.

### Claim 3 (depends on 1)

The system of claim 1 further comprising a microphone array base, and wherein said microphones are mounted to said microphone array base.

### Claim 4 (depends on 3)

The system of claim 3 wherein the height of said microphone array base is small enough to position said microphone array to minimize sound reflections picked up by the array from any surface.

### Claim 5 (depends on 1)

The system of claim 1 wherein the frequency range to which the cylinder is acoustically transparent is 50-4000 hertz.

### Claim 6 (depends on 1)

The system of claim 1 wherein the audio signals are input into a computer, said computer using said audio signals for sound source localization, said sound source localization determining the direction of sounds producing the audio signals.

### Claim 7 (depends on 1)

The system of claim 1 wherein the audio signals are input into a computer, said computer using said audio signals for beam-forming, said beam-forming improving the sound quality of a sound from one direction by filtering out the sound from all other directions.

### Claim 8 (depends on 1)

The system of claim 1 wherein the audio signals are input into a computer, said computer using human detection and tracking algorithms to accurately detect a person captured in the video image data and associate the video image data of the person with the audio signals generated by the person speaking.

### Claim 9 (depends on 1)

The system of claim 1 wherein the video image data is input into a computer, said computer using said video image data to stitch images together.

### Claim 10 (depends on 1)

The system of claim 1 wherein microphones in the microphone array are omni-directional.

### Claim 11 (depends on 1)

The system of claim 1 wherein the microphones in the microphone array are unidirectional.

### Claim 12 (depends on 1)

The system of claim 1 wherein the microphone array comprises three microphones.

### Claim 13 (depends on 1)

The system of claim 1 wherein the microphone array comprises eight microphones.

### Claim 14 (depends on 1)

The system of claim 1 wherein the microphone array comprises a plurality of microphones that are equilaterally disposed in a circle around the circumference of a planar microphone base.

### Claim 15 (depends on 14)

The system of claim 14 wherein at least one microphone is mounted in a rubber casing to protect it from extraneous sound reflections.

### Claim 16 (depends on 14)

The system of claim 14 wherein sound insulation is placed below at least one microphone to protect it from extraneous sound reflections.

### Claim 17 (depends on 1)

The system of claim 1 wherein said camera further comprises a lens cover, which is up in a normal operating mode, and down in a privacy mode.

### Claim 18 (depends on 17)

The system of claim 17 wherein the microphones are turned off when the camera is in said privacy mode is evoked.

### Claim 19 (depends on 1)

The system of claim 1 wherein a light on camera is on when the camera is active.

### Claim 20 (depends on 1)

The system of claim 1 wherein the audio signals are transmitted over a network.

### Claim 21 (depends on 1)

The system of claim 1 wherein the audio signals are saved to a computer-readable medium.

### Claim 22 (depends on 1)

The system of claim 1 wherein the video image data is transmitted over a network.

### Claim 23 (depends on 1)

The system of claim 1 wherein the video image data is saved to a computer-readable medium.

### Claim 24 (depends on 1)

The system of claim 1 wherein the video signals are transferred to a computer using a 1394 bus.

### Claim 25 (depends on 1)

The system of claim 1 wherein the audio signals are transferred to a computer using analog cables.

### Claim 26 (depends on 1)

The system of claim 1 wherein said cameras are IEEE 1394 cameras.

### Claim 27

A process of capturing video and audio for teleconferencing and meeting recording, comprising the following process actions:
capturing images of persons in a meeting with an omni-directional camera,
capturing audio signals of sounds occurring in said meeting with a microphone array, said microphone array being housed in a base which is connected to said omni-directional camera by an acoustically transparent rod.

### Claim 28 (depends on 27)

The process of claim 27 wherein said microphone array is placed on a table and said wherein said persons in said meeting are seated around said table.

### Claim 29 (depends on 28)

The process of claim 28 wherein the microphone array is as close to said table as possible to minimize sound reflections from the table being picked up by the array.

### Claim 30 (depends on 27)

The process of claim 27 wherein the microphone array comprises 3 or more microphones.

### Claim 31 (depends on 30)

The process of claim 30 wherein said microphone array comprises microphones that are equally spaced in a circle adjacent the circumference of the microphone base.

### Claim 32 (depends on 27)

The process of claim 27 wherein the omni-directional camera comprises multiple video sensors.

### Claim 33 (depends on 32)

The process of claim 32 wherein said video sensors are equally spaced around the circumference of a circle.

### Claim 34 (depends on 27)

The process of claim 27 wherein the omni-directional camera comprises a single video sensor with a hyperbolic lens that captures light rays from 360 degrees.

### Claim 35 (depends on 27)

The process of claim 27 wherein said acoustically transparent rod is hollow.

### Claim 36 (depends on 27)

The process of claim 27 wherein said acoustically transparent rod is attached to a microphone base and wherein camera cables go through the acoustically transparent rod so said cables do not cause acoustic barriers.

### Claim 37 (depends on 27)

The process of claim 27 wherein the outside diameter of said acoustically transparent rod is 2 centimeters or less.

### Claim 38 (depends on 27)

The process of claim 27 wherein the acoustically transparent rod has a diameter so as to permit audio frequencies from 50 to 4000 Hz to pass by the acoustically transparent rod essentially undisturbed from a person speaking to all microphones in the microphone array.

### Claim 39 (depends on 27)

The acoustically transparent rod of claim 27 wherein said rod height is adjustable.

### Claim 40

An integrated omni-directional camera and microphone array comprising:
an omni-directional camera;
an acoustically transparent rod that elevates said camera to provide optimum camera coverage; and
a microphone array.

### Claim 41 (depends on 40)

The integrated omni-directional camera and microphone array of claim 40 wherein said omni-directional camera employs multiple video sensors to achieve 360 degree camera coverage.

### Claim 42 (depends on 40)

The integrated omni-directional camera and microphone array of claim 40 wherein said omni-directional camera employs one video sensor and a hyperbolic lens that captures light from 360 degrees to achieve panoramic coverage.

### Claim 43 (depends on 40)

The integrated omni-directional camera and microphone array of claim 40 wherein the acoustically transparent cylindrical rod elevates said camera to provide a frontal view of people sitting around a table.

### Claim 44 (depends on 40)

The integrated omni-directional camera and microphone array of claim 40 wherein the microphone array comprises a plurality of microphones disposed at equilateral distances around the circumference of a circle.

### Claim 45 (depends on 40)

The integrated omni-directional camera and microphone array of claim 40 wherein the microphone array is sitting on a surface, as close to the surface as possible to minimize sound reflections from said surface.

### Claim 46 (depends on 41)

The integrated camera and microphone array of claim 41 wherein video sensors with longer, narrower fields of view are used to capture images at the longer distances, and video sensors with wider, shorter fields of view are used to capture images at shorter distances.

### Claim 47

An omni-directional microphone array comprising:
a plurality of microphones that are equilaterally disposed in a circle around the circumference of a planar microphone base; and wherein
said microphone base in low enough to position said microphones array to minimize sound reflections picked up by the array from any surface the microphone base is sitting on.

### Claim 48 (depends on 47)

The microphone array of claim 47 wherein audio signals captured by said plurality of microphones are input into a computer, said computer using said audio signals for sound source localization, said sound source localization determining the direction of sounds producing the audio signals.

### Claim 49 (depends on 47)

The microphone array of claim 47 wherein audio signals captured by said plurality of microphones are input into a computer, said computer using said audio signals for beam-forming, said beam-forming improving the sound quality of a sound from one direction by filtering out the sound from all other directions.

### Claim 50 (depends on 47)

The microphone array of claim 47 wherein the microphones in the microphone array are omni-directional.

### Claim at position 51 (depends on 47)

51 The microphone array of claim 47 wherein the microphones in the microphone array are unidirectional.

### Claim 52 (depends on 47)

The microphone array of claim 47 wherein the microphone array comprises eight microphones.

### Claim 53 (depends on 47)

The microphone array of claim 47 wherein at least one microphone is mounted in a rubber casing to protect it from extraneous sound reflections.

### Claim 54 (depends on 47)

The microphone array of claim 47 wherein sound insulation is placed below at least one microphone to protect it from extraneous sound reflections.
