# US 2019/0317178 A1 · Device control method, apparatus and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0317178 A1
- Document ID: US20190317178A1
- Status: Published application
- Kind: A1
- Application number: 16/462,453
- Filed: 2017-10-19
- Published: 2019-10-17
- Earliest claimed: 2016-11-23
- Assignees: Hangzhou Hikvision Digital Technology Co., Ltd.
- Applicants: Hangzhou Hikvision Digital Technology Co., Ltd.
- Inventors: Yangkun CHEN; Saijuan HE; Zhan CHEN
- Web page: https://patentgrep.com/patent/US20190317178A1

## CPC

- G01S 3/7864: {T.V. type tracking systems}
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- G01S 5/00: Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations {(using active systems G01S13/00, G01S15/00, G01S17/00)}
- G01S 5/18: using ultrasonic, sonic or infrasonic waves
- G01S 5/22: Position of source determined by co-ordinating a plurality of position lines defined by path-difference measurements (G01S5/28 takes precedence)
- H04N 7/15: Conference systems
- H04N 21/42204: {User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor (constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205)}
- H04N 23/60: Control of cameras or camera modules
- H04N 23/62: Control of parameters via user interfaces
- H04N 23/675: {comprising setting of focusing regions}
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- H04R 1/406: {microphones}
- H04N 21/42206: {characterized by hardware details}
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Embodiments of the present application provide a method, apparatus and system for controlling a device. The method is applicable to a video capture device (610) in a sound source localization system. The sound source localization system further includes microphones (620) disposed outside the video capture device (610). The method includes: acquiring a voice signal captured by each of the microphones (620), and acquiring coordinate information of each of the microphones (620) and coordinate information of each of preset positions (S101), calculating a voice power value corresponding to each of the preset positions based on the voice signal captured by each of the microphones (620), the coordinate information of each of the microphones (620) and the coordinate information of each of the preset positions (S102), identifying a preset position with a maximum voice power value, determining the preset position as a position of the sound source, and controlling the video capture device to direct at the position of the sound source (S103). In the present application, microphones (620) are disposed outside the video capture device (610), and the sound in the scene to be captured can all be captured by the microphones (620). Therefore, the accuracy of the sound source localization can be improved, thereby improving the accuracy of the device control.

## Claims

### Claim 1

A method for controlling a device, which is applicable to a video capture device in a sound source localization system; wherein, the sound source localization system further comprises microphones disposed outside the video capture device, and the method comprises:
acquiring a voice signal captured by each of the microphones, and acquiring coordinate information of each of the microphones and coordinate information of each of preset positions;
calculating a voice power value corresponding to each of the preset positions based on the voice signal captured by each of the microphones, the coordinate information of each of the microphones and the coordinate information of each of the preset positions;
identifying a preset position with a maximum voice power value, determining the preset position as a position of a sound source, and controlling the video capture device to direct at the position of the sound source.

### Claim 2 (depends on 1)

The method of claim 1, wherein, calculating a voice power value corresponding to each of the preset positions based on the voice signal captured by each of the microphones, the coordinate information of each of the microphones and the coordinate information of each of the preset positions, comprises:
calculating a Fourier transform of each voice signal captured by each of the microphones;
for each of the preset positions:
calculating a difference of delay between the preset position and every two adjacent microphones based on the coordinate information of the preset position and the coordinate information of each of the microphones;
calculating, based on the Fourier transform of each voice signal and the difference of delay between the preset position and every two adjacent microphones, a generalized cross-correlation of the preset position with every two adjacent microphones;
calculating, based on the generalized cross-correlation of the preset position with every two adjacent microphones, the voice power value corresponding to the preset position.

### Claim 3 (depends on 2)

The method of claim 2, wherein, for each of the preset positions, calculating a difference of delay between the preset position and every two adjacent microphones based on the coordinate information of the preset position and the coordinate information of each of the microphones, comprises:
calculating a difference of delay τmkl between any preset position m and any two adjacent microphones k, l according to a following formula:
τ
mkl
=

D
mk
-
D
ml

c
wherein, Dmk is a distance from the preset position m to the microphone k, Dml is the a distance from the preset position m to the microphone 1, and c is the sound speed.

### Claim 4 (depends on 3)

The method of claim 3, wherein, calculating, based on the Fourier transform of each voice signal and the difference of delay between the preset position and every two adjacent microphones, a generalized cross-correlation of the preset position with every two adjacent microphones, comprises:
calculating a generalized cross-correlation R(τmkl) of any preset position m with any two adjacent microphones k, l according to a following formula:
R(τmkl)=∫−∞+∞ϕkl(w)Mk(w)M*l(w)ejwτdw
wherein, Mk(w) is a Fourier transform of a voice signal received by the microphone k; M*l(w) is a conjugate of a Fourier transform of a voice signal received by the microphone 1; w is a frequency of a voice signal; ϕkl(w) is determined by a following formula:
φ
kl

(
w
)
=
1

M
k

(
w
)

M
l
*

(
w
)

.

### Claim 5 (depends on 4)

The method of claim 4, wherein, calculating, based on the generalized cross-correlation of the preset position with every two adjacent microphones, the voice power value corresponding to the preset position, comprises:
calculating a voice power value P(m) corresponding to any preset position m according to a following formula:
P

(
m
)
=
∑
k
=
1
M

∑
l
=
k
+
1
M

R

(
τ
mkl
)
wherein, M is a total number of the microphones.

### Claim 6 (depends on 1)

The method of claim 1, wherein, the method further comprises:
determining a height of a person at the position of the sound source;
calculating, based on coordinate information of the video capture device, a height of the video capture device, coordinate information of the position of the sound source and the height of the person at the position of the sound source, a target distance from the video capture device to the person;
identifying, based on a pre-stored correspondence between distances and focal lengths, a target focal length corresponding to the target distance, and adjusting a focal length of the video capture device to the target focal length.

### Claim 7 (depends on 6)

The method of claim 6, wherein, determining a height of a person at the position of the sound source, comprises:
acquiring a pre-stored average height of people, and determining the acquired average height of people as the height of the person at the position of the sound source; or
capturing an image comprising the person at the position of the sound source, and analyzing the image to obtain the height of the person at the position of the sound source.

### Claim 8

An apparatus for controlling a device, which is applicable to a video capture device in a sound source localization system; wherein, the sound source localization system further comprises microphones disposed outside the video capture device, and the apparatus comprises:
an acquisition module, configured for acquiring a voice signal captured by each of the microphones, and acquiring coordinate information of each of the microphones and coordinate information of each of preset positions;
a first calculation module, configured for calculating a voice power value corresponding to each of the preset positions based on the voice signal captured by each of the microphones, the coordinate information of each of the microphones and the coordinate information of each of the preset positions;
a control module, configured for identifying a preset position with a maximum voice power value, determining the preset position as a position of a sound source, and controlling the video capture device to direct at the position of the sound source.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein, the calculation module comprises:
a first calculation submodule, configured for calculating a Fourier transform of each voice signal captured by each of the microphones;
a second calculation submodule, configured for, for each of the preset positions, calculating a difference of delay between the preset position and every two adjacent microphones based on the coordinate information of the preset position and the coordinate information of each of the microphones;
a third calculation submodule, configured for, for each of the preset positions, calculating a generalized cross-correlation of the preset position with every two adjacent microphones based on the Fourier transform of each voice signal and the difference of delay between the preset position and every two adjacent microphones;
a fourth calculation submodule, configured for, for each of the preset positions, calculating the voice power value corresponding to the preset position based on the generalized cross-correlation of the preset position with every two adjacent microphones.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein, the second calculation submodule is specifically configured for calculating a difference of delay τmkl between any preset position m and any two adjacent microphones k, l according to a following formula:
τ
mkl
=

D
mk
-
D
ml

c
wherein, Dmk is a distance from the preset position m to the microphone k, Dml is the a distance from the preset position m to the microphone l, and c is the sound speed.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein, the third calculation submodule is specifically configured for calculating a generalized cross-correlation R(τmkl) of any preset position m with any two adjacent microphones k, l according to a following formula:
R(τmkl)=∫−∞+∞ϕkl(w)Mk(w)M*l(w)ejwτdw
wherein, Mk(w) is a Fourier transform of a voice signal received by the microphone k; M*l(w) is a conjugate of a Fourier transform of a voice signal received by the microphone l; w is a frequency of a voice signal; ϕkl(w) is determined by a following formula:
φ
kl

(
w
)
=
1

M
k

(
w
)

M
l
*

(
w
)

.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein, the fourth calculation submodule is specifically configured for calculating a voice power value P(m) corresponding to any preset position m according to a following formula:
P

(
m
)
=
∑
k
=
1
M

∑
l
=
k
+
1
M

R

(
τ
mkl
)
wherein, M is a total number of the microphones.

### Claim 13 (depends on 8)

The apparatus of claim 8, wherein, the apparatus further comprises:
a determination module, configured for determining a height of a person at the position of the sound source;
a second calculation module, configured for calculating, based on coordinate information of the video capture device, a height of the video capture device, coordinate information of the position of the sound source and the height of the person at the position of the sound source, a target distance from the video capture device to the person;
an adjustment module, configured for identifying, based on a pre-stored correspondence between distances and focal lengths, a target focal length corresponding to the target distance, and adjusting a focal length of the video capture device to the target focal length.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein, the determination module is specifically configured for:
acquiring a pre-stored average height of people, and determining the acquired average height of people as the height of the person at the position of the sound source; or
capturing an image comprising the person at the position of the sound source, and analyzing the image to obtain the height of the person at the position of the sound source.

### Claim 15

A system for controlling a device, comprising a video capture device and microphones disposed outside the video capture device; wherein,
the video capture device is configured for acquiring a voice signal captured by each of the microphones, and acquiring coordinate information of each of the microphones and coordinate information of each of preset positions, calculating a voice power value corresponding to each of the preset positions based on the voice signal captured by each of the microphones, the coordinate information of each of the microphones and the coordinate information of each of the preset positions, identifying a preset position with a maximum voice power value, determining the preset position as a position of a sound source, and controlling the video capture device to direct at the position of the sound source;
each of the microphones is configured for capturing a voice signal, and sending the captured voice signal to the video capture device.

### Claim 16 (depends on 1)

A video capture device, comprising:
a processor, a memory, a communication interface and a bus; wherein,
the processor, the memory, and the communication interface are connected and communicate with each other via the bus;
the memory stores executable program code;
the processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, to implement the method for controlling a device of claim 1.

### Claim 17 (depends on 1)

A storage medium, configured for storing executable program code; wherein, the executable program code is configured for, when executed, implementing the method for controlling a device of claim 1.

### Claim 18

(canceled)