# US 10,768,292 B2 · Surveillance apparatus having a radar sensor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,768,292 B2 - Document ID: US10768292B2 - Status: Granted - Kind: B2 - Application number: 14/978,347 - Filed: 2015-12-22 - Published: 2016-06-30 - Granted: 2020-09-08 - Earliest claimed: 2014-12-29 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Qi Wang - Web page: https://patentgrep.com/patent/US10768292B2 ## CPC - G01S 13/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S13/46) - G01S 13/343: {using sawtooth modulation} - G01S 13/867: {Combination of radar systems with cameras} - G01S 13/886: {for alarm systems (alarms with electrical actuation G08B13/22)} - G01S 7/356: {involving particularities of FFT processing} - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract A surveillance apparatus for determining a position of an object in a field of view, comprising a radar sensor having at least one transmitting antenna configured to transmit electromagnetic radiation and a plurality of receiving antennas configured to receive electromagnetic radiation including a reflection signal of the object, wherein the antennas form an antenna array, and a processing unit connected to the radar sensor to receive signals from the receiving antennas corresponding to the received electromagnetic radiation, wherein the processing unit is configured to estimate a distance and a direction of the position of the object in the field of view with respect to the radar sensor, wherein the processing unit is configured to estimate the distance and the direction separately on the basis of a maximum likelihood based algorithm. ## Claims ### Claim 1 A surveillance apparatus for determining a position of an object among objects in a field of view, comprising: a radar sensor having at least one transmitting antenna configured to transmit electromagnetic radiation and a plurality of receiving antennas configured to receive electromagnetic radiation including a reflection signal of the object, wherein the antennas form an antenna array, and processing circuitry configured to receive signals from the receiving antennas corresponding to the received electromagnetic radiation, calculate, as a beat frequency, a frequency of mixed signals of the received signals and signals transmitted from the transmitting antenna, estimate a correlation matrix on the basis of the received signals, estimate a number of the objects in the field of view from the estimated correlation matrix, perform a maximum likelihood estimation using the calculated beat frequency, the estimated correlation matrix and the estimated number of the objects to calculate an angle of the object, determine whether an angle of the object has been calculated, repeat the calculation of the beat frequency, the estimation of the correlation matrix and the number of the objects, and the performance of the maximum likelihood estimation in a case where it is determined that the angle of the object has not been calculated, determine frequency peaks in a transformed signal spectrum of the received signals in a case where it is determined that the angle of the object has been calculated, determine each distance of the objects in the field of view based on the determined frequency peaks, determine a map of amplitudes for different positions in the field of view based on a corresponding plurality of beamformed signals, determine a power of signals of the different positions received by the radar sensor based on the determined map and corresponding values of the estimated correlation matrix, and output the direction, the distance, and the power of signals of the position of the object in the field of view with respect to the radar sensor separately. ### Claim 2 (depends on 1) The surveillance apparatus as claimed in claim 1, wherein the processing circuitry is configured to determine the distance on the basis of a Fourier transformation of the plurality of received signals. ### Claim 3 (depends on 2) The surveillance apparatus as claimed in claim 2, wherein the frequency peaks correspond to the same object in the field of view. ### Claim 4 (depends on 1) The surveillance apparatus as claimed in claim 1, wherein the processing circuitry is configured to determine the direction of the position of the object as an angle with respect to the radar sensor. ### Claim 5 (depends on 4) The surveillance apparatus as claimed in claim 4, wherein the angle and the direction are determined with respect to a center of the radar sensor. ### Claim 6 (depends on 4) The surveillance apparatus as claimed in claim 4, wherein the processing circuitry is configured to determine the angle on the basis of a phase angle of the received signals. ### Claim 7 (depends on 4) The surveillance apparatus according claim 4, wherein the processing circuitry is configured to determine the angle using the determined distance. ### Claim 8 (depends on 1) The surveillance apparatus as claimed in claim 1, wherein the processing circuitry is configured to use a near-field wave front model in order to determine the distance and the direction of the position of the object in the field of view. ### Claim 9 (depends on 1) The surveillance apparatus as claimed in claim 1, wherein the correlation matrix is determined in different consecutive measurements of the radar sensor. ### Claim 10 (depends on 1) The surveillance apparatus as claimed in claim 1, wherein the processing circuitry is configured to determine an amount of different reflection signals from the received signals corresponding to different objects in the field of view on the basis of eigenvalues of the correlation matrix. ### Claim 11 (depends on 10) The surveillance apparatus as claimed in claim 10, wherein the eigenvalues are compared to a standard deviation of noise to determine the number of the objects. ### Claim 12 (depends on 11) The surveillance apparatus as claimed in claim 11, wherein the eigenvalues are sorted in an ascending order, a ratio of consecutive eigenvalues in the ascending order is determined, and the processing circuitry is configured to determine the number of the objects on the basis of a threshold level for the ratio of the consecutive eigenvalues. ### Claim 13 (depends on 1) The surveillance apparatus as claimed in claim 1, wherein the radar sensor comprises emitting antennas configured to emit electromagnetic radiation signals, which are formed as frequency modulated continuous wave signals, and the signals corresponding to the received electromagnetic radiation have different frequencies each dependent on a distance to the object in the field of view to the respective receiving antenna. ### Claim 14 (depends on 1) The surveillance apparatus as claimed in claim 1, further comprising an optical camera configured to capture images based on received light. ### Claim 15 (depends on 14) The surveillance apparatus as claimed in claim 14, wherein the optical camera comprises a camera aperture, and the transmitting antennas and/or the receiving antennas are arranged around the camera aperture. ### Claim 16 A surveillance method for estimating a position of an object among objects in a field of view of a radar sensor, comprising: transmitting electromagnetic radiation by at least one transmitting antenna, receiving electromagnetic radiation from the field of view by a plurality of receiving antennas, receiving signals from the receiving antennas corresponding to the received electromagnetic radiation, calculating, using processing circuitry, as a beat frequency, a frequency of mixed signals of the received signals and signals transmitted from the transmitting antenna, estimating a correlation matrix on the basis of the received signals, estimating a number of the objects in the field of view from the estimated correlation matrix, performing a maximum likelihood estimation using the calculated beat frequency, the estimated correlation matrix and the estimated number of the objects to calculate an angle of the object, determining whether an angle of the object has been calculated, repeating the calculation of the beat frequency, the estimation of the correlation matrix and the number of the objects, and the performance of the maximum likelihood estimation in a case where it is determined that the angle of the object has not been calculated, determining frequency peaks in a transformed signal spectrum of the received signals in a case where it is determined that the angle of the object has been calculated, determining each distance of the objects in the field of view based on the determined frequency peaks, determining a map of amplitudes for different positions in the field of view based on a corresponding plurality of beamformed signals, determining a power of signals of the different positions received by the radar sensor based on the determined map and corresponding values of the estimated correlation matrix, and outputting the direction, the distance, and the power of signals of the position of the object in the field of view with respect to the radar sensor separately. ### Claim 17 A non-transitory computer-readable recording medium including executable instructions, which when executed by a computer cause the computer to execute a method for estimating a position of an object among objects in a field of view of a radar sensor, the method comprising: transmitting electromagnetic radiation by at least one transmitting antenna, receiving electromagnetic radiation from the field of view by a plurality of receiving antennas, receiving signals from the receiving antennas corresponding to the received electromagnetic radiation, calculating, as a beat frequency, a frequency of mixed signals of the received signals and signals transmitted from the transmitting antenna, estimating a correlation matrix on the basis of the received signals, estimating a number of the objects in the field of view from the estimated correlation matrix, performing a maximum likelihood estimation using the calculated beat frequency, the estimated correlation matrix and the estimated number of the objects to calculate an angle of the object, determining whether an angle of the object has been calculated, repeating the calculation of the beat frequency, the estimation of the correlation matrix and the number of the objects, and the performance of the maximum likelihood estimation in a case where it is determined that the angle of the object has not been calculated, determining frequency peaks in a transformed signal spectrum of the received signals in a case where it is determined that the angle of the object has been calculated, determining each distance of the objects in the field of view based on the determined frequency peaks, determining a map of amplitudes for different positions in the field of view based on a corresponding plurality of beamformed signals, determining a power of signals of the different positions received by the radar sensor based on the determined map and corresponding values of the estimated correlation matrix, and outputting the direction, the distance, and the power of signals of the position of the object in the field of view with respect to the radar sensor separately.