# US 10,122,972 B2 · System and method for localizing a talker using audio and video information > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,122,972 B2 - Document ID: US10122972B2 - Status: Granted - Kind: B2 - Application number: 15/808,665 - Filed: 2017-11-09 - Published: 2018-03-08 - Granted: 2018-11-06 - Earliest claimed: 2014-11-17 - Assignees: Polycom, Inc. - Applicants: Polycom, Inc. - Inventors: Jinwei Feng - Web page: https://patentgrep.com/patent/US10122972B2 ## CPC - H04N 7/142: {Constructional details of the terminal equipment, e.g. arrangements of the camera and the display} - G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) - G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)} - G06V 40/161: {Detection; Localisation; Normalisation} - H04N 7/15: Conference systems - H04R 1/406: {microphones} - G06T 2207/10016: Video; Image sequence - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract A videoconferencing endpoint includes at least one processor a number of microphones and at least one camera. The endpoint can receive audio information and visual motion information during a teleconferencing session. The audio information includes one or more angles with respect to the microphone from a location of a teleconferencing session. The system evaluates the audio information is evaluated to determine at least one candidate angle corresponding to a possible location of an active talker. The candidate angle can be analyzed further with respect to the motion information to determine whether the candidate angle correctly corresponds to person who is speaking during the teleconferencing session. The person's face can then be framed within a frame view. ## Claims ### Claim 1 A method of localizing an active talker, the method comprising: receiving, using one or more microphones, audio information corresponding to one or more angles with respect to the one or more microphones; determining, from amongst the one or more angles, at least one candidate angle corresponding to a possible location of an active talker, using an algorithm which gives greater weight to lower frequency bands than to higher frequency bands from within the received audio information; receiving motion information; analyzing the candidate angle with respect to the received motion information; and determining whether the candidate angle indicates an active talker based, at least in part, on the analysis. ### Claim 2 (depends on 1) The method of claim 1, wherein the algorithm used to determine the at least one candidate angle is implemented in the frequency domain. ### Claim 3 (depends on 2) The method of claim 2, wherein the algorithm used to determine the at least one candidate angle is implemented in the sub-band domain. ### Claim 4 (depends on 1) The method of claim 1, wherein using one or more microphones comprises using a plurality of microphones, at least three of which are in a planar configuration. ### Claim 5 (depends on 4) The method of claim 4, wherein at least three of microphones within the plurality of microphones are at least partially housed within a base of a teleconferencing device. ### Claim 6 (depends on 5) The method of claim 5, wherein the teleconferencing device includes at least one camera configured to receive visual information in a 360-degree plane. ### Claim 7 (depends on 1) The method of claim 1, further comprising capturing the motion information using at least one camera configured to receive visual information in a 360-degree angle of rotation. ### Claim 8 (depends on 1) The method of claim 1, further comprising capturing the motion information using at least one cardioid camera. ### Claim 9 (depends on 1) The method of claim 1, wherein receiving audio information using one or more microphones comprises receiving audio information using three microphones. ### Claim 10 A method of locating an active talker, the method comprising: receiving audio information using a predetermined number of microphones, the audio information corresponding to one or more angles with respect to the microphones with respect to a predetermined location; analyzing the received audio information according to at least one algorithm to determine at least one candidate angle, the candidate angle corresponding to a possible location of an active talker; receiving motion information from at least one camera; and analyzing the candidate angle with respect to the received motion information, wherein analyzing the candidate angle with respect to the received motion involves determining whether motion has been detected at the candidate angle within a predetermined period of time. ### Claim 11 (depends on 10) The method of claim 10, wherein the algorithm gives greater weight to lower frequency bands than to higher frequency bands from within the received audio information. ### Claim 12 (depends on 10) The method of claim 10, wherein analyzing the candidate angle with respect to the received motion further involves determining whether motion has been detected within a predetermined range of the candidate angle. ### Claim 13 (depends on 12) The method of claim 12, further comprising designating the candidate angle as the angle of the active talker when no motion has been detected within the predetermined range within a predetermined period. ### Claim 14 (depends on 12) The method of claim 12, wherein the predetermined range is plus or minus ten degrees of the candidate angle. ### Claim 15 (depends on 14) The method of claim 14, wherein the predetermined period is two milliseconds. ### Claim 16 (depends on 10) The method of claim 10, wherein at least one of the microphones is a cardioid microphone. ### Claim 17 (depends on 10) The method of claim 10, further comprising determining whether a view of a location corresponding to the candidate angle frames a face. ### Claim 18 (depends on 10) The method of claim 10, wherein receiving motion information comprises determining the absolute difference between two video frames captured 160 milliseconds apart. ### Claim 19 (depends on 18) The method of claim 18, wherein determining the absolute difference between two video frames captured 160 milliseconds apart comprises declaring at least one pixel to be a motion pixel when the absolute difference between the values of that pixel in the two video frames exceeds a predefined threshold.