# US 10,873,666 B2 · Camera tracking method and director device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,873,666 B2 - Document ID: US10873666B2 - Status: Granted - Kind: B2 - Application number: 16/791,268 - Filed: 2020-02-14 - Published: 2020-06-11 - Granted: 2020-12-22 - Earliest claimed: 2017-08-16 - Assignees: Huawei Technologies Co., Ltd. - Applicants: Huawei Technologies Co., Ltd. - Inventors: Zhiwei Zheng - Web page: https://patentgrep.com/patent/US10873666B2 ## CPC - H04M 3/567: {Multimedia conference systems} - G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10) - H04N 7/142: {Constructional details of the terminal equipment, e.g. arrangements of the camera and the display} - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - H04N 7/152: {Multipoint control units therefor} - H04N 23/60: Control of cameras or camera modules - H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In a camera tracking method and apparatus, and a device, a director device determines speech information of speaker in a preset time period. The director device determines a current speaker based on second video information collected by the first camera at a current moment and second audio information collected by the plurality of MICs at the current moment. The director device controls a director status of at least one director camera based on the speech information of speaker, the current speaker, and a speaker shot by the at least one director camera at the current moment. The director camera is configured to send a director video stream to another conference site. This avoids frequent and unnecessary switching of the director camera. ## Claims ### Claim 1 A camera tracking method, comprising: determining historical speech information of a speaker in a past time period based on first video information collected by a first camera in the past time period and first audio information collected by a plurality of microphones (MICs) in the past time period; determining a current speaker based on second video information currently collected by the first camera and second audio information currently collected by the plurality of MIC5; and controlling a director status of director cameras based on the speech information of a speaker, the current speaker, and a speaker the image of whom is currently captured by at least one director camera, wherein the director status of a director camera comprises a shooting angle or a focal length, and wherein when the current speaker is different from the speaker shot by the first director camera at the current moment, the director device adjusts the shooting angle or the focal length of at least one of the first director camera and a second director camera based on the historical speech information, wherein the at least one director camera is configured to send a director video stream to a conference site. ### Claim 2 (depends on 1) The method according to claim 1, wherein: the controlling the director status further comprises: when the current speaker is the same as a speaker shot by the first director camera at the current moment, keeping a shooting angle and a focal length of the first director camera unchanged, wherein a pilot video shot by the first director camera is sent to the another conference site at the current moment. ### Claim 3 (depends on 1) The method according to claim 1, wherein when the current speaker is different from the speaker shot by the first director camera at the current moment, determining a speech mode in a preset past time period based on the speech information of the speaker, wherein the speech mode comprises at least one of a single-person speech mode, a double-person debate mode, or a multi-person discussion mode; and adjusting the shooting angle or the focal length of at least one of the first director camera and the second director camera based on the speech mode in the preset past time period. ### Claim 4 (depends on 3) The method according to claim 3, wherein the determining the speech mode comprises: determining a quantity of effective speakers based on the speech information of the speakers; and determining the speech mode based on the quantity of effective speakers. ### Claim 5 (depends on 4) The method according to claim 4, wherein the obtaining a quantity of effective speakers comprises: determining a quantity of effective speech times of each speaker based on a priority of each speaker in the preset time period and speech duration of each speaker in each speech in a preset time period; determining a speaker whose quantity of effective speech times is greater than or equal to 1 as an effective speaker; and determining a quantity of effective speakers as the quantity of effective speakers. ### Claim 6 (depends on 4) The method according to claim 4, wherein the determining the speech mode comprises: when the quantity of effective speakers is 1, determining that the speech mode in a preset time period is the single-person speech mode; when the quantity of effective speakers is 2, if the two effective speakers speak alternately, determining that the speech mode in the preset time period is the single-person speech mode or the double-person debate mode; or when the quantity of effective speakers is greater than 2, determining, based on priorities of at least two effective speakers in the preset time period, that the speech mode in the preset time period is the single-person speech mode or the multi-person discussion mode. ### Claim 7 (depends on 3) The method according to claim 3, wherein the speech mode is the single-person speech mode; and the adjusting the shooting angle or the focal length of at least one of the first director camera and the second director camera based on the speech comprises: determining a target speaker from the effective speakers; and adjusting the shooting angle or the focal length of the second director camera, so that a face image of the target speaker is located at a target shooting location of the second director camera. ### Claim 8 (depends on 3) The method according to claim 3, wherein the speech mode is the double-person debate mode; and the adjusting the shooting angle or the focal length of at least one of the first director camera and the second director camera based on the speech mode comprises: if a distance between two effective speakers in the preset time period is less than a preset distance, adjusting the shooting angle or the focal length of the second director camera, so that face images corresponding to the two effective speakers are located at a target shooting location of the second director camera; or if a distance between two effective speakers is greater than or equal to a preset distance, adjusting the shooting angle or the focal length of at least one of the first director camera and the second director camera, so that a face image corresponding to one of the two effective speakers is located at a target shooting location of the first director camera and a face image corresponding to the other effective speaker is located at a target shooting location of the second director camera. ### Claim 9 (depends on 3) The method according to claim 3, wherein the speech mode is the multi-person discussion mode; and the adjusting the shooting angle or the focal length of at least one of the first director camera and the second director camera based on the speech mode comprises: if a distance between the at least two effective speakers in the preset time period is less than a preset distance, adjusting the shooting angle or the focal length of the second director camera, so that face images corresponding to the at least two effective speakers are located at a target shooting location of the second director camera; or if a distance between the at least two effective speakers is greater than or equal to a distance threshold, adjusting the shooting angle or the focal length of the second director camera, so that the second director camera shoots a panoramic video. ### Claim 10 (depends on 1) The method according to claim 1, wherein the determining speech information of a speaker based on first video information collected by the first camera and first audio information collected by the plurality of microphones MICs comprises: determining, based on video information and audio information that are corresponding to moments in a preset time period, speakers corresponding to the moments; and collecting statistics about the speakers corresponding to the moments to obtain the speech information of the speakers, wherein the speech information of the speakers comprises at least one piece of the following information: a quantity of speakers in the preset time period, speech duration of each speaker, a quantity of speech times of each speaker, speech content of each speaker, speech duration of each speech, a speech moment of each speech, or a priority of each speaker. ### Claim 11 A director device comprising a processor and interface, wherein the interface is configured to communicate with a camera, and the processor is configured to perform operations comprising: determine historical speech information of speaker in a preset time period based on first video information collected by a first camera in the preset time period and first audio information collected by a plurality of microphones (MICs) in the preset time period, wherein the preset time is in the past; determine a current speaker based on second video information currently collected by the first camera and second audio information currently collected by the plurality of MICs; and control a director status of director cameras based on the speech information of a speaker, the current speaker, and a speaker currently shot by at least one director camera, wherein the director status of a director camera comprises a shooting angle or a focal length, and when the current speaker is different from the speaker shot by the first director camera at the current moment, controlling a director status adjusts the shooting angle or the focal length of at least one of the first director camera and the second director camera based on the historical speech information, wherein the at least one director camera is configured to send a director video stream to another conference site. ### Claim 12 (depends on 11) The director device, according to claim 11, wherein the processor is further configured to: when the current speaker is the same as a speaker shot by the first director camera, keep a shooting angle and a focal length of the first director camera unchanged, wherein a pilot video currently shot by the first director camera is sent to the another conference site. ### Claim 13 (depends on 11) The director device according to claim 11, wherein the processor is configured to: when the current speaker is different from the speaker currently shot by the first director camera, determine a speech mode in the preset time period based on the speech information of the speaker, wherein the speech mode comprises at least one of a single-person speech mode, a double-person debate mode, or a multi-person discussion mode; and adjust the shooting angle or the focal length of at least one of the first director camera and the second director camera based on the speech mode in the preset time period. ### Claim 14 (depends on 13) The director device according to claim 13, wherein the processor is configured to: determine a quantity of effective speakers in the preset time period based on the speech information of the speaker; and determine the speech mode in the preset time period based on the quantity of effective speakers in the preset time period. ### Claim 15 (depends on 14) The director device according to claim 14, wherein the processor is configured to: determine a quantity of effective speech times of each speaker based on a priority of each speaker in the preset time period and speech duration of each speaker in each speech in the preset time period; determine a speaker whose quantity of effective speech times is greater than or equal to 1 as an effective speaker; and determine a quantity of effective speakers as the quantity of effective speakers. ### Claim 16 (depends on 14) The director device according to claim 14, wherein the processor is configured to: when the quantity of effective speakers is 1, determine that the speech mode in the preset time period is the single-person speech mode; when the quantity of effective speakers is 2, if the two effective speakers speak alternately, determine that the speech mode in the preset time period is the single-person speech mode or the double-person debate mode; or when the quantity of effective speakers is greater than 2, determine based on priorities of at least two effective speakers in the preset time period, that the speech mode in the preset time period is the single-person speech mode or the multi-person discussion mode. ### Claim 17 (depends on 13) The director device according to claim 13, wherein for the speech mode being the single-person speech mode, the processor is configured to: determining, by the director device, a target speaker from the effective speakers in the preset time period; and adjusting, by the director device, the shooting angle or the focal length of the second director camera, so that a face image of the target speaker is located at a target shooting location of the second director camera. ### Claim 18 (depends on 13) The director device according to claim 13, wherein for speech mode being the double-person debate mode, the processor is configured to: if a distance between two effective speakers in the preset time period is less than a preset distance, adjust the shooting angle or the focal length of the second director camera, so that face images corresponding to the two effective speakers are located at a target shooting location of the second director camera; or if a distance between two effective speakers in the preset time period is greater than or equal to a preset distance, adjust the shooting angle or the focal length of at least one of the first director camera and the second director camera, so that a face image corresponding to one of the two effective speakers is located at a target shooting location of the first director camera and a face image corresponding to the other effective speaker is located at a target shooting location of the second director camera. ### Claim 19 (depends on 13) The director device according to claim 13, wherein for the speech mode being the multi-person discussion mode, the processor is configured to: if a distance between the at least two effective speakers in the preset time period is less than a preset distance, adjust the shooting angle or the focal length of the second director camera, so that face images corresponding to the at least two effective speakers are located at a target shooting location of the second director camera; or if a distance between the at least two effective speakers in the preset time period is greater than or equal to a preset distance, adjust the shooting angle or the focal length of the second director camera, so that the second director camera shoots a panoramic video. ### Claim 20 (depends on 11) The director device according to claim 11, wherein the processor is configured to: determine based on video information and audio information that are corresponding to moments in the preset time period, speakers corresponding to the moments; and collect statistics about the speakers corresponding to the moments to obtain the speech information of the speakers, wherein the speech information of the speakers comprises at least one piece of the following information: a quantity of speakers in the preset time period, speech duration of each speaker, a quantity of speech times of each speaker, speech content of each speaker, speech duration of each speech, a speech moment of each speech, or a priority of each speaker.