# US 12,114,075 B1 · Object selection in computer vision > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,114,075 B1 - Document ID: US12114075B1 - Status: Granted - Kind: B1 - Application number: 18/218,199 - Filed: 2023-07-05 - Granted: 2024-10-08 - Earliest claimed: 2020-09-22 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Yu Shu; Yahia Chafik Chehadeh; Heliben Dhrumil Desai; Young Sun Kim - Web page: https://patentgrep.com/patent/US12114075B1 ## CPC - H04N 7/142: {Constructional details of the terminal equipment, e.g. arrangements of the camera and the display} - H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking - G06V 10/82: using neural networks - G06V 20/44: {Event detection} - G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames} - G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands - G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition} - H04N 7/141: {between two video terminals, e.g. videophone (telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085)} - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - H04N 23/611: where the recognised objects include parts of the human body - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 2015/223: {Execution procedure of a spoken command} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Devices and techniques are generally described for selection of an object to follow during a video call. In various examples, a first frame of image data may be received from a camera. First image data representing a first person and second image data representing a second person may be determined. A first priority level may be determined for the first person at a first time. A second priority level for the second person may be determined at the first time. The camera may be controlled to follow the first person based at least in part on the first priority level and the second priority level. ## Claims ### Claim 1 A computer-implemented method, comprising: detecting a first person represented in a first video data captured by a camera of a first device; determining a first level of engagement of the first person with the camera at a first time; determining a first ratio proportional to the first level of engagement of the first person with the camera; determining a first weight value for the first person at a second time based at least in part by increasing an initial weight value for the first person at a first rate, wherein the first rate is determined based at least in part on the first ratio; and controlling the first device to track the first person within frames of image data generated by the first device based at least in part on a threshold weight value. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, further comprising: determining a second level of engagement of the first person with the camera at a third time following the first time and the second time, wherein the second level of engagement represents increased engagement relative to the first level of engagement; determining a second ratio proportional to the second level of engagement of the first person with the camera; and determining a second weight value for the person at a fourth time following the third time based at least in part by increasing the first weight value at a second rate, wherein the second rate is determined based at least in part on the second ratio. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, further comprising: determining a first priority level for the first person at the second time based at least in part on a comparison of the first weight value to the threshold weight value; and determining, based on the first priority level, that the first person is not to be tracked by the first device. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, further comprising: determining a second priority level for the first person at the fourth time based at least in part on a comparison of the second weight value to the threshold weight value, wherein the controlling the first device to track the first person within frames of image data generated by the first device is based at least in part on the comparison of the second weight value to the threshold weight value. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, further comprising: determining the initial weight value for the first person, wherein the initial weight value is specific to an engagement property; determining a weight-increase rate associated with the engagement property; and determining the first rate by multiplying the first ratio by the weight-increase rate. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising determining the first level of engagement based at least in part on a gaze direction of the first person detected in the first video data. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, further comprising determining the first level of engagement based at least in part on a determination that the first person is speaking in the first video data. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, further comprising determining the first level of engagement based at least in part on a determination that the first person is facing toward the camera of the first device. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, further comprising: determining a second level of engagement of the first person with the camera at a third time following the first time and the second time, wherein the second level of engagement represents decreased engagement relative to the first level of engagement; determining a second ratio proportional to the second level of engagement of the first person with the camera; and determining a second weight value for the person at a fourth time following the third time based at least in part by decreasing the first weight value at a second rate, wherein the second rate is determined based at least in part on the second ratio. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, further comprising: controlling the first device to stop tracking the first person at the fourth time based at least in part on the second weight value and the threshold weight value. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, further comprising: detecting a second person represented in the first video data; determining a second level of engagement of the second person with the camera at the first time; determining a second ratio proportional to the second level of engagement of the second person with the camera; determining a second weight value for the second person at the second time based at least in part by increasing an initial weight value for the second person at a second rate, wherein the second rate is determined based at least in part on the second ratio; and arbitrating between tracking the first person and the second person based at least in part on the threshold weight value, the first weight value, and the second weight value. ### Claim 12 A system comprising: at least one processor; a display; at least one camera; a microphone; and non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to: detect a first person represented in a first video data; determine a first level of engagement of the first person with the at least one camera at a first time; determine a first ratio proportional to the first level of engagement of the first person with the camera; determine a first weight value for the first person at a second time based at least in part by increasing an initial weight value for the first person at a first rate, wherein the first rate is determined based at least in part on the first ratio; and control the at least one camera to track the first person within frames of image data generated by the first device based at least in part on a threshold weight value. ### Claim 13 (depends on 12) The system of claim 12, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine a second level of engagement of the first person with the camera at a third time following the first time and the second time, wherein the second level of engagement represents increased engagement relative to the first level of engagement; determine a second ratio proportional to the second level of engagement of the first person with the camera; and determine a second weight value for the person at a fourth time following the third time based at least in part by increasing the first weight value at a second rate, wherein the second rate is determined based at least in part on the second ratio. ### Claim 14 (depends on 13) The system of claim 13, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine a first priority level for the first person at the second time based at least in part on a comparison of the first weight value to the threshold weight value; and determine, based on the first priority level, that the first person is not to be tracked by the at least one camera. ### Claim 15 (depends on 14) The system of claim 14, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine a second priority level for the first person at the fourth time based at least in part on a comparison of the second weight value to the threshold weight value, wherein the controlling the at least one camera to track the first person within frames of image data generated by the at least one camera is based at least in part on the comparison of the second weight value to the threshold weight value. ### Claim 16 (depends on 12) The system of claim 12, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine the initial weight value for the first person, wherein the initial weight value is specific to an engagement property; determine a weight-increase rate associated with the engagement property; and determine the first rate by multiplying the first ratio by the weight-increase rate. ### Claim 17 (depends on 12) The system of claim 12, the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to determine the first level of engagement based at least in part on a gaze direction of the first person detected in the first video data. ### Claim 18 A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, are effective to perform a method comprising: detecting a first person represented in a first video data; determining a first level of engagement of the first person with a camera at a first time; determining a first ratio proportional to the first level of engagement of the first person with the camera; determining a first weight value for the first person at a second time based at least in part by increasing an initial weight value for the first person at a first rate, wherein the first rate is determined based at least in part on the first ratio; and controlling the camera to track the first person within frames of image data generated by the camera based at least in part on a threshold weight value. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, storing further instructions that, when executed by the at least one processor, are effective to perform the method further comprising: determining a second level of engagement of the first person with the camera at a third time following the first time and the second time, wherein the second level of engagement represents increased engagement relative to the first level of engagement; determining a second ratio proportional to the second level of engagement of the first person with the camera; and determining a second weight value for the person at a fourth time following the third time based at least in part by increasing the first weight value at a second rate, wherein the second rate is determined based at least in part on the second ratio. ### Claim 20 (depends on 18) The non-transitory computer-readable storage medium of claim 18, storing further instructions that, when executed by the at least one processor, are effective to perform the method further comprising: determining a first priority level for the first person at the second time based at least in part on a comparison of the first weight value to the threshold weight value; and determining, based on the first priority level, that the first person is not to be tracked by the camera.