# US 12,183,006 B2 · Target positioning method, apparatus and system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,006 B2 - Document ID: US12183006B2 - Status: Granted - Kind: B2 - Application number: 17/677,990 - Filed: 2022-02-22 - Published: 2022-06-09 - Granted: 2024-12-31 - Earliest claimed: 2020-02-08 - Assignees: Tencent Technology (Shenzhen) Company Limited - Applicants: Tencent Technology (Shenzhen) Company Limited - Inventors: Hong Shang; Zijian Zhang; Zhongqian Sun - Web page: https://patentgrep.com/patent/US12183006B2 ## CPC - G06T 7/0014: {using an image reference approach} - G06T 7/248: {involving reference images or patches} - G06T 2207/10016: Video; Image sequence - G06T 2207/10068: Endoscopic image - G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30096: Tumor; Lesion - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A target positioning method is provided to include: determining, in response to determining that a video frame image including a target lesion is detected from a video stream, location information of the target lesion on the video frame image; tracking the target lesion according to the location information of the target lesion on the video frame image, and determining location information of the target lesion on a video frame image in the video stream. ## Claims ### Claim 1 A target positioning method, comprising: determining, in response to determining that a video frame image including a target lesion is detected from a video stream, location information of the target lesion on the video frame image; and tracking the target lesion according to the location information of the target lesion on the video frame image, comprising: using a region image corresponding to the location information of the target lesion on the video frame image as a template region image; selecting a preset range region image centered on the location information from a video frame image in the video stream as a search region image; respectively mapping the search region image and the template region image to a set-dimensional feature space based on a convolutional neural network, to obtain search feature vectors corresponding to the search region image and template feature vectors corresponding to the template region image; performing a two-dimensional convolution operation on the search feature vectors and the template feature vectors to determine a second similarity value between each image block in the search region image and the template region image, each image block being obtained by sliding a sliding window on the search region image with a preset stride; using a two-dimensional matrix formed by arranging the second similarity values according to positions of corresponding image blocks on the search region image as a first similarity value between the search region image and the template region image; in response to determining that the first similarity value satisfies a similarity condition, determining that the target lesion is tracked down, and determining location information of the target lesion on the search region image. ### Claim 2 (depends on 1) The method according to claim 1, wherein a size of the sliding window is the same as that of the template region image. ### Claim 3 (depends on 1) The method according to claim 1, wherein determining that the target lesion is tracked down, and determining the location information of the target lesion comprises: determining that the target lesion is tracked down in response to determining that a second similarity value not less than a preset threshold exists in the two-dimensional matrix of the first similarity value; and determining location information of an image block corresponding to the largest second similarity value among the second similarity values as the location information of the target lesion on the search region image. ### Claim 4 (depends on 1) The method according to claim 1, further comprising: issuing an alert in a preset manner to indicate that the target lesion appears. ### Claim 5 (depends on 1) The method according to claim 1, wherein tracking the target lesion comprises: triggering a trained tracking model, tracking the target lesion based on the tracking model by using the location information of the target lesion on the video frame image as an input parameter, and determining the location information of the target lesion on the video frame image in the video stream, the tracking model being obtained through training according to a training image sample pair set, the training image sample pair set including a plurality of training image sample pairs with similarity value labels, the training image sample pair being constructed based on two video frame images selected from a video stream sample in which the target lesion appears. ### Claim 6 (depends on 5) The method according to claim 5, wherein tracking the target lesion comprises: using the location information of the target lesion on the video frame image as the input parameter, and respectively mapping the template region image and the search region image to the set-dimensional feature space by using two identical neural networks, to obtain the search feature vectors corresponding to the search region image and the template region image, a network structure of the tracking model including at least two identical neural networks, the neural network including at least a convolutional layer and a pooling layer; performing the two-dimensional convolution operation on the search feature vectors corresponding to the search region image and the template region image by using the convolutional layer to obtain the first similarity value between the search region image and the template region image; and in response to determining that the first similarity value satisfies the similarity condition, determining that the target lesion is tracked down, and determining location information of the target lesion on the search region image. ### Claim 7 (depends on 5) The method according to claim 5, further comprising: obtaining the training image sample pair set; and inputting the training image sample pairs to the tracking model for training, to obtain a trained tracking model. ### Claim 8 (depends on 7) The method according to claim 7, wherein obtaining the training image sample pair set comprises: obtaining a video stream sample set in which the target lesion appears, the video stream sample set including a plurality of video stream samples in which the target lesion appears; for each video frame image included in the video stream samples and by using the target lesion as a center, selecting a first preset range of template region image and a second preset range of search region image from the each video frame image, the second preset range being greater than the first preset range; selecting two video frame images from video frame images included in the video stream samples, and selecting a template region image of one video frame image and a search region image of the other video frame image among the two selected video frame images; and using the selected template region image and search region image as a training image sample pair, and generating a similarity value label of the training image sample pair, the similarity value label being the two-dimensional matrix. ### Claim 9 A target positioning apparatus, comprising: a memory storing computer program instructions; and a processor coupled to the memory and configured to execute the computer program instructions and perform: determining, in response to determining that a video frame image including a target lesion is detected from a video stream, location information of the target lesion on the video frame image; and tracking the target lesion according to the location information of the target lesion on the video frame image, comprising: using a region image corresponding to the location information of the target lesion on the video frame image as a template region image; selecting a preset range region image centered on the location information from a video frame image in the video stream as a search region image; respectively mapping the search region image and the template region image to a set-dimensional feature space based on a convolutional neural network, to obtain search feature vectors corresponding to the search region image and template feature vectors corresponding to the template region image; performing a two-dimensional convolution operation on the search feature vectors and the template feature vectors to determine a second similarity value between each image block in the search region image and the template region image, each image block being obtained by sliding a sliding window on the search region image with a preset stride; using a two-dimensional matrix formed by arranging the second similarity values according to positions of corresponding image blocks on the search region image as a first similarity value between the search region image and the template region image; in response to determining that the first similarity value satisfies a similarity condition, determining that the target lesion is tracked down, and determining location information of the target lesion on the search region image. ### Claim 10 (depends on 9) The target positioning apparatus according to claim 9, wherein a size of the sliding window is the same as that of the template region image. ### Claim 11 (depends on 9) The target positioning apparatus according to claim 9, wherein determining that the target lesion is tracked down, and determining the location information of the target lesion includes: determining that the target lesion is tracked down in response to determining that a second similarity value not less than a preset threshold exists in the two-dimensional matrix of the first similarity value; and determining location information of an image block corresponding to the largest second similarity value among the second similarity values as the location information of the target lesion on the search region image. ### Claim 12 (depends on 9) The target positioning apparatus according to claim 9, wherein the processor is further configured to execute the computer program instructions and perform: issuing an alert in a preset manner to indicate that the target lesion appears. ### Claim 13 (depends on 9) The target positioning apparatus according to claim 9, wherein tracking the target lesion includes: triggering a trained tracking model, tracking the target lesion based on the tracking model by using the location information of the target lesion on the video frame image as an input parameter, and determining the location information of the target lesion on the video frame image in the video stream, the tracking model being obtained through training according to a training image sample pair set, the training image sample pair set including a plurality of training image sample pairs with similarity value labels, the training image sample pair being constructed based on two video frame images selected from a video stream sample in which the target lesion appears. ### Claim 14 (depends on 13) The target positioning apparatus according to claim 13, wherein tracking the target lesion includes: using the location information of the target lesion on the video frame image as the input parameter, and respectively mapping the template region image and the search region image to the set-dimensional feature space by using two identical neural networks, to obtain the search feature vectors corresponding to the search region image and the template region image, a network structure of the tracking model including at least two identical neural networks, the neural network including at least a convolutional layer and a pooling layer; performing the two-dimensional convolution operation on the search feature vectors corresponding to the search region image and the template region image by using the convolutional layer to obtain the first similarity value between the search region image and the template region image; and in response to determining that the first similarity value satisfies the similarity condition, determining that the target lesion is tracked down, and determining location information of the target lesion on the search region image. ### Claim 15 (depends on 13) The target positioning apparatus according to claim 13, wherein the processor is further configured to execute the computer program instructions and perform: obtaining the training image sample pair set; and inputting the training image sample pairs to the tracking model for training, to obtain a trained tracking model. ### Claim 16 (depends on 15) The target positioning apparatus according to claim 15, wherein obtaining the training image sample pair set includes: obtaining a video stream sample set in which the target lesion appears, the video stream sample set including a plurality of video stream samples in which the target lesion appears; for each video frame image included in the video stream samples and by using the target lesion as a center, selecting a first preset range of template region image and a second preset range of search region image from the each video frame image, the second preset range being greater than the first preset range; selecting two video frame images from video frame images included in the video stream samples, and selecting a template region image of one video frame image and a search region image of the other video frame image among the two selected video frame images; and using the selected template region image and search region image as a training image sample pair, and generating a similarity value label of the training image sample pair, the similarity value label being the two-dimensional matrix. ### Claim 17 A non-transitory computer-readable storage medium storing computer program instructions executable by at least one processor to perform: determining, in response to determining that a video frame image including a target lesion is detected from a video stream, location information of the target lesion on the video frame image; and tracking the target lesion according to the location information of the target lesion on the video frame image, comprising: using a region image corresponding to the location information of the target lesion on the video frame image as a template region image; selecting a preset range region image centered on the location information from a video frame image in the video stream as a search region image; respectively mapping the search region image and the template region image to a set-dimensional feature space based on a convolutional neural network, to obtain search feature vectors corresponding to the search region image and template feature vectors corresponding to the template region image; performing a two-dimensional convolution operation on the search feature vectors and the template feature vectors to determine a second similarity value between each image block in the search region image and the template region image, each image block being obtained by sliding a sliding window on the search region image with a preset stride; using a two-dimensional matrix formed by arranging the second similarity values according to positions of corresponding image blocks on the search region image as a first similarity value between the search region image and the template region image; in response to determining that the first similarity value satisfies a similarity condition, determining that the target lesion is tracked down, and determining location information of the target lesion on the search region image.