# US 12,112,516 B2 · Non-intrusive detection method and device for pop-up window button > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,516 B2 - Document ID: US12112516B2 - Status: Granted - Kind: B2 - Application number: 17/498,566 - Filed: 2021-10-11 - Published: 2022-12-29 - Granted: 2024-10-08 - Earliest claimed: 2021-06-25 - Assignees: Adlink Technology Inc. - Applicants: ADLINK Technology Inc. - Inventors: Chien-Chung Lin; Wei-Jyun Tu; Yu-Yen Chen - Web page: https://patentgrep.com/patent/US12112516B2 ## CPC - G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components - G06T 5/30: Erosion or dilatation, e.g. thinning - G06T 7/13: Edge detection - G06T 7/62: of area, perimeter, diameter or volume - G06T 7/73: using feature-based methods - G06V 10/28: Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns - G06V 10/471: {using approximation functions} - G06V 10/752: {Contour matching} - G06T 2207/10016: Video; Image sequence - G06T 2207/20044: Skeletonization; Medial axis transform - G06T 2207/20164: Salient point detection; Corner detection - G06T 2207/20224: Image subtraction - G06V 30/10: Character recognition - G06V 2201/02: Recognising information on displays, dials, clocks - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A non-intrusive detection method for detecting at least one pop-up window button of the pop-up window includes the following steps: retrieving a screen image on a display device; comparing the screen image with a preset screen image and generating a differential image area according the screen image and the preset screen image; determining the differential image area as the pop-up window when the differential image area is greater than an image area threshold value; selecting a plurality of contour lengths of the pop-up window matching up with a contour length threshold value by Canny edge detector; and analyzing the contour lengths according to Douglas-Peucker algorithm and an amount of endpoints to generate a contour edge corresponding to the pop-up window button. ## Claims ### Claim 1 A non-intrusive detection method for pop-up window button configured for recognizing at least one pop-up window button of a pop-up window on a display device, the non-intrusive detection method comprising the following steps of: S1: retrieving a screen image of the display device; S2: comparing the screen image with a preset screen image, and generating a differential image area according the screen image and the preset screen image; S3: determining whether the differential image area is greater than an image area threshold value; S4: determining the differential image area as the pop-up window when the differential image area is greater than the image area threshold value; S5: presetting a contour length threshold value of the pop-up window, and selecting a plurality of contour lengths greater than the contour length threshold value by Canny edge detector; and S6: presetting an amount of endpoints of the at least one pop-up window button, and interpreting the plurality of contour lengths according to Douglas-Peucker algorithm and the amount of endpoints to generate a contour edge corresponding to the at least one pop-up window button. ### Claim 2 (depends on 1) The method of claim 1, further comprising the following steps of: S11: determining whether the preset screen image exists, and executing step S2 if the determining result is yes; and S12: replacing the preset screen image with the screen image if the determining result of step S11 is no. ### Claim 3 (depends on 1) The method of claim 1, wherein step S2 further comprising the following steps of: S21: subtracting the screen image from the preset screen image to generate a differential image; and S22: calculating the differential image with binarization and image erosion methods to generate the differential image area. ### Claim 4 (depends on 1) The method of claim 1, wherein step S6 further comprising the following steps of: S61: determining the plurality of contour lengths according to the Douglas-Peucker algorithm and the amount of endpoints to generate a button contour similar to the at least one pop-up window button; and S62: generating the contour edge corresponding to the at least one pop-up window button according to the button contour and a contour correction accuracy, wherein the contour correction accuracy is between 1% and 10% of the contour length. ### Claim 5 (depends on 1) The method of claim 1, further comprising the following step of: S7: analyzing the at least one pop-up window button of the pop-up window by optical character recognition and generating a button text of the at least one pop-up window button. ### Claim 6 (depends on 1) The method of claim 1, wherein the minimum value of the amount of endpoints is 4. ### Claim 7 A non-intrusive detection device for pop-up window button configured for recognizing at least one pop-up window button of a pop-up window on a display device, the non-intrusive detection device comprising: an image retrieving module, configured to retrieve a screen image of the display device; an image analyzing module, connected to the image retrieving module and stored a preset screen image and an image area threshold value, wherein the image analyzing module is configured to compare the screen image with the preset screen image and generate a differential image area according the screen image and the preset screen image, the image analyzing module determines the differential image area as the pop-up window when the differential image area is greater than the image area threshold value; and a computing module, connected to the image analyzing module and stored a contour length threshold value and an amount of endpoints, wherein the computing module is configured to select a plurality of contour lengths greater than the contour length threshold value by Canny edge detector, and analyze the contour lengths according to Douglas-Peucker algorithm and the amount of endpoints to generate a contour edge corresponding to the at least one pop-up window button. ### Claim 8 (depends on 7) The device of claim 7, wherein the image analyzing module is configured to replace the preset screen image with the screen image when the preset screen image does not exist. ### Claim 9 (depends on 7) The device of claim 7, wherein the image analyzing module subtracts the screen image from the preset screen image to generate a differential image, and calculates the differential image with binarization and image erosion methods to generate the differential image area. ### Claim 10 (depends on 7) The device of the claim 7, wherein the computing module pre-stores a contour correction accuracy, the computing module generates a button contour similar to the at least one pop-up window button according to Douglas-Peucker algorithm and the amount of endpoints, and generates a contour edge corresponding to the at least one pop-up window button according to the button contour and the contour correction accuracy, wherein the contour correction accuracy is between 1% and 10% of the contour length. ### Claim 11 (depends on 7) The device of claim 7, further comprising an optical character recognizing module connected to the computing module, the optical character recognizing module is configured to analyze the at least one pop-up window button of the pop-up window and generates a button text of the at least one pop-up window button. ### Claim 12 (depends on 7) The device of claim 7, wherein the minimum value of the amount of endpoints is 4.