# US 12,183,012 B2 · High-precision semi-automatic image data labeling method, electronic apparatus, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,012 B2 - Document ID: US12183012B2 - Status: Granted - Kind: B2 - Application number: 17/772,843 - Filed: 2020-07-06 - Published: 2022-12-29 - Granted: 2024-12-31 - Earliest claimed: 2019-11-15 - Assignees: Wuyi University - Applicants: Wuyi University - Inventors: Fuqin Deng; Yongshen Huang; Jianfeng Peng; Hua Feng; Yingying Chen; Weike Li - Web page: https://patentgrep.com/patent/US12183012B2 ## CPC - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 7/12: Edge-based segmentation - G06T 7/13: Edge detection - G06T 7/162: involving graph-based methods - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/20104: Interactive definition of region of interest [ROI] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Disclosed are a high-precision semi-automatic image data labeling method, an electronic apparatus and a non-transitory computer-readable storage medium. The high-precision semi-automatic image data labeling method may include: displaying a to-be-labeled image, the to-be-labeled image comprising a selected area and an unselected area; acquiring a coordinate point of the unselected area and a first range value; executing a grabcut algorithm based on the coordinate point of the unselected area and the first range value acquired, and obtaining a binarized image divided by the grabcut algorithm; executing an edge tracking algorithm on the binarized image to acquire current edge coordinates; updating a local coordinate set based on the current edge coordinates acquired; updating the selected area of the to-be-labeled image based on the local coordinate set acquired. ## Claims ### Claim 1 A high-precision semi-automatic image data labeling method, comprising: S100: displaying a to-be-labeled image, the to-be-labeled image comprising a selected area and an unselected area; S200: acquiring a coordinate point of the unselected area and a first range value; S300: executing a grabcut algorithm based on the coordinate point of the unselected area and the first range value acquired, and obtaining a binarized image divided by the grabcut algorithm; S400: executing an edge tracking algorithm on the binarized image to acquire current edge coordinates; S500: updating a local coordinate set based on the current edge coordinates acquired; S600: updating the selected area of the to-be-labeled image based on the local coordinate set acquired; S700: determining whether an all-selected instruction is received; in response to being yes, generating a whole coordinate set, and in response to being not, returning to S200, wherein the generating the whole coordinate set further comprises the following steps of: S710: establishing a set A′, adding any coordinate point p0 the local coordinate set to the set A′, establishing other coordinate points except p0 in the local coordinate set as a set A2, and setting a first labeled coordinate point p as p0; S720: determining whether a number of elements in the set A2 is zero; in response to being not, executing S720a, and in response to being yes, executing S720b; S720a: calculating distances d between all coordinate points in the set A2 and the first labeled coordinate point p, setting the first labeled coordinate point p as a coordinate point corresponding to the minimum distance d in the set A2, adding the coordinate point corresponding to the minimum distance d in the set A2 into the set A′ and deleting the coordinate point from the set A2, and returning to S720; S720b: sorting the coordinate points according to an order in which the coordinate points are added to the set A′; S730: establishing the whole coordinate set, adding p0 into the whole coordinate set and deleting p0 from the set A′, and setting a second labeled coordinate point p′ as p0; S740: determining whether a number of elements in the set A′ is 1; in response to being not, executing S740a, and in response to being yes, executing S740b; S740a: determining whether the second labeled coordinate point p′ is collinear with the two coordinate points ranked in the first two places in the set A′; in response to being yes, executing S740a1, and in response to being not, executing S740a2; S740a1: deleting the coordinate point in the first place in the set A′ from the set A′, and returning to S740; S740a2: setting the second labeled coordinate point p′ as the coordinate point ranked in the first place in the set A′, adding the coordinate point ranked in the first place in the set A′ to the whole coordinate set and deleting the coordinate point from the set A′, and returning to S740; and S740b: adding the coordinate points in the set A′ to the whole coordinate set and outputting the whole coordinate set; S800: determining whether an instruction indicating that an erroneously selected area occurs is received; in response to being yes, executing S900, and in response to being not, executing S1000; S900: acquiring a coordinate point of the erroneously selected area and a second range, and displaying a first square on the to-be-labeled image by the coordinate point of the erroneously selected area and the second range; S910: determining whether a coordinate in the whole coordinate set exists in the first square; in response to being not, returning to S900, and in response to being yes, executing S920; S920: acquiring a first coordinate point and a second coordinate point in the whole coordinate set that are duplicated with coordinates on edges of the first square, deleting the coordinate point in the whole coordinate set that are duplicated within a range of the first square, and adding the first coordinate point, the second coordinate point and a third coordinate point into the whole coordinate set, wherein the third coordinate point is a diagonal point among four diagonal points of the first square with a shortest path to a sum of the first and second coordinate points along the edges of the first square, and returning to S700; and S1000: acquiring category information and saving the category information and the whole coordinate set as json files. ### Claim 2 (depends on 1) The high-precision semi-automatic image data labeling method of claim 1, wherein the first range value is used for constructing a second square, and acquiring the first range value comprises the following steps of: S210: determining whether the selected area is vacant; in response to being yes, executing S300, and in response to being not, executing S220; and S220: determining whether a range of the second square constructed by the first range value contains a coordinate in the local coordinate set, and in response to being yes, executing S300. ### Claim 3 (depends on 1) The high-precision semi-automatic image data labeling method of claim 1, wherein the updating the selected area of the to-be-labeled image based on the local coordinate set acquired comprises the following of: highlighting or shadowing an area enclosed by coordinates in the local coordinate set. ### Claim 4 (depends on 1) The high-precision semi-automatic image data labeling method of claim 1, wherein the updating the local coordinate set based on the current edge coordinates acquired comprises the following steps of: S510: determining whether the selected area is vacant; in response to being yes, adding the current edge coordinates into the local coordinate set, and in response to being not, executing S520; and S520: deleting a coordinate in the current edge coordinates that is duplicated with a coordinate in the local coordinate set and then adding the remaining coordinates to the local coordinate set. ### Claim 5 An electronic apparatus, comprising a memory, a processor and a computer program stored in the memory and executable by the processor, wherein the computer program, when executed by the processor, causes the processor to perform the steps of a high-precision semi-automatic image data labeling method, comprising: S100: displaying a to-be-labeled image, the to-be-labeled image comprising a selected area and an unselected area; S200: acquiring a coordinate point of the unselected area and a first range value; S300: executing a grabcut algorithm based on the coordinate point of the unselected area and the first range value acquired, and obtaining a binarized image divided by the grabcut algorithm; S400: executing an edge tracking algorithm on the binarized image to acquire current edge coordinates; S500: updating a local coordinate set based on the current edge coordinates acquired; S600: updating the selected area of the to-be-labeled image based on the local coordinate set acquired; S700: determining whether an all-selected instruction is received; in response to being yes, generating a whole coordinate set, and in response to being not, returning to S200, wherein the generating the whole coordinate set further comprises the following steps of: S710: establishing a set A′, adding any coordinate point p0 in the local coordinate set to the set A′, establishing other coordinate points except p0 in the local coordinate set as a set A2, and setting a first labeled coordinate point p as p0; S720: determining whether a number of elements in the set A2 is zero; in response to being not, executing S720a, and in response to being yes, executing S720b; S720a: calculating distances d between all coordinate points in the set A2 and the first labeled coordinate point p, setting the first labeled coordinate point p as a coordinate point corresponding to the minimum distance d in the set A2, adding the coordinate point corresponding to the minimum distance d in the set A2 into the set A′ and deleting the coordinate point from the set A2, and returning to S720; S720b: sorting the coordinate points according to an order in which the coordinate points are added to the set A′; S730: establishing the whole coordinate set, adding p0 into the whole coordinate set and deleting p0 from the set A′, and setting a second labeled coordinate point p′ as p0; S740: determining whether a number of elements in the set A′ is 1; in response to being not, executing S740a, and in response to being yes, executing S740b; S740a: determining whether the second labeled coordinate point p′ is collinear with the two coordinate points ranked in the first two places in the set A′; in response to being yes, executing S740a1, and in response to being not, executing S740a2; S740a1: deleting the coordinate point in the first place in the set A′ from the set A′, and returning to S740; S740a2: setting the second labeled coordinate point p′ as the coordinate point ranked in the first place in the set A′, adding the coordinate point ranked in the first place in the set A′ to the whole coordinate set and deleting the coordinate point from the set A′, and returning to S740; and S740b: adding the coordinate points in the set A′ to the whole coordinate set and outputting the whole coordinate set; S800: determining whether an instruction indicating that an erroneously selected area occurs is received; in response to being yes, executing S900, and in response to being not, executing S1000; S900: acquiring a coordinate point of the erroneously selected area and a second range, and displaying a first square on the to-be-labeled image by the coordinate point of the erroneously selected area and the second range; S910: determining whether a coordinate in the whole coordinate set exists in the first square; in response to being not, returning to S900, and in response to being yes, executing S920; S920: acquiring a first coordinate point and a second coordinate point in the whole coordinate set that are duplicated with coordinates on edges of the first square, deleting the coordinate point in the whole coordinate set that are duplicated within a range of the first square, and adding the first coordinate point, the second coordinate point and a third coordinate point into the whole coordinate set, wherein the third coordinate point is a diagonal point among four diagonal points of the first square with a shortest path to a sum of the first and second coordinate points along the edges of the first square, and returning to S700; and S1000: acquiring category information and saving the category information and the whole coordinate set as json files. ### Claim 6 A non-transitory computer-readable storage medium, storing computer-executable instructions which, when executed by a processor, causes the processor to perform the steps of a high-precision semi-automatic image data labeling method, comprising: S100: displaying a to-be-labeled image, the to-be-labeled image comprising a selected area and an unselected area; S200: acquiring a coordinate point of the unselected area and a first range value; S300: executing a grabcut algorithm based on the coordinate point of the unselected area and the first range value acquired, and obtaining a binarized image divided by the grabcut algorithm; S400: executing an edge tracking algorithm on the binarized image to acquire current edge coordinates; S500: updating a local coordinate set based on the current edge coordinates acquired; S600: updating the selected area of the to-be-labeled image based on the local coordinate set acquired; S700: determining whether an all-selected instruction is received; in response to being yes, generating a whole coordinate set, and in response to being not, returning to S200, wherein the generating the whole coordinate set further comprises the following steps of: S710: establishing a set A′, adding any coordinate point p0 in the local coordinate set to the set A′, establishing other coordinate points except p0 in the local coordinate set as a set A2, and setting a first labeled coordinate point p as p0; S720: determining whether a number of elements in the set A2 is zero; in response to being not, executing S720a, and in response to being yes, executing S720b; S720a: calculating distances d between all coordinate points in the set A2 and the first labeled coordinate point p, setting the first labeled coordinate point p as a coordinate point corresponding to the minimum distance d in the set A2, adding the coordinate point corresponding to the minimum distance d in the set A2 into the set A′ and deleting the coordinate point from the set A2, and returning to S720; S720b: sorting the coordinate points according to an order in which the coordinate points are added to the set A′; S730: establishing the whole coordinate set, adding p0 into the whole coordinate set and deleting p0 from the set A′, and setting a second labeled coordinate point p′ as p0; S740: determining whether a number of elements in the set A′ is 1; in response to being not, executing S740a, and in response to being yes, executing S740b; S740a: determining whether the second labeled coordinate point p′ is collinear with the two coordinate points ranked in the first two places in the set A′; in response to being yes, executing S740a1, and in response to being not, executing S740a2; S740a1: deleting the coordinate point in the first place in the set A′ from the set A′, and returning to S740; S740a2: setting the second labeled coordinate point p′ as the coordinate point ranked in the first place in the set A′, adding the coordinate point ranked in the first place in the set A′ to the whole coordinate set and deleting the coordinate point from the set A′, and returning to S740; and S740b: adding the coordinate points in the set A′ to the whole coordinate set and outputting the whole coordinate set; S800: determining whether an instruction indicating that an erroneously selected area occurs is received; in response to being yes, executing S900, and in response to being not, executing S1000; S900: acquiring a coordinate point of the erroneously selected area and a second range, and displaying a first square on the to-be-labeled image by the coordinate point of the erroneously selected area and the second range; S910: determining whether a coordinate in the whole coordinate set exists in the first square; in response to being not, returning to S900, and in response to being yes, executing S920; S920: acquiring a first coordinate point and a second coordinate point in the whole coordinate set that are duplicated with coordinates on edges of the first square deleting the coordinate point in the whole coordinate set that are duplicated within a range of the first square, and adding the first coordinate point, the second coordinate point and a third coordinate point into the whole coordinate set, wherein the third coordinate point is a diagonal point among four diagonal points of the first square with a shortest path to a sum of the first and second coordinate points along the edges of the first square, and returning to S700; and S1000: acquiring category information and saving the category information and the whole coordinate set as json files.