# US 12,223,574 B2 · Automatic area detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,574 B2 - Document ID: US12223574B2 - Status: Granted - Kind: B2 - Application number: 18/399,334 - Filed: 2023-12-28 - Published: 2024-06-27 - Granted: 2025-02-11 - Earliest claimed: 2021-10-12 - Assignees: Procore Technologies, Inc. - Applicants: Procore Technologies, Inc. - Inventors: Mohammad Soltani; Farid Mirahadi; Azadeh Yazdan Panah Gohar Rizi; Fiona Liu - Web page: https://patentgrep.com/patent/US12223574B2 ## CPC - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - G06T 7/11: Region-based segmentation - G06T 7/12: Edge-based segmentation - G06T 9/002: {using neural networks} - G06T 11/23: using straight lines or curves - G06T 11/60: Creating or editing images; Combining images with text - G06T 9/20: Contour coding, e.g. using detection of edges - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30176: Document - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An example computing platform is configured to (i) receive a two-dimensional (2D) image file comprising a construction drawing, (ii) generate, via semantic segmentation, a first set of polygons corresponding to respective areas of the 2D image file, (iii) generate, via instance segmentation, a second set of polygons corresponding to respective areas of the 2D image file, (iv) generate, via unsupervised image processing, a third set of polygons corresponding to respective areas of the 2D image file, (v) based on (a) overlap between polygons in the first, second, and third sets of polygons and (b) respective confidence scores for each of the overlapping polygons, determine a set of merged polygons corresponding to respective areas of the 2D image file, and (vi) cause a client station to display a visual representation of the 2D image file where each merged polygon is overlaid as a respective selectable region of the 2D image file. ## Claims ### Claim 1 A computing platform comprising: a network interface; at least one processor; at least one non-transitory computer-readable medium; and program instructions stored on the at least one non-transitory computer-readable medium that are executable by the at least one processor such that the computing platform is configured to: receive a two-dimensional (2D) image file comprising a construction drawing; generate, using a first supervised image processing model, (i) a first set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the first set of polygons; generate, using a second supervised image processing model, (i) a second set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the second set of polygons; generate, using an unsupervised image processing model, (i) a third set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the third set of polygons; for each polygon from each set of polygons, determine a respective overlap with each polygon from the other sets of polygons; based on (i) the determined overlap between polygons and (ii) the respective confidence scores for each of the overlapping polygons, determine a set of merged polygons corresponding to respective areas of the 2D image file; and based on (i) the 2D image file and (ii) the set of merged polygons corresponding to the respective areas of the 2D image file, train one or both of the first supervised image processing model and the second supervised image processing model. ### Claim 2 (depends on 1) The computing platform of claim 1, further comprising program instructions stored on the at least one non-transitory computer-readable medium that are executable by the at least one processor such that the computing platform is configured to: cause a client station to display a visual representation of the 2D image file, wherein each merged polygon in the set of merged polygons is overlaid as a respective selectable region of the 2D image file. ### Claim 3 (depends on 1) The computing platform of claim 1, wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to train one or both of the first supervised image processing model and the second supervised image processing model comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: determine training data for training one or both of the first supervised image processing model and the second supervised image processing model by annotating and splitting the 2D image file and the set of merged polygons into training and validation data sets. ### Claim 4 (depends on 3) The computing platform of claim 3, wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to train one or both of the first supervised image processing model and the second supervised image processing model comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: merge the training and validation data sets with existing training and validation data sets to generate merged training data. ### Claim 5 (depends on 1) The computing platform of claim 1, further comprising program instructions stored on the at least one non-transitory computer-readable medium that are executable by the at least one processor such that the computing platform is configured to: cause a client station to display a visual representation of the 2D image file, wherein each merged polygon in the set of merged polygons is overlaid as a respective selectable region of the 2D image file; and determine a quantity estimation for a construction project associated with the construction drawing based on at least one selectable region of the 2D image file. ### Claim 6 (depends on 5) The computing platform of claim 5, wherein the quantity estimation is a material quantity estimation based on at least one selectable region of the 2D image file. ### Claim 7 (depends on 5) The computing platform of claim 5, wherein the quantity estimation is a labor quantity estimation based on at least one selectable region of the 2D image file. ### Claim 8 (depends on 1) The computing platform of claim 1, further comprising program instructions stored on the at least one non-transitory computer-readable medium that are executable by the at least one processor such that the computing platform is configured to: determine that the 2D image file comprises a vector image; extract vector metadata from the 2D image file, wherein the vector metadata comprises an attribute list; and generate a raster image of the 2D image file, by using the first supervised image processing model, the second supervised image processing model, and the unsupervised image processing model. ### Claim 9 (depends on 8) The computing platform of claim 8, wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to generate, using an unsupervised image processing model, the third set of polygons comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: generate, using the unsupervised image processing model, the third set of polygons based on the attribute list. ### Claim 10 (depends on 1) The computing platform of claim 1, further comprising program instructions stored on the at least one non-transitory computer-readable medium that are executable by the at least one processor such that the computing platform is configured to: for each determined overlap between two polygons that exceeds a coverage threshold, assign the two polygons to a respective polygon group, wherein each polygon group comprises up to one polygon from each of the first, second, and third sets of polygons; and for each polygon group, determine a combined confidence score for the polygon group based on a respective individual confidence score of each polygon in the polygon group. ### Claim 11 (depends on 10) The computing platform of claim 10, wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to, for each polygon group, determine the combined confidence score for the polygon group comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: apply a respective weight variable to each polygon in the polygon group, wherein the respective weight variable is applied based on whether the polygon was generated by using the first supervised image processing model, using the second supervised image processing model, or using the unsupervised image processing. ### Claim 12 (depends on 10) The computing platform of claim 10, wherein the program instructions that are executable by the at least one processor such that the computing platform is configured to determine the set of merged polygons comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: determine that the combined confidence score from a given polygon group exceeds a confidence threshold; and based on the polygons within the given polygon group, determine a merged polygon to be added to the set of merged polygons. ### Claim 13 A non-transitory computer-readable medium, wherein the non-transitory computer-readable medium is provisioned with program instructions that, when executed by at least one processor, cause a computing platform to: receive a two-dimensional (2D) image file comprising a construction drawing; generate, using a first supervised image processing model, (i) a first set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the first set of polygons; generate, using a second supervised image processing model, (i) a second set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the second set of polygons; generate, using an unsupervised image processing model, (i) a third set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the third set of polygons; for each polygon from each set of polygons, determine a respective overlap with each polygon from the other sets of polygons; based on (i) the determined overlap between polygons and (ii) the respective confidence scores for each of the overlapping polygons, determine a set of merged polygons corresponding to respective areas of the 2D image file; and based on (i) the 2D image file and (ii) the set of merged polygons corresponding to respective areas of the 2D image file, train one or both of the first supervised image processing model and the second supervised image processing model. ### Claim 14 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the program instructions that, when executed by the at least one processor, cause the computing platform to train one or both of the first supervised image processing model and the second supervised image processing model comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: determine training data for training one or both of the first supervised image processing model and the second supervised image processing model by annotating and splitting the 2D image file and the set of merged polygons into training and validation data sets. ### Claim 15 (depends on 14) The non-transitory computer-readable medium of claim 14, wherein the program instructions that, when executed by the at least one processor, cause the computing platform to train one or both of the first supervised image processing model and the second supervised image processing model comprise program instructions that are executable by the at least one processor such that the computing platform is configured to: merge the training and validation data sets with existing training and validation data sets to generate merged training data. ### Claim 16 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the non-transitory computer-readable medium is also provisioned with program instructions that, when executed by at least one processor, cause the computing platform to: cause a client station to display a visual representation of the 2D image file, wherein each merged polygon in the set of merged polygons is overlaid as a respective selectable region of the 2D image file; and determine a quantity estimation for a construction project associated with the construction drawing based on at least one selectable region of the 2D image file. ### Claim 17 A method carried out by a computing platform, the method comprising: receiving a two-dimensional (2D) image file comprising a construction drawing; generating, using a first supervised image processing model, (i) a first set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the first set of polygons; generating, using a second supervised image processing model, (i) a second set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the second set of polygons; generating, using an unsupervised image processing model, (i) a third set of polygons corresponding to respective areas of the 2D image file and (ii) a respective confidence score for each polygon in the third set of polygons; for each polygon from each set of polygons, determining a respective overlap with each polygon from the other sets of polygons; based on (i) the determined overlap between polygons and (ii) the respective confidence scores for each of the overlapping polygons, determining a set of merged polygons corresponding to respective areas of the 2D image file; and based on (i) the 2D image file and (ii) the set of merged polygons corresponding to the respective areas of the 2D image file, train one or both of the first supervised image processing model and the second supervised image processing model. ### Claim 18 (depends on 17) The method of claim 17, wherein training one or both of the first supervised image processing model and the second supervised image processing model comprises determining training data for training one or both of the first supervised image processing model and the second supervised image processing model by annotating and splitting the 2D image file and the set of merged polygons into training and validation data sets. ### Claim 19 (depends on 17) The method of claim 17, wherein training one or both of the first supervised image processing model and the second supervised image processing model comprises merging the training and validation data sets with existing training and validation data sets to generate merged training data. ### Claim 20 (depends on 17) The method of claim 17, further comprising: causing a client station to display a visual representation of the 2D image file, wherein each merged polygon in the set of merged polygons is overlaid as a respective selectable region of the 2D image file; and determining a quantity estimation for a construction project associated with the construction drawing based on at least one selectable region of the 2D image file.