# US 12,182,721 B2 · Deep learning-based anomaly detection in images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,721 B2 - Document ID: US12182721B2 - Status: Granted - Kind: B2 - Application number: 17/913,905 - Filed: 2021-03-25 - Published: 2023-09-07 - Granted: 2024-12-31 - Earliest claimed: 2020-03-25 - Assignees: Yissum Research Development Company of the Hebrew University of Jerusalem - Applicants: Yissum Research Development Company of the Hebrew University of Jerusalem Ltd. - Inventors: Yedid Hoshen; Liron Bergman; Niv Cohen; Tal Reiss - Web page: https://patentgrep.com/patent/US12182721B2 ## CPC - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06F 18/24147: {Distances to closest patterns, e.g. nearest neighbour classification} - G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/096: Transfer learning - G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components - G06V 10/761: {Proximity, similarity or dissimilarity measures} - G06V 10/762: using clustering, e.g. of similar faces in social networks - G06V 10/82: using neural networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method comprising: receiving, as input, training images, wherein at least a majority of the training images represent normal data instances; receiving, as input, a target image; extracting (i) a set of feature representations from a plurality of image locations within each of the training images, and (ii) target feature representations from a plurality of target image locations within the target image; calculating, with respect to a target image location of the plurality of target image locations in the target image, a distance between (iii) the target feature representation of the target image location, and (iv) a subset from the set of feature representations comprising the k nearest the feature representations to the target feature representation; and determining that the target image location is anomalous, when the calculated distance exceeds a predetermined threshold. ## Claims ### Claim 1 A system comprising: at least one hardware processor; and a non-transitory computer-readable storage medium having stored thereon program instruction, the program instructions executable by the at least one hardware processor to: receive, as input, training images, wherein at least a majority of said training images represent normal data instances, receive, as input, a target image, extract (i) a set of feature representations from a plurality of sub-image locations within each of said training images, and (ii) target feature representations from a plurality of target sub-image locations within said target image, calculate, with respect to a target sub-image location of said plurality of target sub-image locations in said target image, a distance between (iii) said target feature representation of said target sub-image location, and (iv) a subset from said set of feature representations comprising the k nearest said feature representations to said target feature representation, and determine that said target sub-image location is anomalous, when said calculated distance exceeds a predetermined threshold. ### Claim 2 (depends on 1) The system of claim 1, wherein said program instructions are further executable to perform said calculating and said determining with respect to all of said plurality of target sub-image locations. ### Claim 3 (depends on 2) The system of claim 2, wherein said program instructions are further executable to designate a segment of said target image as comprising anomalous target sub-image locations, based, at least in part, on said determining. ### Claim 4 (depends on 1) The system of claim 1, wherein said program instructions are further executable to apply a clustering algorithm to said set of feature representations, to obtain clusters of said feature representations, wherein said calculating comprises calculating, with respect to a target sub-image location of said plurality of target sub-image locations, a distance between (i) said target feature representation of said target sub-image location, and (ii) the k nearest means of said clusters to said target feature representation. ### Claim 5 (depends on 1) The system of claim 1, wherein said extracting is performed by applying a pre-trained machine learning model to said training images and said target image, wherein said pre-trained machine learning model comprises a deep-learning neural network architecture comprising a feature pyramid having a plurality of layers, wherein each of said plurality of layers has a specified resolution, and wherein said extracting comprises concatenating features from two or more of said plurality of layers. ### Claim 6 (depends on 1) The system of claim 1, wherein said calculating comprises: (i) selecting, from said training images, a specified number n of nearest images to said target image; and (ii) calculating, with respect to a target sub-image location of said plurality of target sub-image locations in said target image, a distance between (a) said target feature representation of said target sub-image location, and (b) said feature representations from all of said sub-image locations in said n nearest images; and (iii) determining that said target sub-image location is anomalous, when said calculated distance exceeds a predetermined threshold. ### Claim 7 (depends on 1) The system of claim 1, wherein each of said sub-image locations represents one pixel in (i) each of said training images, and (ii) said target image. ### Claim 8 A computer-implemented method comprising: receiving, as input, training images, wherein at least a majority of said training images represent normal data instances; receiving, as input, a target image; extracting (i) a set of feature representations from a plurality of sub-image locations within each of said training images, and (ii) target feature representations from a plurality of target sub-image locations within said target image; calculating, with respect to a target sub-image location of said plurality of target sub-image locations in said target image, a distance between (iii) said target feature representation of said target sub-image location, and (iv) a subset from said set of feature representations comprising the k nearest said feature representations to said target feature representation; and determining that said target sub-image location is anomalous, when said calculated distance exceeds a predetermined threshold. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, further comprising performing said calculating and said determining with respect to all of said plurality of target sub-image locations. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, further comprising designating a segment of said target image as comprising anomalous target sub-image locations, based, at least in part, on said determining. ### Claim 11 (depends on 8) The computer-implemented method of claim 8, further comprising applying a clustering algorithm to said set of feature representations, to obtain clusters of said feature representations, wherein said calculating comprises calculating, with respect to a target sub-image location of said plurality of target sub-image locations, a distance between (i) said target feature representation of said target sub-image location, and (ii) the k nearest means of said clusters to said target feature representation. ### Claim 12 (depends on 8) The computer-implemented method of claim 8, wherein said extracting is performed by applying a pre-trained machine learning model to said training images and said target image, wherein said pre-trained machine learning model comprises a deep-learning neural network architecture comprising a feature pyramid having a plurality of layers, wherein each of said plurality of layers has a specified resolution, and wherein said extracting comprises concatenating features from two or more of said plurality of layers. ### Claim 13 (depends on 8) The computer-implemented method of claim 8, wherein said calculating comprises: (i) selecting, from said training images, a specified number n of nearest images to said target image; and (ii) calculating, with respect to a target sub-image location of said plurality of target sub-image locations in said target image, a distance between (a) said target feature representation of said target sub-image location, and (b) said feature representations from all of said sub-image locations in said n nearest images; and (iii) determining that said target sub-image location is anomalous, when said calculated distance exceeds a predetermined threshold. ### Claim 14 (depends on 8) The computer-implemented method of claim 8, wherein each of said sub-image locations represents one pixel in (i) each of said training images, and (ii) said target image. ### Claim 15 A computer program product comprising a non-transitory computer-readable storage medium having program instructions embodied therewith, the program instructions executable by at least one hardware processor to comprising: receive, as input, training images, wherein at least a majority of said training images represent normal data instances; receive, as input, a target image; extract (i) a set of feature representations from a plurality of sub-image locations within each of said training images, and (ii) target feature representations from a plurality of target sub-image locations within said target image; calculate, with respect to a target sub-image location of said plurality of target sub-image locations in said target image, a distance between (iii) said target feature representation of said target sub-image location, and (iv) a subset from said set of feature representations comprising the k nearest said feature representations to said target feature representation; and determine that said target sub-image location is anomalous, when said calculated distance exceeds a predetermined threshold. ### Claim 16 (depends on 15) The computer program product of claim 15, wherein said program instructions are further executable to perform said calculating and said determining with respect to all of said plurality of target sub-image locations. ### Claim 17 (depends on 16) The computer program product of claim 16, wherein said program instructions are further executable to designate a segment of said target sub-image as comprising anomalous target image locations, based, at least in part, on said determining. ### Claim 18 (depends on 15) The computer program product of claim 15, wherein said program instructions are further executable to apply a clustering algorithm to said set of feature representations, to obtain clusters of said feature representations, wherein said calculating comprises calculating, with respect to a target sub-image location of said plurality of target sub-image locations, a distance between (i) said target feature representation of said target sub-image location, and (ii) the k nearest means of said clusters to said target feature representation. ### Claim 19 (depends on 15) The computer program product of claim 15, wherein said extracting is performed by applying a pre-trained machine learning model to said training images and said target image, wherein said pre-trained machine learning model comprises a deep-learning neural network architecture comprising a feature pyramid having a plurality of layers, wherein each of said plurality of layers has a specified resolution, and wherein said extracting comprises concatenating features from two or more of said plurality of layers. ### Claim 20 (depends on 15) The computer program product of claim 15, wherein said calculating comprises: (i) selecting, from said training images, a specified number n of nearest images to said target image; and (ii) calculating, with respect to a target sub-image location of said plurality of target sub-image locations in said target image, a distance between (a) said target feature representation of said target sub-image location, and (b) said feature representations from all of said sub-image locations in said n nearest images; and (iii) determining that said target sub-image location is anomalous, when said calculated distance exceeds a predetermined threshold.