# US 2019/0244362 A1 · Differentiable Jaccard Loss Approximation for Training an Artificial Neural Network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0244362 A1 - Document ID: US20190244362A1 - Status: Published application - Kind: A1 - Application number: 15/889,440 - Filed: 2018-02-06 - Published: 2019-08-08 - Earliest claimed: 2018-02-06 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Yair Movshovitz-Attias; Elad Edwin Tzvi Eban - Web page: https://patentgrep.com/patent/US20190244362A1 ## CPC - G06T 7/136: involving thresholding - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/048: Activation functions - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06T 7/11: Region-based segmentation - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Systems and methods described herein may relate to training an artificial neural network (ANN) using a differentiable Jaccard Loss approximation. An example embodiment may involve obtaining a training image and a corresponding ground truth mask that represents a desired segmentation of the training image. The embodiment may further involve applying an ANN on the training image to generate an output segmentation of the training image that depends on a plurality of weights of the ANN and determining a differentiable Jaccard Loss approximation based on the output segmentation of the training image and the ground truth mask. The embodiment also involves modifying one or more weights of the ANN based on the differentiable Jaccard Loss approximation and providing a representation of the ANN as modified to a mobile computing device. ## Claims ### Claim 1 A method comprising: obtaining, by a computing system, a training image; obtaining, by the computing system, a ground truth mask corresponding to the training image, wherein the ground truth mask represents a desired segmentation of the training image; applying an artificial neural network (ANN) on the training image to generate an output segmentation of the training image, wherein the output segmentation of the training image depends on a plurality of weights of the ANN; determining a differentiable Jaccard Loss approximation based on the output segmentation of the training image and the ground truth mask; based on the differentiable Jaccard Loss approximation, modifying one or more weights of the ANN; and providing a representation of the ANN as modified to a mobile computing device. ### Claim 2 (depends on 1) The method of claim 1, wherein the training image represents a scene, wherein the scene includes an object positioned in a foreground, and wherein the scene further includes a background positioned behind the object. ### Claim 3 (depends on 2) The method of claim 2, wherein the ground truth mask represents a desired segmentation of the scene, and wherein the desired segmentation of the scene specifies differences between the object, the foreground, and the background. ### Claim 4 (depends on 1) The method of claim 1, wherein the differentiable Jaccard Loss approximation represents a ratio of a true positive count of the output segmentation of the training image relative to the ground truth mask over a union between a cardinality of a binary vector of the differentiable Jaccard Loss approximation and a false-positive count of the output segmentation of the training image relative to the ground truth mask, wherein the true positive count represents a first quantity of pixels in the output segmentation of the training image that accurately match corresponding pixels in the desired segmentation of the training image represented by the ground truth mask, and wherein the false-positive count represents a second quantity of pixels in the output segmentation of the training image that fail to accurately match corresponding pixels in the desired segmentation of the training image represented by the ground truth mask. ### Claim 5 (depends on 4) The method of claim 4, wherein the differentiable Jaccard Loss approximation further comprises: a first smooth loss function that represents a lower limit for the true positive count of the output relative to the ground truth mask. ### Claim 6 (depends on 5) The method of claim 5, wherein the first smooth loss function is a hinge loss function. ### Claim 7 (depends on 5) The method of claim 5, wherein the first smooth loss function is a sigmoid loss function. ### Claim 8 (depends on 4) The method of claim 4, wherein the differentiable Jaccard Loss approximation further comprises: a second smooth loss function that represents an upper limit for the false-positive count of the output relative to the ground truth mask. ### Claim 9 (depends on 8) The method of claim 8, wherein the second smooth loss function is the same as the first smooth loss function. ### Claim 10 (depends on 1) The method of claim 1, wherein the ANN is a gradient descent based activation function. ### Claim 11 (depends on 1) The method of claim 1, wherein modifying one or more weights of the ANN comprises: modifying at least one weight of the ANN such that subsequent application of the ANN on the training image generates a second output segmentation of the training image. ### Claim 12 (depends on 11) The method of claim 11, further comprising: determining a second differentiable Jaccard Loss approximation based on the second output segmentation of the training image and the ground truth mask, wherein the second differentiable Jaccard Loss approximation is greater than the first differentiable Jaccard Loss approximation. ### Claim 13 A non-transitory computer readable medium having stored therein instructions executable by a computing system to cause the computing system to perform operations comprising: obtaining, by a computing system, a training image; obtaining, by the computing system, a ground truth mask corresponding to the training image, wherein the ground truth mask represents a desired segmentation of the training image; applying an artificial neural network (ANN) on the training image to generate an output segmentation of the training image, wherein the output segmentation of the training image depends on a plurality of weights of the ANN; determining a differentiable Jaccard Loss approximation based on the output segmentation of the training image and the ground truth mask; based on the differentiable Jaccard Loss approximation, modifying one or more weights of the ANN; and providing a representation of the ANN as modified to a mobile computing device. ### Claim 14 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the training image represents a scene, wherein the scene includes an object positioned in a foreground, and wherein the scene further includes a background positioned behind the object. ### Claim 15 (depends on 14) The non-transitory computer-readable medium of claim 14, wherein the ground truth mask represents a desired segmentation of the scene, and wherein the desired segmentation of the scene specifies differences between the object, the foreground, and the background. ### Claim 16 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the differentiable Jaccard Loss approximation represents a ratio of a true positive count of the output segmentation of the training image relative to the ground truth mask over a union between a cardinality of a binary vector of the differentiable Jaccard Loss approximation and a false-positive count of the output segmentation of the training image relative to the ground truth mask, wherein the true positive count represents a first quantity of pixels in the output segmentation of the training image that accurately match corresponding pixels in the desired segmentation of the training image represented by the ground truth mask, and wherein the false-positive count represents a second quantity of pixels in the output segmentation of the training image that fail to accurately match corresponding pixels in the desired segmentation of the training image represented by the ground truth mask. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the differentiable Jaccard Loss approximation further comprises: a first smooth loss function that represents a lower limit for the true positive count of the output relative to the ground truth mask. ### Claim 18 (depends on 13) The non-transitory computer-readable medium of claim 13, wherein the ANN is a convolution neural network (CNN). ### Claim 19 A system comprising: one or more processors; and a non-transitory computer-readable medium, configured to store instructions, that when executed by the one or more processors, cause the system to perform operations comprising: obtaining a training image; obtaining a ground truth mask corresponding to the training image, wherein the ground truth mask represents a desired segmentation of the training image; applying an artificial neural network (ANN) on the training image to generate an output segmentation of the training image, wherein the output segmentation of the training image depends on a plurality of weights of the ANN; determining a differentiable Jaccard Loss approximation based on the output segmentation of the training image and the ground truth mask; based on the differentiable Jaccard Loss approximation, modifying one or more weights of the ANN; and providing a representation of the ANN as modified to a mobile computing device. ### Claim 20 (depends on 19) The system of claim 19, wherein the differentiable Jaccard Loss approximation represents a ratio of a true positive count of the output segmentation of the training image relative to the ground truth mask over a union between a cardinality of a binary vector of the differentiable Jaccard Loss approximation and a false-positive count of the output segmentation of the training image relative to the ground truth mask, wherein the true positive count represents a first quantity of pixels in the output segmentation of the training image that accurately match corresponding pixels in the desired segmentation of the training image represented by the ground truth mask, and wherein the false-positive count represents a second quantity of pixels in the output segmentation of the training image that fail to accurately match corresponding pixels in the desired segmentation of the training image represented by the ground truth mask.