# US 11,908,133 B2 · Method for cell image segmentation using scribble labels, recording medium and device for performing the method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,908,133 B2
- Document ID: US11908133B2
- Status: Granted
- Kind: B2
- Application number: 17/477,827
- Filed: 2021-09-17
- Published: 2022-03-31
- Granted: 2024-02-20
- Earliest claimed: 2020-09-25
- Assignees: Korea University Research and Business Foundation
- Applicants: Korea University Research and Business Foundation
- Inventors: Won-Ki Jeong; Hyunsoo Lee
- Web page: https://patentgrep.com/patent/US11908133B2

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 20/20: Ensemble learning
- G06T 5/70: Denoising; Smoothing
- G06T 7/11: Region-based segmentation
- G06T 7/194: involving foreground-background segmentation
- G06V 10/82: using neural networks
- G06T 2207/10056: Microscopic image
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20096: Interactive definition of curve of interest
- G06T 2207/20216: Image averaging
- G06T 2207/30024: Cell structures in vitro; Tissue sections in vitro
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A cell image segmentation method using scribble labels includes iteratively pre-training via an image segmentation network (U-Net) using a cell image and scribble labels indicating a cell region and a background region as training data, calculating an exponential moving average (EMA) of image segmentation prediction probabilities at a predetermined interval during the pre-training, self-training by assigning the cell region and the background region for which the EMA of image segmentation prediction probabilities is over a preset threshold to be a pseudo-label, and iteratively refining the image segmentation prediction probability based on a scribbled loss (Lsp) obtained through a result of the training and an unscribbled loss (Lup). Accordingly, it is possible to achieve cell image segmentation with high reliability using only scribble labels.

## Claims

### Claim 1

A cell image segmentation method using scribble labels, comprising:
iteratively pre-training via an image segmentation network (U-Net) using a cell image and scribble labels indicating a cell region and a background region as training data;
calculating an exponential moving average (EMA) of image segmentation prediction probabilities at a predetermined interval during the pre-training;
self-training by assigning the cell region and the background region for which the EMA of image segmentation prediction probabilities is over a preset threshold to be a pseudo-label; and
iteratively refining the image segmentation prediction probability based on a scribbled loss (Lsp) obtained through a result of the training and an unscribbled loss (Lup).

### Claim 2 (depends on 1)

The cell image segmentation method using scribble labels according to claim 1, wherein the pre-training includes predicting the scribbled loss (Lsp) by applying a standard cross entropy loss to only labeled pixels.

### Claim 3 (depends on 2)

The cell image segmentation method using scribble labels according to claim 2, wherein the pre-training is iteratively performed until a preset epoch is reached.

### Claim 4 (depends on 1)

The cell image segmentation method using scribble labels according to claim 1, wherein in the pre-training step, scribbled pixels indicating the cell region and the background region include pixels of 10% or less of the cell image.

### Claim 5 (depends on 1)

The cell image segmentation method using scribble labels according to claim 1, wherein calculating the EMA of image segmentation prediction probabilities includes automatically generating labels for unlabeled pixels using the EMA of image segmentation prediction probabilities.

### Claim 6 (depends on 1)

The cell image segmentation method using scribble labels according to claim 1, wherein calculating the EMA of image segmentation prediction probabilities includes averaging every γ epoch which is an interval of a prediction ensemble process of the scribbled loss (Lsp) and the unscribbled loss (Lup).

### Claim 7 (depends on 1)

The cell image segmentation method using scribble labels according to claim 1, wherein the self-training includes removing noise in pseudo labels through pseudo labeling and label filtering.

### Claim 8 (depends on 1)

The cell image segmentation method using scribble labels according to claim 1, wherein the self-training includes periodically predicting the unscribbled loss (Lup).

### Claim 9 (depends on 1)

The cell image segmentation method using scribble labels according to claim 1, wherein iteratively refining the image segmentation prediction probability includes applying a relative weight to the unscribbled loss (Lup).

### Claim 10 (depends on 1)

A non-transitory computer-readable storage medium having recorded thereon a computer program for performing the cell image segmentation method using scribble labels according to claim 1.

### Claim 11

A cell image segmentation device using scribble labels, comprising:
a warm-up unit to iteratively pre-train via an image segmentation network (U-Net) using a cell image and scribble labels indicating a cell region and a background region as training data;
a probability prediction unit to calculate an exponential moving average (EMA) of image segmentation prediction probabilities at a predetermined interval during the pre-training;
a self-training unit to self-train by assigning the cell region and the background region for which the EMA of image segmentation prediction probabilities is over a preset threshold to be a pseudo-label; and
a refinement unit to iteratively refine the image segmentation prediction probability based on a scribbled loss (Lsp) obtained through a result of the training and an unscribbled loss (Lup).