# US 2023/0267616 A1 · Image segmentation system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267616 A1
- Document ID: US20230267616A1
- Status: Published application
- Kind: A1
- Application number: 18/043,552
- Filed: 2021-09-02
- Published: 2023-08-24
- Earliest claimed: 2020-09-02
- Assignees: Singapore Health Services Pte Ltd; Nanyang Technological University
- Applicants: Singapore Health Services PTE LTD; Nanyang Technological University
- Inventors: Dheo Arokhim Yusufi CAHYO; Han Nian Marcus ANG; Leopold SCHMETTERER; Ai Ping YOW; Wing Kee Damon WONG
- Web page: https://patentgrep.com/patent/US20230267616A1

## CPC

- G06T 7/11: Region-based segmentation
- A61B 3/102: {for optical coherence tomography [OCT]}
- A61B 3/1225: {using coherent radiation}
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/174: involving the use of two or more images
- G06T 12/00: Tomographic reconstruction from projections
- G06N 3/048: Activation functions
- G06T 2207/10101: Optical tomography; Optical coherence tomography [OCT]
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30041: Eye; Retina; Ophthalmic
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Disclosed herein is a method of segmenting a volumetric image comprising a plurality of slices, the method comprising: inputting a target slice of the volumetric image to a deep neural network (DNN) having a multi-task learning architecture, the multi-task learning architecture comprising: a segmentation DNN that is configured to output a segmentation of the target slice; and a reconstruction DNN that is configured to: receive a plurality of adjacent slices to the target slice; and output a reconstruction of the target slice based on the plurality of adjacent slices; wherein the reconstruction DNN is further configured to share spatial information with the segmentation DNN, the spatial information being indicative of correlations between the adjacent slices and the target slice.

## Claims

### Claim at position 1

1-17. (canceled)

### Claim 18

A method of segmenting a volumetric image comprising a plurality of slices, the method comprising:
inputting a target slice of the volumetric image to a deep neural network (DNN) having a multi-task learning architecture, the multi-task learning architecture comprising:
a segmentation DNN that is configured to output a segmentation of the target slice; and
a reconstruction DNN that is configured to:
receive a plurality of adjacent slices to the target slice; and
output a reconstruction of the target slice based on the plurality of adjacent slices;
wherein the reconstruction DNN is further configured to share spatial information with the segmentation DNN, the spatial information being indicative of correlations between the adjacent slices and the target slice.

### Claim 19 (depends on 18)

A method according to claim 18, wherein the reconstruction DNN comprises a convolutional feature extractor for generating first feature data from the adjacent slices, and a reconstruction downsampler for generating first reduced-dimension feature data from the first feature data at one or more scales.

### Claim 20 (depends on 19)

A method according to claim 19, wherein the reconstruction DNN comprises a reconstruction upsampler for transforming the first reduced-dimension feature data to first upsampled data having the same dimensions as the first feature data.

### Claim 21 (depends on 19)

A method according to claim 19, wherein the reconstruction DNN comprises one or more dimension reduction layers for applying a dimension reduction mechanism to the first feature data and/or to the first reduced-dimension feature data.

### Claim 22 (depends on 21)

A method according to claim 21, wherein the dimension reduction mechanism comprises:
inputting the first feature data and/or the first reduced-dimension feature data to a 3D convolution layer;
applying an aggregation of features between adjacent slices;
applying batch normalization to the output of the 3D convolution layer; and
applying a ReLU activation function to the output of the batch normalization.

### Claim 23 (depends on 21)

A method according to claim 21, wherein layers of the reconstruction downsampler are connected to layers of the reconstruction upsampler via respective ones of the dimension reduction layers by concatenation.

### Claim 24 (depends on 18)

A method according to claim 18, wherein the segmentation DNN comprises a convolutional feature extractor for generating second feature data from the target slice, and a segmentation downsampler for generating second reduced-dimension feature data from the second feature data at one or more scales.

### Claim 25 (depends on 24)

A method according to claim 24, wherein the segmentation DNN comprises a segmentation upsampler for transforming the second reduced-dimension feature data to second upsampled data having the same dimensions as the second feature data.

### Claim 26 (depends on 24)

A method according to claim 24, wherein layers of the segmentation downsampler are connected to layers of the segmentation upsampler.

### Claim 27 (depends on 24)

A method according to claim 24, wherein the reconstruction DNN is configured to share spatial information with the segmentation DNN by element-wise addition of output of layers of the reconstruction upsampler to output of layers of the segmentation upsampler.

### Claim 28 (depends on 18)

A method according to claim 18, wherein the loss function of the segmentation DNN is the 2D Intersection over Union (IoU) loss function.

### Claim 29 (depends on 18)

A method according to claim 18, wherein the volumetric image is a 3D medical image.

### Claim 30 (depends on 29)

A method according to claim 29, wherein the 3D medical image is a 3D optical coherence tomography (OCT) image.

### Claim 31 (depends on 30)

A method according to claim 30, wherein the 3D OCT image is a retinal image, and wherein the target slice corresponds to a layer of the choroid.

### Claim 32 (depends on 31)

A method according to claim 31, wherein the method is repeated for a plurality of target slices, and wherein the method further comprises generating a choroidal thickness map from segmentation of the plurality of target slices.

### Claim 33 (depends on 18)

A system for segmentation of a volumetric image comprising a plurality of slices, comprising:
at least one processor; and
computer-readable storage having stored thereon instructions for causing the at least one processor to carry out a method according to claim 18.

### Claim 34 (depends on 18)

Non-transitory computer-readable storage having instructions stored thereon for causing at least one processor to carry out a method according to claim 18.
