# US 2022/0092806 A1 · Image contouring using spiking neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0092806 A1
- Document ID: US20220092806A1
- Status: Published application
- Kind: A1
- Application number: 17/027,891
- Filed: 2020-09-22
- Published: 2022-03-24
- Earliest claimed: 2020-09-22
- Assignees: Varian Medical Systems, Inc.
- Applicants: Varian Medical Systems, Inc.
- Inventors: Wenlong YANG
- Web page: https://patentgrep.com/patent/US20220092806A1

## CPC

- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G06T 7/55: from multiple images
- G06N 3/0499: Feedforward networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/12: Edge-based segmentation
- G16H 30/40: for processing medical images, e.g. editing
- G16H 40/67: for remote operation
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30061: Lung
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Disclosed herein are methods and system for training artificial intelligence models configured to execute image segmentation techniques. The methods and system describe a server that receives a first image including a set of pixels depicting multiple objects. The server also receives a second image having a second set of pixels depicting the same set of objects. The server then analyzes the pixels from the first and second images. When a difference between at least one visual attribute of a pixel within the second image and a corresponding pixel within the first image satisfies a predetermined threshold, it will be encoded as spikes to send to the model, the model will be trained using supervised STDP rule by revising weights associated with the nodes within the AI model where the node corresponds to the pixels within the first and/or the second image.

## Claims

### Claim 1

A method comprising:
executing, by a computer, an artificial intelligence model utilizing a spiking neural network to segment one or more objects represented by a first set of pixels in a first image, the spiking neural network comprising a set of nodes corresponding to the first set of pixels, each node having a synaptic weight, wherein at least a first portion of nodes within the set of nodes represents input nodes corresponding to the first set of pixels and a second portion of the nodes within the set of nodes represents output nodes associated with a likelihood of each respective pixel corresponding to at least one object within the one or more objects;
comparing, by the computer, a grayscale attribute of each pixel within the first set of pixels with a corresponding pixel within a second set of pixels of a second image of the one or more objects;
iteratively training, by the computer using a supervised spiked-timing dependent plasticity method, the spiking neural network by:
in response to a difference between grayscale attributes of a first pixel within the first set of pixels with a second pixel within the second set of pixels satisfying a predetermined threshold:
a node corresponding to the first pixel transmits a spike, and
revising a synaptic weight associated with the first or the second pixel with each iteration;
upon receiving a third image, executing, by the computer, the artificial intelligence model utilizing a trained spiking neural network to segment the one or more objects in the third image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the computer revises the synaptic weight using the following formula:
Δw=Xtrace(Ttarget−Tactual)
wherein Δw represents a required weight change, wherein Xtrace represents a trace voltage from pre-node for post-node, wherein Ttarget represents a ground truth value, and wherein Tactual represents an actual spike output during training.

### Claim 3 (depends on 2)

The method of claim 2, wherein the computer depresses the synaptic weight when Ttarget is lower than Tactual.

### Claim 4 (depends on 2)

The method of claim 2, wherein the computer potentiates the synaptic weight when the Ttarget is bigger than Tactual.

### Claim 5 (depends on 1)

The method of claim 1, wherein the computer extracts at least one of the first image or the second image from a media element.

### Claim 6 (depends on 5)

The method of claim 5, wherein the media element is a four-dimensional CT scan image.

### Claim 7 (depends on 6)

The method of claim 6, wherein the computer extract the at least one of the first image or the second image from the media element based on a predetermined time period.

### Claim 8 (depends on 7)

The method of claim 7, wherein the predetermined time period is received and revisable by a user operating a computing device that displays results of execution of the artificial intelligence model.

### Claim 9 (depends on 5)

The method of claim 5, wherein the third image is extracted from the media element.

### Claim 10 (depends on 1)

The method of claim 1, wherein the third image represents a neighboring phase of a four dimensional CT scan image.

### Claim 11

A system comprising:
a server comprising a processor and a non-transitory computer-readable medium containing instructions that when executed by the processor causes the processor to perform operations comprising:
execute an artificial intelligence model utilizing a spiking neural network to segment one or more objects represented by a first set of pixels in a first three-dimensional image, the spiking neural network comprising a set of nodes corresponding to the first set of pixels, each node having a synaptic weight, wherein at least a first portion of nodes within the set of nodes represents input nodes corresponding to the first set of pixels and a second portion of the nodes within the set of nodes corresponds to a likelihood of each respective pixel corresponding to at least one object within the one or more objects;
compare a grayscale attribute of each pixel within the first set of pixels with a corresponding pixel within a second set of pixels of a second image of the one or more objects;
iteratively train using a supervised spiked-timing dependent plasticity method, the spiking neural network by:
in response to a difference between grayscale attributes of a first pixel within the first set of pixels with a second pixel within the second set of pixels satisfying a predetermined threshold:
a node corresponding to the first pixel transmits a spike having a synaptic weight, and
revising the synaptic weight associated with the first or the second pixel with each iteration;
upon receiving a third image, execute the artificial intelligence model utilizing a trained spiking neural network to identify the one or more objects in the third image.

### Claim 12 (depends on 11)

The system of claim 11, wherein the server revises the synaptic weight using the following formula:
Δw=Xtrace(Ttarget−Tactual)
wherein Δw represents a required weight change, wherein Xtrace represents a trace voltage from pre-node for post-node, wherein Ttarget represents a ground truth value, and wherein Tactual represents an actual spike output during training.

### Claim 13 (depends on 11)

The system of claim 11, wherein the server depresses the synaptic weight when Ttarget is lower than Tactual.

### Claim 14 (depends on 11)

The system of claim 11, wherein the server potentiates the synaptic weight when the Ttarget is bigger than Tactual.

### Claim 15 (depends on 11)

The system of claim 11, wherein the server extract at least one of the first image or the second image from a media element.

### Claim 16 (depends on 15)

The system of claim 15, wherein the media element is a four-dimensional CT scan image.

### Claim 17 (depends on 16)

The system of claim 16, wherein the server extract the at least one of the first image or the second image from the media element based on a predetermined time period.

### Claim 18 (depends on 17)

The system of claim 17, wherein the predetermined time period is received and revisable by a user operating a computing device that displays results of execution of the artificial intelligence model.

### Claim 19 (depends on 15)

The system of claim 15, wherein the third image is extracted from the media element.

### Claim 20 (depends on 11)

The system of claim 11, wherein the third image represents a neighboring phase of a four dimensional CT scan image.
