# US 11,992,702 B2 · Machine learning optimization of fluence maps for radiotherapy treatment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,992,702 B2
- Document ID: US11992702B2
- Status: Granted
- Kind: B2
- Application number: 16/948,486
- Filed: 2020-09-21
- Published: 2022-03-24
- Granted: 2024-05-28
- Earliest claimed: 2020-09-21
- Assignees: Elekta Limited
- Applicants: Elekta, Inc.
- Inventors: Lyndon Stanley Hibbard
- Web page: https://patentgrep.com/patent/US11992702B2

## CPC

- A61N 5/103: {Treatment planning systems}
- A61N 5/1031: {using a specific method of dose optimization}
- A61N 5/1038: {taking into account previously administered plans applied to the same patient, i.e. adaptive radiotherapy}
- A61N 5/1045: {using a multi-leaf collimator, e.g. for intensity modulated radiation therapy or IMRT}
- A61N 5/1081: {Rotating beam systems with a specific mechanical construction, e.g. gantries}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- A61N: ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY (measurement of bioelectric currents A61B; surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body A61B18/00; anaesthetic apparatus in general A61M; incandescent lamps H01K; infrared radiators for heating H05B)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Systems and methods are disclosed for generating fluence maps for a radiotherapy treatment plan that uses machine learning prediction. The systems and methods include identifying image data that indicates treatment constraints for target dose areas and organs at risk areas in an anatomy of the subject, generating anatomy projection images that represent a view of the subject from respective beam angles, using a trained neural network model to generate the computer-simulated fluence map representations based on the anatomy projection images, where the fluence maps indicate a fluence distribution of the radiotherapy treatment at each of the beam angles.

## Claims

### Claim 1

A computer-implemented method for generating fluence maps used in a radiotherapy treatment plan, the method comprising:
obtaining a three-dimensional set of image data corresponding to a subject of radiotherapy treatment, the image data indicating one or more target dose areas and one or more organs-at-risk areas in an anatomy of the subject;
generating anatomy projection images from the image data, each image of the anatomy projection images providing a view of the subject from a respective beam angle of the radiotherapy treatment; and
using a trained neural network model to generate estimated fluence maps based on the anatomy projection images, each of the estimated fluence maps indicating a fluence distribution of the radiotherapy treatment at a respective beam angle,
wherein the neural network model is a trained generative model produced from training in a generative adversarial network (GAN) that includes at least one generative model and at least one discriminative model,
wherein the neural network model is trained in the GAN with corresponding pairs of training anatomy projection images and training fluence maps, and
wherein training of the at least one generative model of the GAN is based on generating synthetic fluence maps from the training anatomy projection images,
wherein training of the at least one discriminative model of the GAN is based on discriminating the synthetic fluence maps from the training fluence maps, and
wherein neural network parameter values learned by the at least one generative model of the GAN and the at least one discriminative model of the GAN are established from adversarial training.

### Claim 2 (depends on 1)

The method of claim 1, wherein each of the estimated fluence maps is a two-dimensional array of beamlet weights normal to a respective beam direction, and wherein beam angles of the radiotherapy treatment correspond to gantry angles of a radiotherapy treatment machine.

### Claim 3 (depends on 2)

The method of claim 2, wherein obtaining the three-dimensional set of image data corresponding to the subject includes obtaining image data for each gantry angle of the radiotherapy treatment machine, and wherein each generated anatomy projection image represents a view of the anatomy of the subject from a given gantry angle used to provide treatment with a given radiotherapy beam.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
using the estimated fluence maps to determine radiation doses in the radiotherapy treatment plan, wherein the radiotherapy treatment comprises a volume modulated arc therapy (VMAT) radiotherapy performed by a radiotherapy treatment machine, wherein multiple radiotherapy beams are shaped to achieve a modulated dose for target areas, from among multiple beam angles, to deliver a prescribed radiation dose.

### Claim 5 (depends on 1)

The method of claim 1, wherein each image of the anatomy projection images is generated by forward projection of the three-dimensional set of image data from respective angles of multiple beam angles.

### Claim 6 (depends on 1)

The method of claim 1, wherein training the neural network model uses pairs of anatomy projection images and fluence maps from a plurality of human subjects, wherein each individual pair is provided from a same human subject, and wherein the neural network model is trained with operations comprising:
obtaining multiple sets of training anatomy projection images, each set of the training anatomy projection images indicating one or more target dose areas and one or more organs-at-risk areas in the anatomy of a respective human subject of the plurality of human subjects;
obtaining multiple sets of training fluence maps corresponding to the training anatomy projection images, each set of the training fluence maps indicating a fluence distribution for the respective human subject of the plurality of human subjects; and
training the neural network model based on the training anatomy projection images that correspond to the training fluence maps.

### Claim 7 (depends on 6)

The method of claim 6, wherein the corresponding pairs of the anatomy projection images and the fluence maps used for training the neural network model are obtained for each beam angle of a radiotherapy treatment machine.

### Claim 8 (depends on 6)

The method of claim 6, wherein the at least one generative model and the at least one discriminative model of the GAN correspond to respective generative and discriminative convolutional neural networks.

### Claim 9 (depends on 8)

The method of claim 8, wherein the GAN comprises a conditional adversarial network (cGAN) or a cycle-consistent generative adversarial network (CycleGAN).

### Claim 10 (depends on 8)

The method of claim 8,
wherein the adversarial training comprises:
training the at least one generative model to generate a first estimated fluence map at a first beam angle from a projection image that represents a view of the anatomy of the respective human subject from the first beam angle; and
training the at least one discriminative model to classify the first estimated fluence map as a generated or as a real training fluence map projection image; and
wherein an output of the at least one generative model is used for training the at least one discriminative model and an output of the at least one discriminative model is used for training the at least one generative model.

### Claim 11 (depends on 8)

The method of claim 8, wherein the GAN is a cycle-consistent generative adversarial network (CycleGAN) comprising the at least one generative model and the at least one discriminative model, wherein the at least one generative model comprises a first generative model and the at least one discriminative model comprises a first discriminative model, wherein the CycleGAN further comprises:
a second generative model trained to:
process, from a given pair of the pairs of anatomy projection images and fluence maps, a given fluence map at a given beam angle as an input; and
generate an estimated anatomy projection image that represents a view of the anatomy of the respective human subject from the given beam angle as an output; and
a second discriminative model trained to classify the estimated anatomy projection image as a generated or as a real anatomy projection image.

### Claim 12 (depends on 11)

The method of claim 11, wherein the CycleGAN comprises a first portion to train the first generative model, the first portion being trained to:
obtain a set of training anatomy projection images representing different views of a patient anatomy from prior treatments that are paired with a set of training fluence maps corresponding to each of the different views, each of the training fluence maps being aligned with a respective one of the training anatomy projection images;
input the set of training anatomy projection images to the first generative model, and output a set of estimated fluence maps from the first generative model;
input the set of estimated fluence maps to the first discriminative model, and classify the set of estimated fluence maps with the first discriminative model as a generated or as a real set of fluence maps; and
input the set of estimated fluence maps to the second generative model, and generate a set of estimated anatomy projection images, to calculate cycle-consistency losses.

### Claim 13 (depends on 12)

The method of claim 12, wherein the CycleGAN comprises a second portion that is trained to:
input the set of training fluence maps corresponding to each of the different views to the second generative model, and output a set of generated anatomy projection images from the second generative model;
input the set of generated anatomy projection images to the second discriminative model, and classify the set of generated anatomy projection images as a generated or as a real set of anatomy projection images; and
input the set of generated anatomy projection images to the first generative model to generate a set of estimated fluence maps to calculate the cycle-consistency losses.

### Claim 14 (depends on 12)

The method of claim 12, wherein:
the cycle-consistency losses are generated based on:
a) a comparison of the set of generated anatomy projection images with the set of training anatomy projection images, and
b) a comparison of the set of generated fluence maps with the set of training fluence maps;
wherein the first generative model is trained to minimize a first loss term that represents an expectation of a difference between a plurality of the estimated fluence maps and respectively paired training fluence maps; and
wherein the second generative model is trained to minimize a second loss term that represents an expectation of difference between a plurality of the estimated anatomy projection images and respectively paired training anatomy projection images.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
generating a set of estimated fluence maps using the neural network model; and
performing numerical optimization with the estimated fluence maps as input to the optimization, wherein the optimization incorporates radiotherapy treatment constraints to produce a pareto-optimal fluence plan used in the radiotherapy treatment plan for the subject.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
performing arc sequencing based on the pareto-optimal fluence plan, to generate a set of initial control points corresponding to each of multiple radiotherapy beams; and
performing direct aperture optimization, to generate a set of final control points corresponding to each of the multiple radiotherapy beams.

### Claim 17 (depends on 16)

The method of claim 16, wherein the radiotherapy treatment comprises a volume modulated arc therapy (VMAT) radiotherapy performed by a radiotherapy treatment machine, and wherein the arc sequencing based on the pareto-optimal fluence plan is performed such that the multiple radiotherapy beams are shaped to achieve a modulated coverage of the one or more target dose areas, from among multiple beam angles, to deliver a prescribed radiation dose.

### Claim 18 (depends on 16)

The method of claim 16, further comprising:
performing the radiotherapy treatment, using the set of final control points, wherein the set of final control points are used to control multi-leaf collimator (MLC) leaf positions of a radiotherapy treatment machine at a given gantry angle corresponding to a given beam angle.

### Claim 19 (depends on 1)

The method of claim 1, further comprising:
comparing a fluence map produced from the neural network model in response to an input set of anatomy projection images, with a fluence map produced from another source.

### Claim 20 (depends on 1)

The method of claim 1, wherein the adversarial training comprises:
training the at least one generative model to generate a first estimated fluence map at a first beam angle from a projection image that represents a view of anatomy from the first beam angle; and
training the at least one discriminative model to classify the first estimated fluence map as a generated or as a real training fluence map projection image;
wherein an output of the at least one generative model is used for training the at least one discriminative model and an output of the at least one discriminative model is used for training the at least one generative model.

### Claim 21 (depends on 1)

The method of claim 1, wherein the GAN comprises a conditional adversarial network (cGAN),
wherein training of the at least one generative model of the cGAN is based on generating the training fluence maps from the training anatomy projection images and wherein training of the at least one discriminative model of the cGAN is based on discriminating data from the training fluence maps, and
wherein the training in the cGAN is conditioned by at least one radiotherapy treatment target area or at least one organs at risk area defined in the training anatomy projection images.

### Claim 22

A system for generating fluence maps used in a radiotherapy treatment plan, the system comprising:
one or more memory devices to store a three-dimensional set of image data corresponding to a subject of radiotherapy treatment, the image data indicating one or more target dose areas and one or more organs-at-risk areas in an anatomy of the subject; and
one or more processors configured to perform operations comprising:
obtaining anatomy projection images from the image data, each image of the anatomy projection images providing a view of the subject from a respective beam angle of the radiotherapy treatment; and
executing a trained neural network model to generate computer-estimated fluence maps based on input of the anatomy projection images, each of the estimated fluence maps indicating a fluence distribution of the radiotherapy treatment at a respective beam angle,
wherein the neural network model is a generative model produced from training in a generative adversarial network (GAN) that includes at least one generative model and at least one discriminative model,
wherein the neural network model is trained in the GAN with corresponding pairs of training anatomy projection images and training fluence maps, and
wherein training of the at least one generative model of the GAN is based on generating synthetic fluence maps from the training anatomy projection images,
wherein training of the at least one discriminative model of the GAN is based on discriminating the synthetic fluence maps from the training fluence maps, and
wherein neural network parameter values learned by the at least one generative model of the GAN and the at least one discriminative model of the GAN are established from adversarial training.

### Claim 23 (depends on 22)

The system of claim 22, the one or more processors further configured to perform operations comprising:
generating a set of estimated fluence maps using the neural network model; and
performing numerical optimization with the estimated fluence maps as input to the optimization, wherein the optimization incorporates radiotherapy treatment constraints to produce a pareto-optimal fluence plan used in the radiotherapy treatment plan for the subject;
wherein each of the estimated fluence maps is a two-dimensional array of beamlet weights normal to a respective beam direction, and wherein beam angles of the radiotherapy treatment correspond to gantry angles of a radiotherapy treatment machine.

### Claim 24 (depends on 23)

The system of claim 23, the one or more processors further configured to perform operations comprising:
performing arc sequencing based on the pareto-optimized fluence plan, to generate a set of initial control points corresponding to each of multiple radiotherapy beams; and
performing direct aperture optimization, to generate a set of final control points corresponding to each of the multiple radiotherapy beams.

### Claim 25 (depends on 22)

The system of claim 22, the one or more processors further configured to perform operations comprising:
using the estimated fluence maps to determine radiation doses in the radiotherapy treatment plan, wherein the radiotherapy treatment comprises a volume modulated arc therapy (VMAT) radiotherapy performed by a radiotherapy treatment machine, wherein multiple radiotherapy beams are shaped to achieve a modulated dose for target areas, from among multiple beam angles, to deliver a prescribed radiation dose.

### Claim 26 (depends on 25)

The system of claim 25, wherein training the neural network model uses pairs of anatomy projection images and fluence maps from a plurality of human subjects, wherein each individual pair is provided from a same human subject, and wherein the neural network model is trained with operations comprising:
obtaining multiple sets of training anatomy projection images, each set of the training anatomy projection images indicating one or more target dose areas and one or more organs-at-risk areas in the anatomy of a respective human subject of the plurality of human subjects;
obtaining multiple sets of training fluence maps corresponding to the training anatomy projection images, each set of the training fluence maps indicating a fluence distribution for the respective human subject of the plurality of human subjects; and
training the neural network model based on the training anatomy projection images that correspond to the training fluence maps;
wherein the at least one generative model and the at least one discriminative model correspond to respective generative and discriminative convolutional neural networks.

### Claim 27 (depends on 22)

The system of claim 22, wherein the adversarial training comprises:
training the at least one generative model to generate a first estimated fluence map at a first beam angle from a projection image that represents a view of anatomy from the first beam angle; and
training the at least one discriminative model to classify the first estimated fluence map as a generated or as a real training fluence map projection image;
wherein an output of the at least one generative model is used for training the at least one discriminative model and an output of the at least one discriminative model is used for training the at least one generative model.

### Claim 28 (depends on 22)

The system of claim 22, wherein the GAN comprises a conditional adversarial network (cGAN),
wherein training of the at least one generative model of the cGAN is based on generating the training fluence maps from the training anatomy projection images and wherein training of the at least one discriminative model of the cGAN is based on discriminating data from the training fluence maps, and
wherein the training in the cGAN is conditioned by at least one radiotherapy treatment target area or at least one organs at risk area defined in the training anatomy projection images.

### Claim 29

A non-transitory computer-readable storage medium comprising computer-readable instructions for generating fluence maps used in a radiotherapy treatment plan, the instructions performing operations comprising:
identifying a three-dimensional set of image data corresponding to a subject of radiotherapy treatment, the image data indicating one or more target dose areas and one or more organs-at-risk areas in an anatomy of the subject;
generating anatomy projection images from the image data, each image of the anatomy projection images providing a view of the subject from a respective beam angle of the radiotherapy treatment; and
using a trained neural network model to generate computer-estimated fluence maps based on input of the anatomy projection images, each of the estimated fluence maps indicating a fluence distribution of the radiotherapy treatment at a respective beam angle,
wherein the neural network model is a generative model produced from training in a generative adversarial network (GAN) that includes at least one generative model and at least one discriminative model,
wherein the neural network model is trained in the GAN with corresponding pairs of training anatomy projection training images and training fluence maps, and
wherein training of the at least one generative model of the GAN is based on generating synthetic fluence maps from the training anatomy projection images,
wherein training of the at least one discriminative model of the GAN is based on discriminating the synthetic fluence maps from the training fluence maps, and
wherein neural network parameter values learned by the at least one generative model of the GAN and the at least one discriminative model of the GAN are established from adversarial training.

### Claim 30 (depends on 29)

The computer-readable storage medium of claim 29, the instructions further performing operations comprising:
generating a set of estimated fluence maps using the neural network model; and
performing numerical optimization with the estimated fluence maps as input to the optimization, wherein the optimization incorporates radiotherapy treatment constraints to produce a pareto-optimal fluence plan used in the radiotherapy treatment plan for the subject;
wherein each of the estimated fluence maps is a two-dimensional array of beamlet weights normal to a respective beam direction, and wherein beam angles of the radiotherapy treatment correspond to gantry angles of a radiotherapy treatment machine.

### Claim 31 (depends on 30)

The computer-readable storage medium of claim 30, the instructions further performing operations comprising:
performing arc sequencing based on the pareto-optimal fluence plan, to generate a set of initial control points corresponding to each of multiple radiotherapy beams; and
performing direct aperture optimization, to generate a set of final control points corresponding to each of the multiple radiotherapy beams.

### Claim 32 (depends on 29)

The computer-readable storage medium of claim 29, the instructions further performing operations comprising:
using the estimated fluence maps to determine radiation doses in the radiotherapy treatment plan, wherein the radiotherapy treatment comprises a volume modulated arc therapy (VMAT) radiotherapy performed by a radiotherapy treatment machine, wherein multiple radiotherapy beams are shaped to achieve a modulated dose for target areas, from among multiple beam angles, to deliver a prescribed radiation dose.

### Claim 33 (depends on 29)

The computer-readable storage medium of claim 29, wherein training the neural network model uses pairs of anatomy projection images and fluence maps from a plurality of human subjects, wherein each individual pair is provided from a same human subject, and wherein the neural network model is trained with operations comprising:
obtaining multiple sets of training anatomy projection images, each set of the training anatomy projection images indicating one or more target dose areas and one or more organs-at-risk areas in the anatomy of a respective human subject of the plurality of human subjects;
obtaining multiple sets of training fluence maps corresponding to the training anatomy projection images, each set of the training fluence maps indicating a fluence distribution for the respective human subject of the plurality of human subjects; and
training the neural network model based on the training anatomy projection images that correspond to the training fluence maps;
wherein the at least one generative model and the at least one discriminative model correspond to respective generative and discriminative convolutional neural networks.

### Claim 34 (depends on 29)

The computer-readable storage medium of claim 29, wherein the adversarial training comprises:
training the at least one generative model to generate a first estimated fluence map at a first beam angle from a projection image that represents a view of anatomy from the first beam angle; and
training the at least one discriminative model to classify the first estimated fluence map as a generated or as a real training fluence map projection image;
wherein an output of the at least one generative model is used for training the at least one discriminative model and an output of the at least one discriminative model is used for training the at least one generative model.

### Claim 35 (depends on 29)

The computer-readable storage medium of claim 29, wherein the GAN comprises a conditional adversarial network (cGAN),
wherein training of the at least one generative model of the cGAN is based on generating the training fluence maps from the training anatomy projection images and wherein training of the at least one discriminative model of the cGAN is based on discriminating data from the training fluence maps, and
wherein the training in the cGAN is conditioned by at least one radiotherapy treatment target area or at least one organs at risk area defined in the training anatomy projection images.