# US 2024/0203038 A1 · Systems and methods for volume data rendering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203038 A1
- Document ID: US20240203038A1
- Status: Published application
- Kind: A1
- Application number: 18/589,389
- Filed: 2024-02-27
- Published: 2024-06-20
- Earliest claimed: 2021-09-01
- Assignees: Shanghai United Imaging Healthcare Co., Ltd.
- Applicants: Shanghai United Imaging Healthcare Co., Ltd.
- Inventors: Wenqing LIU; Zhiguo ZHANG
- Web page: https://patentgrep.com/patent/US20240203038A1

## CPC

- G06T 15/08: Volume rendering
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06T 15/06: Ray-tracing
- G06T 15/20: Perspective computation
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 30/40: for processing medical images, e.g. editing
- G06N 3/08: Learning methods
- G06T 2210/41: Medical
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure relates to systems and methods for volume data rendering. The methods may include obtaining an initial image relating to volume data of a subject by performing a path tracing operation on the volume data, obtaining a total sampling operation count corresponding to the initial image, determining, based on the volume data, one or more reference parameters relating to the initial image, and generating, based on historical image data relating to the volume date of the subject, the initial image, the total sampling operation count, and the one or more reference parameters, a target image using an image processing model.

## Claims

### Claim 1

A method for volume data rendering, implemented on a computing device having at least one storage device storing a set of instructions, and at least one processor in communication with the at least one storage device, the method comprising:
obtaining an initial image relating to volume data of a subject by performing a path tracing operation on the volume data;
obtaining a total sampling operation count corresponding to the initial image;
determining, based on the volume data, one or more reference parameters relating to the initial image; and
generating, based on historical image data relating to the volume date of the subject, the initial image, the total sampling operation count, and the one or more reference parameters, a target image using an image processing model.

### Claim 2 (depends on 1)

The method of claim 1, wherein the historical image data relating to the volume data includes at least one of: a historical target image relating to the volume data, a historical total sampling operation count corresponding to the historical target image, or historical reference parameters relating to the historical target image.

### Claim 3 (depends on 2)

The method of claim 2, wherein the obtaining an initial image relating to the volume data by performing a path tracing operation on the volume data includes:
determining a viewpoint corresponding to the initial image;
obtaining one or more tracing parameters relating to the path tracing operation; and
for each pixel on an image plane corresponding to the initial image,
determining one or more offset vendors in the image plane; and
generating the initial image by performing, based on the viewpoint, the one or more tracing parameters, the one or more offset vendors, and the historical total sampling operation count corresponding to the historical image, the path tracing operation on the volume data.

### Claim 4

(canceled)

### Claim 5 (depends on 3)

The method of claim 3, wherein the generating the initial image by performing, based on the viewpoint, the one or more tracing parameters, the one or more offset vendors, and the historical total sampling operation count corresponding to the historical image, the path tracing operation on the volume data includes:
obtaining a global sampling operation count range;
determining, based on the global sampling operation count range and the historical total sampling operation count, sampling operation count corresponding to the initial image; and
generating the initial image by performing, based on the viewpoint, the one or more tracing parameters, the one or more offset vendors, and the sampling operation count corresponding to the initial image, the path tracing operation on the volume data.

### Claim 6

(canceled)

### Claim 7 (depends on 1)

The method of claim 1, wherein the one or more reference parameters relating to the initial image include at least one of: a depth of each pixel in the initial image, a normal vector corresponding to the depth, label information of each pixel in the initial image, or a motion vector of each pixel in the initial image.

### Claim 8 (depends on 7)

The method of claim 7, wherein the obtaining, based on the volume data, one or more reference parameters relating to the initial image includes:
for each pixel on an image plane corresponding to the initial image,
obtaining the depth of each pixel in the initial image by performing, based on the viewpoint and one or more tracing parameters, a ray casting operation on the volume data;
determining the normal vector corresponding to the depth; and
obtaining the motion vector based on the depth.

### Claim 9

(canceled)

### Claim 10

(canceled)

### Claim 11 (depends on 1)

The method of claim 1, wherein the image processing model is generated according to a model training process including:
obtaining a sample initial image relating to sample volume data of a sample subject by performing a first path tracing operation on the sample volume data;
obtaining a sample total sampling operation count corresponding to the initial image;
determining, based on the sample volume data, one or more sample reference parameters relating to the sample initial image;
obtaining a reference image by performing a second path tracing operation on the sample volume data, wherein the second path tracing operation satisfies a path tracing condition; and
generating the image processing model by training, based on sample historical image data, the sample initial image, the sample total sampling operation count, the one or more sample reference parameters, and the reference image, a preliminary machine learning model.

### Claim 12

(canceled)

### Claim 13

A system for volume data rendering, comprising:
at least one storage medium including a set of instructions; and
at least one processor in communication with the at least one storage medium, wherein when executing the set of instructions, the at least one processor is directed to cause the system to:
obtain an initial image relating to volume data of a subject by performing a path tracing operation on the volume data;
obtain a total sampling operation count corresponding to the initial image;
determine, based on the volume data, one or more reference parameters relating to the initial image; and
generate, based on historical image data relating to the volume date of the subject, the initial image, the total sampling operation count, and the one or more reference parameters, a target image using an image processing model.

### Claim 14 (depends on 13)

The system of claim 13, wherein the historical image data relating to the volume data includes at least one of: a historical target image relating to the volume data, a historical total sampling operation count corresponding to the historical target image, or historical reference parameters relating to the historical target image.

### Claim 15 (depends on 14)

The system of claim 14, wherein to obtain an initial image relating to the volume data by performing a path tracing operation on the volume data, the at least one processor is directed to cause the system to:
determine a viewpoint corresponding to the initial image;
obtain one or more tracing parameters relating to the path tracing operation; and
for each pixel on an image plane corresponding to the initial image,
determine one or more offset vendors in the image plane; and
generate the initial image by performing, based on the viewpoint, the one or more tracing parameters, the one or more offset vendors, and the historical total sampling operation count corresponding to the historical image, the path tracing operation on the volume data.

### Claim 16 (depends on 15)

The system of claim 15, wherein the one or more tracing parameters include a transfer function or material information relating to the volume data.

### Claim 17 (depends on 15)

The system of claim 15, wherein to generate the initial image by performing, based on the viewpoint, the one or more tracing parameters, the one or more offset vendors, and the historical total sampling operation count corresponding to the historical image, the path tracing operation on the volume data, the at least one processor is directed to cause the system to:
obtain a global sampling operation count range;
determine, based on the global sampling operation count range and the historical total sampling operation count, sampling operation count corresponding to the initial image; and
generate the initial image by performing, based on the viewpoint, the one or more tracing parameters, the one or more offset vendors, and the sampling operation count corresponding to the initial image, the path tracing operation on the volume data.

### Claim 18 (depends on 17)

The system of claim 17, wherein to obtain a total sampling operation count corresponding to the initial image, the at least one processor is directed to cause the system to:
determine the total sampling operation count corresponding to the initial image based on the sampling operation count and the historical total sampling operation count.

### Claim 19 (depends on 13)

The system of claim 13, wherein the one or more reference parameters relating to the initial image include at least one of: a depth of each pixel in the initial image, a normal vector corresponding to the depth, label information of each pixel in the initial image, or a motion vector of each pixel in the initial image.

### Claim 20 (depends on 19)

The system of claim 19, wherein to obtain, based on the volume data, one or more reference parameters relating to the initial image, the at least one processor is directed to cause the system to:
for each pixel on an image plane corresponding to the initial image,
obtain the depth of each pixel in the initial image by performing, based on the viewpoint and one or more tracing parameters, a ray casting operation on the volume data;
determine the normal vector corresponding to the depth; and
obtain the motion vector based on the depth.

### Claim 21 (depends on 20)

The system of claim 20, wherein to obtain the depth by performing, based on the viewpoint and the one or more tracing parameters, a ray casting operation on the volume data, the at least one processor is directed to cause the system to:
for each ray in the ray casting operation,
obtain an opacity on each position in the volume data along the ray;
determine an overlay opacity on the each position; and
determine the depth based on the overlay opacity and an opacity threshold.

### Claim 22 (depends on 19)

The system of claim 19, wherein to obtain, based on the volume data, one or more reference parameters relating to the initial image, the at least one processor is directed to cause the system to:
determine the label information of each pixel in the initial image by performing an image segmentation operation on the volume data.

### Claim 23 (depends on 13)

The system of claim 13, wherein to obtain the image processing model, the at least one processor is directed to cause the system to:
obtain a sample initial image relating to sample volume data of a sample subject by performing a first path tracing operation on the sample volume data;
obtain a sample total sampling operation count corresponding to the initial image;
determine, based on the sample volume data, one or more sample reference parameters relating to the sample initial image;
obtain a reference image by performing a second path tracing operation on the sample volume data, wherein the second path tracing operation satisfies a path tracing condition; and
generate the image processing model by training, based on sample historical image data, the sample initial image, the sample total sampling operation count, the one or more sample reference parameters, and the reference image, a preliminary machine learning model.

### Claim 24 (depends on 23)

The system of claim 23, wherein training, based on the sample historical image data, the sample initial image, the sample total sampling operation count, the one or more sample reference parameters, and the reference image, a preliminary machine learning model includes one or more iterations, and in at least one of the one or more iterations, the at least one processor is directed to cause the system to:
generate, based on the sample historical image data, the sample initial image, the sample total sampling operation count, and the one or more sample reference parameters, a predicted image using the preliminary machine learning model or an intermediate machine learning model determined in a prior iteration;
determine, based on the predicted image and the reference image, a value of a loss function;
determine, based on the value of the loss function, whether a termination condition is satisfied in the current iteration; and
in response to determining that the termination condition is satisfied in the current iteration, designate the preliminary machine learning model or the intermediate machine learning model as the image processing model.

### Claim 25

A non-transitory computer readable medium, comprising executable instructions that, when executed by at least one processor, direct the at least one processor to perform a method, the method comprising:
obtaining an initial image relating to volume data of a subject by performing a path tracing operation on the volume data;
obtaining a total sampling operation count corresponding to the initial image;
determining, based on the volume data, one or more reference parameters relating to the initial image; and
generating, based on historical image data relating to the volume date of the subject, the initial image, the total sampling operation count, and the one or more reference parameters, a target image using an image processing model.
