# US 2026/0256525 A1 · Real-Time Dynamic Organ Modelling and Image Augmentation in Surgery

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0256525 A1
- Document ID: US20260256525A1
- Status: Published application
- Kind: A1
- Application number: 19/552,728
- Filed: 2026-02-27
- Published: 2026-09-03
- Earliest claimed: 2025-02-28
- Applicants: Orsi Academy bv
- Inventors: Jasper HOFMAN; Alexandre MOTTRIE; Pieter DE BACKER
- Web page: https://patentgrep.com/patent/US20260256525A1

## CPC

- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- G06N 20/00: Machine learning
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/11: Region-based segmentation
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/764: using classification, e.g. of video objects
- G16H 30/40: for processing medical images, e.g. editing
- A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones}
- A61B 2034/2055: {Optical tracking systems}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image}
- A61B 2090/367: {creating a 3D dataset from 2D images using position information}
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/30004: Biomedical image processing
- G06T 2210/41: Medical
- G06T 2219/2016: Rotation, translation, scaling
- G06V 2201/031: of internal organs
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

This method for real-time organ model modelling is based on medical image data. Computational requirements are reduced. In some aspects, these organ models can be used to augment the medical images in real-time.

## Claims

### Claim 1

A computer-implemented method for real-time organ detection in medical procedures on a patient, comprising the steps of:
receiving a plurality of digital images of a medical procedure, the plurality of digital images being a video;
automatically segmenting the digital images by a pretrained machine learning model into image segments and separating and identifying the segmented image segments in two or more classes, said classes comprising at least a class of organ tissue;
preparing one or more three-dimensional organ models based on historical image data of said patient;
based on the image segments identified as organ tissue, detecting at least one organ in a first digital image of the plurality of digital images;
adapting the organ model of the detected organ for the first digital image, wherein the adaptation of said organ model is performed based on the class of organ tissue of the detected organ in said first digital image such that the organ model substantially corresponds to the detected organ in the first digital image in size and orientation, and optionally position;
characterized in that the method further comprises the steps of:
determining in at least one subsequent digital image to the first digital image, a rotation and/or resizing, and/or an optional translation of the identified organ tissue of the organ with respect to the identified organ tissue of the organ in the first digital image, and adapting the three-dimensional organ for the subsequent digital image by transforming the organ model for the first image according to said determined rotation and/or resizing and/or optional translation;
determining in at least one further subsequent digital image to the first digital image, said further subsequent digital image being subsequent to the subsequent digital image, a further rotation and/or resizing and/or an optional translation of the identified organ tissue of the organ with respect to the identified organ tissue of the organ in a preceding digital image for which a three-dimensional organ model was adapted, and adapting the three-dimensional organ model for the further subsequent digital image by transforming the organ model for the preceding digital image according to said determined further rotation and/or resizing and/or optional translation.

### Claim 2 (depends on 1)

The computer-implemented method according to claim 1, said classes further comprising a class of non-organic objects and a class of background tissue.

### Claim 3 (depends on 1)

The computer-implemented method according to claim 1, wherein said class of organ tissue is subdivided in subclasses per organ.

### Claim 4 (depends on 3)

The computer-implemented method according to claim 3, wherein, when multiple separate organs are detected in the organ tissue in the detecting the at least one organ, then the adapting the prepared organ model, the determining in the at least one subsequent digital image, and the determining in the at least one further subsequent digital image are performed separately for each of said detected separate organs.

### Claim 5 (depends on 1)

The computer-implemented method according to claim 1, wherein the step of automatically segmenting the digital images is performed for each pixel of the digital images, and classifies each of the pixel in one of the classes.

### Claim 6 (depends on 1)

The computer-implemented method according to claim 1, comprising a step of providing the organ model adapted for a digital image as an organ model overlay over the digital image.

### Claim 7 (depends on 2)

The computer-implemented method according to claim 2, comprising a step of creating a non-organic overlay for the digital images, wherein the non-organic overlay only comprises the non-organic objects in the digital images, and wherein said non-organic overlay is applied over the organ model overlay.

### Claim 8 (depends on 1)

The computer-implemented method according to claim 1, the classes further comprise a class of non-organic objects, the method comprising a step of detecting a tool based on the image segments identified as non-organic objects, and comprising a step of determining interaction between a tool and the organ in one or more of the digital images based on the organ model in said one or more digital images and the non-organic object image segments in said one or more digital images.

### Claim 9 (depends on 1)

The computer-implemented method according to claim 1, wherein the digital images comprise stereoscopic image data, and wherein the rotation or further rotation and/or resizing or further resizing and/or optional translation or further translation is determined for the subsequent or further subsequent digital image based on the stereoscopic image data of said subsequent or further subsequent digital image.

### Claim 10 (depends on 1)

The computer-implemented method according to claim 1, wherein in case of insufficient organ tissue being present in a subsequent or further subsequent digital image, no organ model is adapted for said further or subsequent digital image, and no organ model is associated to said further or subsequent digital image.

### Claim 11 (depends on 1)

The computer-implemented method according to claim 1, wherein the preceding digital image and the subsequent further subsequent digital image are immediately subsequent digital images timewise.

### Claim 12 (depends on 10)

The computer-implemented method according to claim 10, wherein, if the preceding digital image does not have an associated organ model, the determining in the at least one further subsequent digital image for said further subsequent digital image is performed with an earlier digital image having an associated organ model serving as the preceding digital image, said earlier digital image being preceding to the preceding digital image, said earlier digital image being the digital image having an associated organ model temporally closest to the further subsequent digital image.

### Claim 13 (depends on 1)

The computer-implemented method according to claim 1, wherein the step of determining a rotation and/or resizing and/or optional translation or a further rotation and/or resizing and/or optional translation is preceded by a step of identifying key features in the organ tissue of the organ in the subsequent digital image and the first digital image or in the further subsequent digital image and the preceding digital image, said key features comprising conjectured edges of the organ, and wherein the step of determining a rotation and/or resizing and/or optional translation or a further rotation and/or resizing and/or optional translation is performed based on said identified key features in the subsequent digital image and the first digital image or in the further subsequent digital image and the preceding digital image.

### Claim 14 (depends on 1)

The computer-implemented method according to claim 1, wherein the determination of the rotation and/or resizing and/or an optional translation or determination of the further rotation and/or resizing and/or an optional translation is based on the subsequent digital image and the first digital image or on the further subsequent digital image and the preceding digital image, and on a segmentation map of the subsequent digital image and the first digital image or a segmentation map of the further subsequent digital image and the preceding digital image, wherein the segmentation map only comprises the image segments identified as organ tissue, and wherein the segmentation map comprises separate organ segmentation map for each of the detected organs separately.

### Claim 15 (depends on 1)

The computer-implemented method according to claim 1, wherein the organ model is a deformable mesh, and wherein the determining in the at least one subsequent digital image and the determining in the at least one further subsequent digital image respectively further determine a deformation of the identified organ tissue of the organ in the subsequent digital image with respect to the identified organ tissue of the organ in the first digital image or of the identified organ tissue of the organ in the further subsequent digital image with respect to the identified organ tissue of the organ in the preceding digital image, and respectively further adapt the three-dimensional organ for the subsequent digital image by transforming the organ model for the first image according to said determined deformation or further adapt the three-dimensional organ model for the further subsequent digital image by transforming the organ model for the preceding digital image according to said determined deformation.
