Foundation model-assisted processing of microscope images
AssigneeCarl Zeiss Microscopy GmbH
InventorsManuel AMTHOR, Daniel HAASE, Ralf WOLLESCHENSKY
Life of this patent
Estimating the term…
Abstract
Techniques for controlling a microscopy system and for processing microscope images are disclosed. In this context, a user input in free-text format is processed in order to create a prompt for a machine-learned text-to-text foundation model. The output of the foundation model can be used subsequently to solve an image processing task or for controlling the microscopy system.
Claims23
A computer-implemented method for processing microscope images, wherein the method comprises:
- receiving a microscope image,
- receiving a user input in free-text format, the user input being indicative of at least one image processing task for evaluating or manipulating microscopic structures displayed in the microscope image,
- on the basis of the user input, triggering the use of a machine-learned text-based foundation model for creating program code that allows the image processing task to be solved, and
- triggering an execution of the program code and, on the basis thereof, receiving results data.
A computer-implemented method for controlling a microscopy system for performing a microscopy task, wherein the microscopy task comprises at least one of an image capture of a microscope image, a display of a microscope image and an image processing of a microscope image, wherein the method comprises:
- receiving a user input in free-text format, the user input being indicative of the microscopy task, on the basis of the user input, triggering the use of a machine-learned text-based foundation model for creating a control instruction for the microscopy system, and controlling the microscopy system on the basis of the control instruction.
An electronic data processing device having a processor and a memory, wherein the processor is configured to load program code from the memory and run said program code, wherein the processor, on the basis of the program code, is configured to:
- receive a microscope image,
- receive a user input in free-text format, the user input being indicative of at least one image processing task for evaluating or manipulating microscopic structures displayed in the microscope image,
- on the basis of the user input, trigger the use of a machine-learned text-based foundation model for creating program code that allows the image processing task to be solved, and
- trigger an execution of the program code and, on the basis thereof, receiving results data.
An electronic data processing device having a processor and a memory, wherein the processor is configured to load program code from the memory and run said program code, wherein the processor, on the basis of the program code, is configured to:
- receive a user input in free-text format, the user input being indicative of a microscopy task,
- on the basis of the user input, triggering use of a machine-learned text-based foundation model for creating a control instruction for a microscopy system, and
- controlling the microscopy system on the basis of the control instruction.
Dependencies are inferred from the claim text.