# US 10,481,377 B2 · Real-time autofocus scanning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,481,377 B2
- Document ID: US10481377B2
- Status: Granted
- Kind: B2
- Application number: 16/147,465
- Filed: 2018-09-28
- Published: 2019-04-04
- Granted: 2019-11-19
- Earliest claimed: 2017-09-29
- Assignees: Leica Biosystems Imaging, Inc.
- Applicants: Leica Biosystems Imaging, Inc.
- Inventors: Leng-Chun Chen; Allen Olson; Yunlu Zou; Peyman Najmabadi; Greg Crandall
- Web page: https://patentgrep.com/patent/US10481377B2

## CPC

- G02B 21/244: {using image analysis techniques}
- G02B 21/006: {focusing arrangements; selection of the plane to be imaged}
- G02B 21/0076: {arrangements using fluorescence or luminescence}
- G02B 21/008: {Details of detection or image processing, including general computer control}
- G06T 7/50: Depth or shape recovery
- G06T 2207/10061: from scanning electron microscope
- G06T 2207/10064: Fluorescence image
- H04N 25/711: Time delay and integration [TDI] registers; TDI shift registers
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Real-time autofocus. In an embodiment, a scanning apparatus includes an imaging sensor, a focusing sensor, an objective lens, and processor(s) configured to analyze image data captured by the imaging and focusing sensors, and move the objective lens. Real-time autofocus during scanning of a sample is achieved by determining a true-Z value for the objective lens for a point on a sample and for each of a plurality of regions on the sample. The true-Z values and/or surfaces calculated therefrom are used to determine a predicted-Z value for an unscanned region of the sample. The objective lens is adjusted to the predicted-Z value at the beginning of the unscanned region. After scanning the region, a true-Z value is determined for the region and compared to the predicted-Z value. A rescan of the region is initiated if the comparison exceeds a predetermined threshold.

## Claims

### Claim 1

A digital scanning apparatus comprising:
an objective lens;
a focusing sensor and an imaging sensor positioned in the optical path of the objective lens to sense a field of view of the objective lens; and
at least one processor configured to adjust a height of the objective lens during scanning of a sample by
selecting a macro-focus point on a sample, wherein the macro-focus point is within a predetermined range of both an edge of the sample and a line representing a maximum length across the sample,
scanning a single field of view that includes the macro-focus point using the focusing sensor to acquire image data that includes the macro-focus point at a plurality of image planes corresponding to a plurality of distance positions of the objective lens from the sample,
determining a contrast value for each of the plurality of image planes,
identifying a true-Z value, Z1, that indicates a distance position of the objective lens at which an image plane having a highest contrast value for the macro-focus point corresponds to an image plane of the imaging sensor,
scanning a plurality of regions along the line representing the maximum length across the sample to acquire a plurality of buffers of image data, wherein each buffer comprises a plurality of scan lines and each scan line comprises a plurality of pixels, such that each buffer comprises a number of rows of pixels equal to a number of the plurality of scan lines and a number of columns of pixels equal to a number of the plurality of pixels within each scan line,
determining a contrast ratio value for each of the plurality of buffers,
identifying a plurality of additional true-Z values, Z2-Zn, corresponding to the determined contrast ratio values for the plurality of buffers, wherein each additional true-Z value indicates, for a respective one of the plurality of buffers, a distance position of the objective lens at which the image plane of the imaging sensor has peak contrast,
storing all of the true-Z values, Z1-Zn, with respective location information, wherein the location information for Z1 indicates a location of the macro-focus point, and wherein the location information for each of Z2-Zn indicates a location of the respective one of the plurality of buffers,
determining a location of a region of the sample to be scanned,
identifying a nearest true-Z value to the region of the sample based on the determined location and the stored location information for the true-Z values, Z1-Zn,
determining a predicted-Z value for the region of the sample, based, at least in part, on the nearest true-Z value, and
adjusting a distance between the objective lens and the sample, based on the predicted-Z value, at a start of scanning the region of the sample.

### Claim 2 (depends on 1)

The digital scanning apparatus of claim 1, wherein determining a contrast ratio value for each of the plurality of buffers comprises determining an average contrast ratio value for each column of pixels in the buffer, and determining a contrast ratio value for the buffer based on the average contrast ratio value for each column of pixels in the buffer.

### Claim 3 (depends on 1)

The digital scanning apparatus of claim 1, wherein the at least one processor is further configured to, after scanning the region of the sample:
determine a true-Z value, Znew, for the region of the sample;
determine a difference between the true-Z value, Znew, for the region of the sample and the predicted-Z value for the region of the sample; and
initiate a rescan of the region of the sample when the difference exceeds a predetermined threshold.

### Claim 4 (depends on 3)

The digital scanning apparatus of claim 3, wherein the at least one processor is further configured to, after scanning a plurality of scanned regions of the sample, if the difference between the true-Z value and the predicted-Z value exceeds the predetermined threshold for a predetermined percentage of the plurality of scanned regions, initiate a rescan of the entire sample.

### Claim 5 (depends on 3)

The digital scanning apparatus of claim 3, wherein the predetermined threshold is within a range of 0.5 to 0.9 microns.

### Claim 6 (depends on 3)

The digital scanning apparatus of claim 3, wherein the at least one processor is further configured to calculate a global sample surface using all of the true-Z values, Z1-Zn, and wherein the predicted-Z value for the region of the sample is based, at least in part, on the nearest true-Z value and the global sample surface.

### Claim 7 (depends on 6)

The digital scanning apparatus of claim 6, wherein the at least one processor is further configured to, after scanning the region of the sample, recalculate the global sample surface using the true-Z value, Znew, for the scanned region of the sample.

### Claim 8 (depends on 6)

The digital scanning apparatus of claim 6, wherein the global surface is continually updated, as new true-Z values are calculated and stored, to reflect all stored true-Z values for the sample.

### Claim 9 (depends on 1)

The digital scanning apparatus of claim 1, wherein the at least one processor is further configured to calculate a local sample surface using a subset of true-Z values for buffers corresponding to regions adjacent to the region of the sample to be scanned, and wherein the predicted-Z value for the region of the sample is based, at least in part, on the nearest true-Z value and the local sample surface.

### Claim 10 (depends on 9)

The digital scanning apparatus of claim 9, wherein the local sample surface is L-shaped.

### Claim 11 (depends on 1)

The digital scanning apparatus of claim 1, wherein the plurality of regions, scanned along the line representing the maximum length across the sample, collectively form a contiguous portion of the sample that extends across the sample.

### Claim 12

A method for focusing a digital scanning apparatus that comprises an objective lens, a focusing sensor and an imaging sensor positioned in the optical path of the objective lens to sense a field of view of the objective lens, and at least one processor, the method comprising, by the at least one processor of the digital scanning apparatus:
selecting a macro-focus point on a sample, wherein the macro-focus point is within a predetermined range of both an edge of the sample and a line representing a maximum length across the sample;
scanning a single field of view that includes the macro-focus point using the focusing sensor to acquire image data that includes the macro-focus point at a plurality of image planes corresponding to a plurality of distance positions of the objective lens from the sample;
determining a contrast value for each of the plurality of image planes;
identifying a true-Z value, Z1, that indicates a distance position of the objective lens at which an image plane having a highest contrast value for the macro-focus point corresponds to an image plane of the imaging sensor;
scanning a plurality of regions along the line representing the maximum length across the sample to acquire a plurality of buffers of image data, wherein each buffer comprises a plurality of scan lines and each scan line comprises a plurality of pixels, such that each buffer comprises a number of rows of pixels equal to a number of the plurality of scan lines and a number of columns of pixels equal to a number of the plurality of pixels within each scan line;
determining a contrast ratio value for each of the plurality of buffers;
identifying a plurality of additional true-Z values, Z2-Zn, corresponding to the determined contrast ratio values for the plurality of buffers, wherein each additional true-Z value indicates, for a respective one of the plurality of buffers, a distance position of the objective lens at which the image plane of the imaging sensor has peak contrast;
storing all of the true-Z values, Z1-Zn, with respective location information, wherein the location information for Z1 indicates a location of the macro-focus point, and wherein the location information for each of Z2-Zn indicates a location of the respective one of the plurality of buffers;
determining a location of a region of the sample to be scanned;
identifying a nearest true-Z value to the region of the sample based on the determined location and the stored location information for the true-Z values, Z1-Zn;
determining a predicted-Z value for the region of the sample, based, at least in part, on the nearest true-Z value; and
adjusting a distance between the objective lens and the sample, based on the predicted-Z value, at a start of scanning the region of the sample.