# US 12,010,430 B2 · Automated application of drift correction to sample studied under electron microscope > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,010,430 B2 - Document ID: US12010430B2 - Status: Granted - Kind: B2 - Application number: 17/822,237 - Filed: 2022-08-25 - Published: 2022-12-22 - Granted: 2024-06-11 - Earliest claimed: 2019-08-16 - Assignees: Protochips, Inc. - Applicants: Protochips, Inc. - Inventors: Franklin Stampley Walden, II; John Damiano, Jr.; David P. Nackashi; Daniel Stephen Gardiner; Mark Uebel; Alan Philip Franks; Benjamin Jacobs - Web page: https://patentgrep.com/patent/US12010430B2 ## CPC - H01J 37/222: {Image processing arrangements associated with the tube} - H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects - G06T 7/215: Motion-based segmentation - G06T 7/337: {involving reference images or patches} - H01J 37/265: {Controlling the tube; circuit arrangements adapted to a particular application not otherwise provided, e.g. bright-field-dark-field illumination} - G06T 2207/10061: from scanning electron microscope - H01J 2237/221: Image processing - H01J 2237/2811: Large objects - H01J: ELECTRIC DISCHARGE TUBES OR DISCHARGE LAMPS (spark-gaps H01T; electric arc lamps H05B31/00; particle accelerators H05H) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Control system configured for sample tracking in an electron microscope environment registers a movement associated with a region of interest located within an active area of a sample under observation with an electron microscope. The registered movement includes at least one directional constituent. The region of interest is positioned within a field of view of the electron microscope. The control system directs an adjustment of the electron microscope control component to one or more of dynamically center and dynamically focus the view through the electron microscope of the region of interest. The adjustment comprises one or more of a magnitude element and a direction element. ## Claims ### Claim 1 A control system configured for tracking a sample under observation during an experimental session in an electron microscope, the control system comprising: a memory; and a processor; wherein the processor of the control system is configured for: tracking movement of the sample under observation as the sample changes during the experimental session, wherein the movement of the sample is tracked within a region of interest positioned within a field of view of the electron microscope, wherein the movement of the sample is tracked by comparing a live image of the sample against a template image of the region of interest; determining a drift vector of the sample based on the tracked movement, wherein the drift vector includes an X-direction component and a Y-direction component; adjusting the region of interest to correct for drift of the sample during the experimental session using the determined drift vector; and morphing the template image to approximate the live image of the sample under observation using image filters. ### Claim 2 (depends on 1) The control system of claim 1, wherein the processor of the control system is further configured for adjusting a plurality of settings of the electron microscope in response to the determined drift vector to improve the live image of the sample. ### Claim 3 (depends on 1) The control system of claim 1, wherein the template image is morphed using frame averaging over the region of interest. ### Claim 4 (depends on 1) The control system of claim 1, wherein the processor of the control system is further configured for recording the movement of the sample over a period of time during the experimental session to generate a map of the movement. ### Claim 5 (depends on 4) The control system of claim 4, wherein the generated map is a two-dimensional map of a history of movements occurring in the region of interest. ### Claim 6 (depends on 4) The control system of claim 4, wherein the generated map is a three-dimensional map of a history of movements occurring in the region of interest. ### Claim 7 (depends on 1) The control system of claim 1, wherein the drift vector further includes a Z-direction component. ### Claim 8 (depends on 7) The control system of claim 7, wherein the processor of the control system is further configured for adjusting a focus level of the electron microscope in the Z-direction to an optimal focus height for the region of interest in response to the determined drift vector. ### Claim 9 (depends on 1) The control system of claim 1, wherein the processor of the control system is further configured for analyzing variance of pixel intensities in the live image to determine a focus score for the region of interest. ### Claim 10 (depends on 9) The control system of claim 9, wherein the focus score is determined using at least one of the following: a Fast Fourier Transform calculation of the pixel intensities, a contrast transfer function analysis of the pixel intensities, and a beam tilt analysis of the pixel intensities. ### Claim 11 A method of tracking a sample under observation during an experimental session in an electron microscope, the method comprising: tracking movement of the sample under observation as the sample changes during the experimental session, wherein the movement of the sample is tracked within a region of interest positioned within a field of view of the electron microscope, wherein the movement of the sample is tracked by comparing a live image of the sample against a template image of the region of interest; determining a drift vector of the sample based on the tracked movement, wherein the drift vector includes an X-direction component and a Y-direction component; adjusting the region of interest to correct for drift of the sample during the experimental session using the determined drift vector; and morphing the template image to approximate the live image of the sample under observation using image filters. ### Claim 12 (depends on 11) The method of claim 11, further comprising adjusting a plurality of settings of the electron microscope in response to the determined drift vector to improve the live image of the sample. ### Claim 13 (depends on 11) The method of claim 11, wherein the template image is morphed using frame averaging over the region of interest. ### Claim 14 (depends on 11) The method of claim 11, further comprising recording the movement of the sample over a period of time during the experimental session to generate a map of the movement. ### Claim 15 (depends on 14) The method of claim 14, wherein the generated map is a two-dimensional map of a history of movements occurring in the region of interest. ### Claim 16 (depends on 14) The method of claim 14, wherein the generated map is a three-dimensional map of a history of movements occurring in the region of interest. ### Claim 17 (depends on 11) The method of claim 11, wherein the drift vector further includes a Z-direction component. ### Claim 18 (depends on 17) The method of claim 17, further comprising adjusting a focus level of the electron microscope in the Z-direction to an optimal focus height for the region of interest in response to the determined drift vector. ### Claim 19 (depends on 11) The method of claim 11, further comprising analyzing variance of pixel intensities in the live image to determine a focus score for the region of interest. ### Claim 20 (depends on 19) The method of claim 19, wherein the focus score is determined using at least one of the following: a Fast Fourier Transform calculation of the pixel intensities, a contrast transfer function analysis of the pixel intensities, and a beam tilt analysis of the pixel intensities.