# US 11,308,633 B2 · Apparatus and method for fiducial marker alignment in electron tomography > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,633 B2 - Document ID: US11308633B2 - Status: Granted - Kind: B2 - Application number: 16/642,590 - Filed: 2018-10-18 - Published: 2020-11-05 - Granted: 2022-04-19 - Earliest claimed: 2017-10-27 - Assignees: King Abdullah University of Science and Technology - Applicants: King Abdullah University of Science and Technology - Inventors: Xin Gao; Renmin Han - Web page: https://patentgrep.com/patent/US11308633B2 ## CPC - G06T 7/33: using feature-based methods - G06T 3/147: using affine transformations - G06T 7/0012: {Biomedical image inspection} - G06T 2207/10061: from scanning electron microscope - G06T 2207/10072: Tomographic images - G06T 2207/30204: Marker - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Provided is an apparatus and method for aligning fiducial markers. The apparatus may align positions of the fiducial markers on the two or more micrographs forming a two or more point sets corresponding to the two or more micrographs; create a first set of matched fiducial markers and unmatched fiducial markers; transform unmatched fiducial markers into transformed point sets and match the unmatched fiducial markers resulting in a second set of matched fiducial markers. The matching of the second set of matched fiducial markers results in improved alignment of a large number of fiducial markers. The aligned positions of fiducial markers may be constrained by an upper bound of transformation deviation of aligning positions of fiducial markers on two or more micrographs. ## Claims ### Claim 1 An apparatus for fiducial marker alignment, the apparatus comprising a processor and a memory storing computer instructions that, when executed by the processor, cause the apparatus to: align positions of fiducial markers on two or more micrographs, wherein each micrograph is formed of a corresponding point set, create a first set of matched fiducial markers and a first set of unmatched fiducial markers for each micrograph of the two or more micrographs; and transform the first set of unmatched fiducial markers of a first micrograph into a transformed point sets and match (1) the transformed point set to (2) the first set of unmatched fiducial markers of a second micrograph, resulting in a second set of matched fiducial markers for the first micrograph, wherein the second set of matched fiducial markers for the first micrograph results in improved alignment of a number of fiducial markers, in addition to the first set of matched fiducial markers, for the first micrograph. ### Claim 2 (depends on 1) The apparatus according to claim 1, wherein aligning positions of fiducial markers is by an affine transformation. ### Claim 3 (depends on 2) The apparatus according to claim 2, wherein the affine transformation is constrained by an upper bound of transformation deviation. ### Claim 4 (depends on 1) The apparatus according to claim 1, wherein creating the first set of matched fiducial markers comprises applying a Guassian mixture model. ### Claim 5 (depends on 1) The apparatus according to claim 1, wherein transforming the first set of unmatched fiducial markers comprises applying a second affine transformation. ### Claim 6 (depends on 1) The apparatus according to claim 1, wherein the computer instructions are configured to, when executed by the processor, cause the apparatus to repeatedly transform the first set of unmatched fiducial markers of another micrograph into an additional transformed point set and match the additional transformed point set to the first set of unmatched fiducial markers of yet another micrograph, resulting in an additional second set of matched fiducial markers until no more additional matched fiducial markers can be created. ### Claim 7 (depends on 1) The apparatus according to claim 1, wherein each of the two or more micrographs have a tilt angle and wherein none of the tilt angles of each of the two or more micrographs are equal. ### Claim 8 (depends on 7) The apparatus according to claim 7, wherein the tilt angles of each of the two or more micrographs differ by a single tilt angle interval and the tilt angle interval is 20° or less. ### Claim 9 (depends on 7) The apparatus according to claim 7, wherein the tilt angles of each of the two or more micrographs each differ by a tilt angle interval and at least one of the tilt angle intervals is at least 50° . ### Claim 10 A method for fiducial marker alignment, the method comprising: aligning positions of fiducial markers on two or more micrographs, wherein each micrograph is formed of a corresponding point set; creating a first set of matched fiducial markers and a first set of unmatched fiducial markers for each micrograph of the two or more micrographs; and transforming the first set of unmatched fiducial markers of a first micrograph into a transformed point sets and matching (1) the transformed point set to (2) the first set of unmatched fiducial markers of a second micrograph, resulting in a second set of matched fiducial markers for the first micrograph, wherein the second set of matched fiducial markers for the first micrograph results in improved alignment of a number of fiducial markers, in addition to the first set of matched fiducial markers, for the first micrograph. ### Claim 11 (depends on 10) The method according to claim 10, wherein aligning positions of fiducial markers is by an affine transformation. ### Claim 12 (depends on 11) The method according to claim 11, wherein the affine transformation is constrained by an upper bound of transformation deviation. ### Claim 13 (depends on 10) The method according to claim 10, wherein creating the first set of matched fiducial markers comprises applying a Guassian mixture model. ### Claim 14 (depends on 10) The method according to claim 10, wherein transforming the first set of unmatched fiducial markers comprises applying a second affine transformation. ### Claim 15 (depends on 10) The method according to claim 10, further comprising repeatedly transforming the first set of unmatched fiducial markers or another micrograph into an additional transformed point set and matching the additional transformed set point to the first set of unmatched fiducial markers if yet another micrograph, resulting in an additional second set of matched fiducial markers until no more additional matched fiducial markers can be created. ### Claim 16 (depends on 10) The method according to claim 10, wherein each of the two or more micrographs have a tilt angle and wherein none of the tilt angles of each of the two or more micrographs are equal. ### Claim 17 (depends on 16) The method according to claim 16, wherein the tilt angles of each of the two or more micrographs differ by a single tilt angle interval and the tilt angle interval is 20° or less. ### Claim 18 (depends on 16) The method according to claim 16, wherein the tilt angles of each of the two or more micrographs each differ by a tilt angle interval and at least one of the tilt angle intervals is at least 50° . ### Claim 19 (depends on 1) A fiducial marker alignment apparatus comprising the apparatus of claim 1. ### Claim 20 (depends on 1) An electron tomography apparatus comprising the apparatus of claim 1.