# US 12,181,553 B2 · Deep learning techniques for magnetic resonance image reconstruction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,181,553 B2 - Document ID: US12181553B2 - Status: Granted - Kind: B2 - Application number: 17/702,545 - Filed: 2022-03-23 - Published: 2022-07-07 - Granted: 2024-12-31 - Earliest claimed: 2018-07-30 - Assignees: Hyperfine Operations, Inc. - Applicants: Hyperfine Operations, Inc. - Inventors: Jo Schlemper; Seyed Sadegh Mohseni Salehi; Michal Sofka; Prantik Kundu; Ziyi Wang; Carole Lazarus; Hadrien A. Dyvorne; Laura Sacolick; Rafael O'Halloran; Jonathan M. Rothberg - Web page: https://patentgrep.com/patent/US12181553B2 ## CPC - G01R 33/5608: {Data processing and visualization specially adapted for MR, e.g. for feature analysis and pattern recognition on the basis of measured MR data, segmentation of measured MR data, edge contour detection on the basis of measured MR data, for enhancing measured MR data in terms of signal-to-noise ratio by means of noise filtering or apodization, for enhancing measured MR data in terms of resolution by means for deblurring, windowing, zero filling, or generation of gray-scaled images, colour-coded images or images displaying vectors instead of pixels (image data processing or generation, in general G06T)} - G01R 33/4824: {using a non-Cartesian trajectory} - G01R 33/561: by reduction of the scanning time, i.e. fast acquiring systems, e.g. using echo-planar pulse sequences - G06F 17/142: {Fast Fourier transforms, e.g. using a Cooley-Tukey type algorithm} - G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)} - G06F 18/24143: {Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN]} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 3/096: Transfer learning - G06T 5/60: using machine learning, e.g. neural networks - G06T 5/70: Denoising; Smoothing - G06T 12/20: Inverse problem, i.e. transformations from projection space into object space - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/764: using classification, e.g. of video objects - G06V 10/768: {using context analysis, e.g. recognition aided by known co-occurring patterns} - G06V 10/82: using neural networks - G01R 33/5611: {Parallel magnetic resonance imaging, e.g. sensitivity encoding [SENSE], simultaneous acquisition of spatial harmonics [SMASH], unaliasing by Fourier encoding of the overlaps using the temporal dimension [UNFOLD], k-t-broad-use linear acquisition speed-up technique [k-t-BLAST], k-t-SENSE (structural details of arrays of sub-coils G01R33/3415)} - G06T 2207/10088: Magnetic resonance imaging [MRI] - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2210/41: Medical - G06V 2201/03: Recognition of patterns in medical or anatomical images - G01R: MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES (indicating correct tuning of resonant circuits H03J3/12) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A magnetic resonance imaging (MRI) system, comprising: a magnetics system comprising: a B0 magnet configured to provide a B0 field for the MRI system; gradient coils configured to provide gradient fields for the MRI system; and at least one RF coil configured to detect magnetic resonance (MR) signals; and a controller configured to: control the magnetics system to acquire MR spatial frequency data using non-Cartesian sampling; and generate an MR image from the acquired MR spatial frequency data using a neural network model comprising one or more neural network blocks including a first neural network block, wherein the first neural network block is configured to perform data consistency processing using a non-uniform Fourier transformation. ## Claims ### Claim 1 A method, comprising: generating a magnetic resonance (MR) image from input MR spatial frequency data using a neural network model configured to process spatial frequency domain data and image domain data, wherein generating the MR image comprises: applying, to image domain data, at least a first non-uniform Fourier transformation to obtain a first output; applying at least a second non-uniform Fourier transformation to the input MR spatial frequency data to obtain a second output; and providing at least the first output and the second output as inputs to a plurality of convolutional layers. ### Claim 2 (depends on 1) The method of claim 1, wherein the neural network model comprises one or more neural network blocks including a first neural network block configured to perform data consistency processing using at least one of the first or second non-uniform Fourier transformations. ### Claim 3 (depends on 1) The method of claim 1, further comprising: obtaining the input MR spatial frequency data; generating an initial image from the input MR spatial frequency data using at least one of the first or second non-uniform Fourier transformations; and applying the neural network model to the initial image at least in part by using a first neural network block of the neural network model to perform data consistency processing using at least one of the first or second non-uniform Fourier transformations. ### Claim 4 (depends on 1) The method of claim 1, wherein a first neural network block of the neural network model is configured to perform data consistency processing at least in part by performing one or more non-uniform Fourier transformations on data by applying a gridding interpolation transformation, a fast Fourier transformation, and a de-apodization transformation to the data. ### Claim 5 (depends on 4) The method of claim 4, wherein applying the gridding interpolation transformation to the data is performed using sparse graphical processing unit (GPU) matrix multiplication. ### Claim 6 (depends on 1) The method of claim 1, wherein a first neural network block of the neural network model comprises: a data consistency block configured to perform data consistency processing; and the plurality of convolutional layers or a second plurality of convolutional layers. ### Claim 7 (depends on 6) The method of claim 6, wherein the data consistency block is configured to: apply the first non-uniform Fourier transformation to a first image, provided as input to the data consistency block, to obtain first MR spatial frequency data; and apply the second non-uniform Fourier transformation to a difference between the first MR spatial frequency data and the input MR spatial frequency data. ### Claim 8 (depends on 7) The method of claim 7, wherein applying the first non-uniform Fourier transformation to the first image domain data comprises: applying, to the first image domain data, a de-apodization transformation followed by a Fourier transformation, and followed by a gridding interpolation transformation. ### Claim 9 (depends on 6) The method of claim 6, wherein at least one of the plurality of convolutional layers or the second plurality of convolutional layers include one or more convolutional layers and one or more transposed convolutional layers. ### Claim 10 (depends on 6) The method of claim 6, wherein at least one of the plurality of convolutional layers or the second plurality of convolutional layers have a U-net structure. ### Claim 11 (depends on 1) The method of claim 1, wherein the neural network model comprises a first neural network block that comprises: the plurality of convolutional layers configured to receive as input: the image domain data; and the second output obtained by applying the second non-uniform Fourier transformation to the input MR spatial frequency data, wherein the second non-uniform Fourier transformation is an adjoint non-uniform Fourier transformation. ### Claim 12 (depends on 11) The method of claim 11, wherein the plurality of convolutional layers is further configured to receive as input: output obtained by applying the first non-uniform Fourier transformation and the second non-uniform Fourier transformation to the image domain data. ### Claim 13 (depends on 1) The method of claim 1, wherein: the first output is obtained by applying, to the image domain data, the first non-uniform Fourier transformation followed by the second non-uniform Fourier transformation. ### Claim 14 (depends on 13) The method of claim 13, wherein the image domain data, the first output, and the second output are provided as inputs to the plurality of convolutional layers. ### Claim 15 (depends on 1) The method of claim 1, further comprising: applying a convolutional neural network to a result of applying the first non-uniform Fourier transformation to the image domain data to obtain an intermediate output; and applying the second non-uniform Fourier transformation to the intermediate output to obtain the first output. ### Claim 16 (depends on 1) The method of claim 1, wherein points in the input MR spatial frequency data were obtained using a non-Cartesian sampling trajectory. ### Claim 17 (depends on 16) The method of claim 16, wherein the first non-uniform Fourier transformation is determined at least in part by using the non-Cartesian sampling trajectory. ### Claim 18 (depends on 1) The method of claim 1, wherein the neural network model comprises a first neural network sub-model configured to process the spatial frequency domain data, and a second neural network sub-model configured to process the image domain data. ### Claim 19 (depends on 1) The method of claim 1, wherein the second non-uniform Fourier transformation is an adjoint non-uniform Fourier transformation. ### Claim 20 At least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform a method comprising: generating a magnetic resonance (MR) image from input MR spatial frequency data using a neural network model configured to process spatial frequency domain data and image domain data, wherein generating the MR image comprises: applying, to the image domain data, at least a first non-uniform Fourier transformation to obtain a first output; applying at least a second non-uniform Fourier transformation to the input MR spatial frequency data to obtain a second output; and providing at least the first output and the second output as inputs to a plurality of convolutional layers. ### Claim 21 A magnetic resonance imaging (MRI) system, comprising: a magnetics system comprising: a B0 magnet configured to provide a B0 field for the MRI system; gradient coils configured to provide gradient fields for the MRI system; and at least one RF coil configured to detect magnetic resonance (MR) signals; a controller configured to: control the magnetics system to acquire MR spatial frequency data using a non-Cartesian sampling trajectory; and generate an MR image from the acquired MR spatial frequency data using a neural network model configured to process spatial frequency domain data and image domain data, wherein generating the MR image comprises: applying, to the image domain data, at least a first non-uniform Fourier transformation to obtain a first output; applying at least a second non-uniform Fourier transformation to the input MR spatial frequency data to obtain a second output; and providing at least the first output and the second output as inputs to a plurality of convolutional layers. ### Claim 22 (depends on 21) The MRI system of claim 21, wherein the B0 magnet is a permanent magnet.