Clough, James R.

Duan, Jinming

Hajnal, Joseph V.

King, Andrew P.

Oksuz, Ilkay

Rueckert, Daniel

Schlemper, Jo

Schnabel, Julia A.

English

AUTOMAP is a promising generalized reconstruction approach, however, it is not scalable and hence the practicality is limited. We present dAUTOMAP, a novel way for decomposing the domain transformation of AUTOMAP, making the model scale linearly. We show dAUTOMAP outperforms AUTOMAP with significantly fewer parameters

Clough, James

University of Birmingham Research Portal

  University of BirminghamdAUTOMAPSchlemper, Jo; Oksuz, Ilkay; Clough, James R.; Duan, Jinming; King, Andrew P.; Schnabel,Julia A.; Hajnal, Joseph V.; Rueckert, DanielCitation for published version (Harvard):Schlemper, J, Oksuz, I, Clough, JR, Duan, J, King, AP, Schnabel, JA, Hajnal, JV & Rueckert, D 2019,dAUTOMAP: decomposing AUTOMAP to achieve scalability and enhance performance. in ISMRM 27th AnnualMeeting & Exhibition (Abstract #658).Link to publication on Research at Birmingham portalGeneral rightsUnless a licence is specified above, all rights (including copyright and moral rights) in this document are retained by the authors and/or thecopyright holders. The express permission of the copyright holder must be obtained for any use of this material other than for purposespermitted by law.•	Users may freely distribute the URL that is used to identify this publication.•	Users may download and/or print one copy of the publication from the University of Birmingham research portal for the purpose of privatestudy or non-commercial research.•	User may use extracts from the document in line with the concept of ‘fair dealing’ under the Copyright, Designs and Patents Act 1988 (?)•	Users may not further distribute the material nor use it for the purposes of commercial gain.Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document.When citing, please reference the published version.Take down policyWhile the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has beenuploaded in error or has been deemed to be commercially or otherwise sensitive.If you believe that this is the case for this document, please contact UBIRA@lists.bham.ac.uk providing details and we will remove access tothe work immediately and investigate.Download date: 28. Jun. 2022dAUTOMAP: decomposing AUTOMAP toachieve scalability and enhance performanceJo Schlemper1( ), Ilkay Oksuz2, James Clough2, Jinming Duan1, Andrew P.King2, Julia A. Schnabel,2 Jo V. Hajnal3, and Daniel Rueckert11 Biomedical Image Analysis Group, Imperial College London, UK{jo.schlemper11,d.rueckert}@imperial.ac.uk2 School of Biomedical Engineering and Imaging Sciences, King’s College London, UK{o.oksuz,j.clough,andrew.king,julia.schnabel}@kcl.ac.uk3 Imaging and Biomedical Engineering Clinical Academic Group, King’s CollegeLondon, UK{jo.hajnal}@kcl.ac.ukAbstract. AUTOMAP [3] is a promising generalized reconstructionapproach, however, it is not scalable and hence the practicality is limited.We present dAUTOMAP, a novel way for decomposing the domaintransformation of AUTOMAP, making the model scale linearly. Weshow dAUTOMAP outperforms AUTOMAP with significantly fewerparameters.1 IntroductionRecently, automated transform by manifold approximation (AUTOMAP) [3] hasbeen proposed as an innovative approach to directly learn the transformationfrom source signal domain to target image domain. While the applicability ofAUTOMAP to a range of tasks has been demonstrated, its practicality remainslimited as the required number of parameters scales quadratically with theinput size. We present a novel way for decomposing the domain transformation,which makes the model scale linearly with the input size. We term the resultingnetwork dAUTOMAP (decomposed - AUTOMAP). We show that, remarkably,the proposed approach outperforms AUTOMAP for the provided dataset withsignificantly fewer parameters.2 MethodsLet x ∈ CN×M be a complex-valued image. The two-dimensional Discrete FourierTransform (DFT) is given by:y[k, l] =N∑n=0M∑m=0x[n,m]e−j2π(nkN +mlM ) (1)Presented at ISMRM 27th Annual Meeting & Exhibition (Abstract #658)arXiv:1909.10995v2  [cs.LG]  25 Sep 20192 J. Schlemper et al.Fig. 1. The proposed network architecture of dAUTOMAP. The network takes k-spacedata on a Cartesian grid and directly recovers grid and directly recovers the underlyingimage. The fully-connected layers in AUTOMAP are replaced by decomposed transform(DT) layers, which are fully-connected along one axis. For two-dimensional input, weapply the DT layer twice, once for each axis. Each DT block is activated by ReLUnonlinearity. DT blocks are followed by a sparse convolutional autoencoder, as proposedin [3].This is commonly written as a matrix product: vec(y) = Evec(x), where we takerow-major vectorization and E = e−j2π(nkN +mlM ), with p = kM + l, q = nM + m.As the matrix E is the Kronecker product of two one-dimensional DFT’s, wehave:Evec(x) = (FN ⊗ FM ) vec(x) = vec(FNxFHM)= vec((FM (FNx)H)H), (2)where (FN )kn = e−j2π nkN , (FM )lm = e−j2πmlM . Observe that FNx can be com-puted using a convolution layer with N kernels of size (N, 1) with no paddingwhere the output tensor has size (Nbatch, N, 1,M). Motivated by this, we proposea decomposed transform layer (DT layer): a convolution layer with the abovekernel size, which is learnable. In the simplest case, the layer can be reducedto the (inverse) Fourier transform or identity. A 2D DFT can be performed byapplying the DT layer twice, where the intermediate tensor is first reshapedinto (Nbatch, 1, N,M) and then conjugate-transposed. Note that the complexnature of the operation is preserved by R2, which doubles the number of outputchannels (i.e. 2N). Therefore, the convolution kernel of the DT layer has theshape: (Ncout , Ncin , kernelx, kernely) = (2N, 2, N, 1). For the second DT layer, Nand M are swapped.The proposed dAUTOMAP, shown in Fig. 1, replaces the fully-connectedlayers in AUTOMAP by DT layers. We used ReLU as the choice of non-linearity.ISMRM Abstract #658: dAUTOMAP 3Table 1. Quantitative comparison between AUTOMAP and dAUTOMAP. In general,dAUTOMAP outperformed AUTOMAP for Mean Squared Error (MSE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and high frequency error norm(HFEN).Ngrid AF Undersampling Model MSE (×10−3) PSNR (dB) SSIM HFEN128 7 VDPAUTOMAP 3.27±1.17 22.07±1.35 0.76±0.03 0.54±0.07dAUTOMAP 1.76±0.47 24.67±1.14 0.82±0.02 0.39±0.03128 4 PoissonAUTOMAP 3.63±1.28 21.61±1.34 0.74±0.03 0.62±0.09dAUTOMAP 1.54±0.43 25.28±1.20 0.84±0.02 0.40±0.03128 2 CartesianAUTOMAP 2.81±1.35 22.84±1.64 0.80±0.03 0.42±0.07dAUTOMAP 1.01±0.39 27.2 ±1.51 0.89±0.02 0.27±0.05256 2 CartesianAUTOMAP n/a n/a n/a n/adAUTOMAP 0.51±0.25 30.31±1.81 0.91±0.02 0.29±0.043 EvaluationWe evaluated the proposed method on a simulation-based study using short-axis (SA) cardiac cine magnitude images from the UK Biobank Study[1] (>1MSA slices). To compare with AUTOMAP, the data were subsampled to central128×128 k-space grid points. Both methods were evaluated on the reconstructiontasks from three undersampling patterns: (1) Cartesian with Acceleration FactorAF=2, (2) Poisson with AF=4, (3) Variable density Poisson (VDP) with AF=7[2].For dAUTOMAP, we also experimented with the images having 256×256 k-spacegrid points, with 2× Cartesian undersampling.Both networks were initialised randomly and trained for 1000 epochs. We usedRMSProp with lr=2×10−5 and Adam optimiser lr=10−3 for AUTOMAP anddAUTOMAP respectively. The reconstructions were evaluated by mean squarederror (MSE), peak signal-to-noise ratio (PSNR), structural similarity (SSIM)and High Frequency Error Norm (HFEN). We also compared the reconstructionspeed and the required parameters.4 ResultsAs shown in Table 1, the proposed approach outperformed AUTOMAP (Wilcoxon,p0.01). Fig. 2 shows sample reconstructions. We notice that AUTOMAP tends toover-smooth the image, whereas dAUTOMAP preserves the fine-structure better,even though the residual artefact is more prominent. The result of dAUTOMAPfor 256×256 k-space data is shown in Fig. 3, demonstrating that the methodsuccessfully learnt a transform which simultaneously dealiases the image. Theexecution speeds were comparable (Table 2. The parameters of the proposedapproach required only 1.5MB of memory for 128×128 k-space data, compared to3.1GB required for AUTOMAP (these numbers increase to 3.1MB vs 56GB for256×256 k-space data).4 J. Schlemper et al.Fig. 2. Reconstruction of AUTOMAP and dAUTOMAP for different undersamplingpatterns and the resulting error maps. One can see that AUTOMAP tends to over-smooth the edges, which were preserved by dAUTOMAP better.Fig. 3. The sample reconstructions of dAUTOMAP from 256=256 Cartesian k-spacedata and the corresponding error maps. The majority of the artefacts were removed,however, some high-frequency aliasing can still be observed.5 Discussion and ConclusionIn this work, we propose a simple architecture which makes AUTOMAP scalable,based on the idea that the original Fourier kernels are linearly separable. WeISMRM Abstract #658: dAUTOMAP 5Table 2. Comparison of the number of parameters and execution speed.Ngrid =128 Ngrid =256Model #Parameters (×106) Speed (ms) #Parameters (×106) Speed (ms)AUTOMAP 806 0.36±0.36 13000 n/adAUTOMAP 0.37 0.48±0.10 1.16 0.50 ± 0.12experimentally found that such an approach yields superior performance inpractice, which is attributed to having significantly fewer parameters, makingit easier to train and less prone to overfitting. In future, we plan to investigatethe performance of non-Cartesian sampling strategies, which would requireregridding, or extensions to 3D data. Finally, the code is available on http://github.com/js3611/dAUTOMAP.6 AcknowledgementsJo Schlemper is partially funded by EPSRC Grant (EP/P001009/1).References[1] Petersen, S.E., Matthews, P.M., Francis, J.M., Robson, M.D., Zemrak, F., Bou-bertakh, R., Young, A.A., Hudson, S., Weale, P., Garratt, S., et al.: Uk biobankscardiovascular magnetic resonance protocol. Journal of cardiovascular magneticresonance 18(1), 8 (2015)[2] Uecker, M., Ong, F., Tamir, J.I., Bahri, D., Virtue, P., Cheng, J.Y., Zhang, T.,Lustig, M.: Berkeley advanced reconstruction toolbox. In: Proc. Intl. Soc. Mag.Reson. Med. vol. 23, p. 2486 (2015)[3] Zhu, B., Liu, J.Z., Cauley, S.F., Rosen, B.R., Rosen, M.S.: Image reconstruction bydomain-transform manifold learning. Nature 555(7697), 487 (2018)

dAUTOMAP:decomposing AUTOMAP to achieve scalability and enhance performance

http://pure-oai.bham.ac.uk/ws/files/148288993/1909.10995v2.pdf

dAUTOMAP:decomposing AUTOMAP to achieve scalability and enhance performance

Abstract

Similar works

Full text

Available Versions

University of Birmingham Research Portal