Elastography Using Multi-Stream GPU: An Application to Online Tracked Ultrasound Elastography, In-Vivo and the da Vinci Surgical System

<div><p>A system for real-time ultrasound (US) elastography will advance interventions for the diagnosis and treatment of cancer by advancing methods such as thermal monitoring of tissue ablation. A multi-stream graphics processing unit (GPU) based accelerated normalized cross-correlation (NCC) elastography, with a maximum frame rate of 78 frames per second, is presented in this paper. A study of NCC window size is undertaken to determine the effect on frame rate and the quality of output elastography images. This paper also presents a novel system for Online Tracked Ultrasound Elastography (O-TRuE), which extends prior work on an offline method. By tracking the US probe with an electromagnetic (EM) tracker, the system selects in-plane radio frequency (RF) data frames for generating high quality elastograms. A novel method for evaluating the quality of an elastography output stream is presented, suggesting that O-TRuE generates more stable elastograms than generated by untracked, free-hand palpation. Since EM tracking cannot be used in all systems, an integration of real-time elastography and the da Vinci Surgical System is presented and evaluated for elastography stream quality based on our metric. The da Vinci surgical robot is outfitted with a laparoscopic US probe, and palpation motions are autonomously generated by customized software. It is found that a stable output stream can be achieved, which is affected by both the frequency and amplitude of palpation. The GPU framework is validated using data from in-vivo pig liver ablation; the generated elastography images identify the ablated region, outlined more clearly than in the corresponding B-mode US images.</p></div

Timing graph to show speed comparison of multi-stream elastography (threaded) and non-stream elastography (normal).

<p>The graphs indicates run times and standard deviation of run time for window size 12, displacement 2 mm, overlap 98% (A, B) and Window size 16, displacement 4 mm, overlap 99% (C, D). The results are per 100 frames. The standard deviation is max 0.13 for Fig. (A), 0.122 for Fig. (B), 0.136 for Fig. (C), 0.167 for Fig. (D), which is very small for 100 frames. This graph also shows that the increased window size reduces the performance of the algorithm due to higher serial search within the large windows.</p

Percentage of consecutive frame pairs above a certain threshold of max correlation for varying σ values as described in eq. 2, 7, 10 and eq. 14.

<p>As an example () and buffer size 10 indicates percent of correlation values above the range 0.6 for the graph in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0115881#pone-0115881-g010" target="_blank">Fig. 10</a>. As can be seen in this table, in most cases, the quality of the output system improves with the increasing buffer size.</p><p>Percentage of consecutive frame pairs above a certain threshold of max correlation for varying σ values as described in eq. 2, 7, 10 and eq. 14.</p

Elastography image fusion.

<p>The images displayed in (a) is elastography image with single image (best O-TRuE) selection, (b) is elastography image for average of top 3 O-TRuE image selections, and (c) is elastography image for average of top 5 O-TRuE image selections. The results indicates that the fusion by averaging the top 5 elastography images from O-TRuE gives good quality indicated by the average CNR and SNR values of 1.327 and 2.210 respectively.</p

Real-time Online tracked Ultrasound Elastography (O-TRuE).

<p>Figure shows the real-time online tracked US elastography (O-TRuE) where the cost function is calculated from combinations of the tracked RF data. Then the elastography images are computed for the top <i>m</i> RF data pairs according to the Crr values. The elastography images can then be fused together by simply averaging the images or by weighted averaging based on average correlation values of each elastography image.</p

Trend of untracked elastography for in-vivo pig data: NCC window size vs. CNR and SNR.

<p>The graph shows variation of CNR and SNR of individual sample points for different NCC window sizes with untracked elastography. The data was obtained from in-vivo experiments on 350 samples and 199 samples were selected after ignoring invalid strain values. (A) Shows snapshot of CNR values and (B) shows snapshot of SNR values varying for a small subset of the 199 samples. The average/min/max values of the CNR and SNR are listed in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0115881#pone-0115881-t003" target="_blank">Table 3</a>. The CNR and SNR across different window sizes are closely related per sample but the global variation in CNR and SNR is high due to wide range of values.</p

Test results for comparing frame rate performance of multi-stream GPU elastography (threaded) with single-stream (streamed) and non-stream (normal) GPU elastography.

<p>This table reports average frames per second (with standard deviation in brackets) of images generated by various versions of the elastography program. The term <i>normal-N</i> indicates the basic GPU implementation of NCC elastography, <i>streamed-N</i> indicates the streamed GPU implementation, and <i>threaded-N</i> indicates the multi-streamed GPU implementation, where <i>N</i> indicates the number of RF lines in each RF image. Four test cases were performed at different NCC window sizes, NCC maximum search distances (displacements), and NCC search step sizes (specified as percentage of window overlap). The computational load increases with larger window size, displacement, and percent overlap. As seen in the results, the highest speed obtained is 78 frames per second (fps) running the multi-streamed GPU implementation.</p><p>Test results for comparing frame rate performance of multi-stream GPU elastography (threaded) with single-stream (streamed) and non-stream (normal) GPU elastography.</p

Selection map of O-TRuE images.

<p>The row above each image sequence indicates the RF data pair index. For e.g. the pair identifier (n1, m1) indicates comparison of radio frequency (RF) data frame acquired at time with that of the frame acquired at time . The pair (image rank, Crr value) below the image sequence indicates the rank and Crr value generated by O-TRuE. The pair (CNR, SNR) indicates contrast-to-noise ratio and signal-to-noise ratio values for each image. O-TRuE selected 90% good elastography images in top 20 ranked images with good CNR and SNR above 0.51 and 2.37 respectively. The Crr above 0.457 is observed to provide with good elastography images.</p

Elastography image stream analysis of consecutive frames in O-TRuE and Untracked elastography.

<p>An analysis of consecutive frames is done to understand the quality of strain images generated by O-TRuE and untracked elastography. (A) Shows a template region selected in the leftmost image and a target region selected in the rightmost image. We apply normalized cross-correlation in these regions as shown in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0115881#pone.0115881.e048" target="_blank">eq. 13</a> to find max correlation value. A max correlation graph for 100 elastography image pairs is shown in (B), where the red dashed line is for O-TRuE and a blue dotted line is for untracked elastography. O-TRuE has a more consistent high correlation value across consecutive images. As indicated in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0115881#pone-0115881-t002" target="_blank">Table 2</a>, O-TRuE (β values) performs better than untracked elastography. (C) Shows the dataset for frames in range [51, 60]; here O-TRuE has its lowest cross-correlation value from 53 to 54; as can be seen, the image quality drastically changes in this range.</p

Elastography Server.

<p>This figure shows real-time pipeline where data is acquired through a radio-frequency (RF) server which runs on a US machine. As can be seen, a combination of queue and threading mechanism is implemented to connect all the components efficiently. Queuing mechanism allows the receiver and processing threads to work independently. The processing threads sleep if there is no data available to process and are triggered by data receiving component whenever data is ready. Elastography threads are the multiple threads that are spawned per consecutive or selected pair of RF data received. Every thread can send out the data over the network using IGTLMessages. The n^th thread can collect data from all the other n-1 threads to perform aggregate operations as averaging or weighted averaging of selective elastography images.</p