de ROUJOUX, Guillaume

Duguet, Florent

English

Archive Ouverte en Sciences de l'Information et de la Communication

© Altimesh 2016 – all rights reserved
Image Processing in Java
Running on GPU
Enabling Accelerators in Java
Automatic Kernel Generation
State-Of-The-Art Performances
© Altimesh 2016 – all rights reserved
OIL AND GAS: COHERENCE IMAGE PROCESSING 
ALGORITHM
Image processing use-case…
© Altimesh 2016 – all rights reserved
Coherence Image Processing Algorithm
• Coherence algorithm measures correlation between 
adjacent seismic traces. It is used to detect faults in 
the underground terrain.
• Coherence is one of the filters used by TOTAL S.A. for 
seismic image analysis.
• Another algorithm, Gradient Correlation Cube, is very 
similar computationally and developed by TOTAL S.A. 
(Patent: EP12306380.2)
• For each element in a neighborhood, the maximum 
correlation between shifted vertical slices is 
calculated.
• This algorithm is neither linear nor separable. 
• The filter parameters are the size of the vertical 
slice, the size of the neighborhood and the maximum 
explored shift.
© Altimesh 2016 – all rights reserved
Coherence Image Processing Algorithm
Input is a cube of scalar floating-
point values
Output is a cube of scalar floating-
points values
Filter parameters: w,ws,shift
ii
jj
shift
𝑎 𝑛 = 𝑑 𝑖, 𝑗, 𝑘 + 𝑛
w
𝑏 𝑛 = 𝑑 𝑖 + 𝑖𝑖, 𝑗 + 𝑗𝑗, 𝑘 + 𝑛 + 𝑠ℎ𝑖𝑓𝑡
i
j
k
Correlation [1]
Coherence Output [1]
𝑑𝑜𝑢𝑡 𝑖, 𝑗, 𝑘 =  
𝑖𝑖=−𝑤𝑠
𝑤𝑠
 
𝑗𝑗=−𝑤𝑠
𝑤𝑠
max
𝑠ℎ𝑖𝑓𝑡
 
𝑖,𝑗
[𝑖𝑖, 𝑗𝑗, 𝑠ℎ𝑖𝑓𝑡]
 
𝑖,𝑗
[𝑖𝑖, 𝑗𝑗, 𝑠ℎ𝑖𝑓𝑡] =  
𝑛=−𝑤
𝑤
𝑎 𝑛 − 𝑎
𝑎²
𝑏 𝑛 − 𝑏
𝑏²
[1] - Seismic Attributes for Prospect Identification and Reservoir Characterization, Satinder Chopra et Kurt J. Marfurt
© Altimesh 2016 – all rights reserved
Issues and Pitfalls
• In a brute-force approach, data is explored an immense 
number of times (several hundreds): 
– Neighborhood search
– Vertical slice size
– Vertical slice shift
• The main difficulty is to express the algorithm to 
reduce this data read overflow.
• We present here different approaches to reduce the 
number of data reads and maximize data reuse.
© Altimesh 2016 – all rights reserved
Why Java
• Java has been around for very long
• Advanced graphical user interfaces can be built
• Rich runtime for complete application implementation
– Network interaction
– File management
– Serialization
– GUI
• Advanced tools in IDE
– Unit testing
– Refactoring
• Garbage collected
• Efficient JIT
• Multiplatform
Java is used in TOTAL S.A. image 
processing software that hosts 
filters such as Coherence and 
Gradient Correlation Cube
© Altimesh 2016 – all rights reserved
OPTIMIZATION STEPS
From Offline Processing To Interactive Feedback
© Altimesh 2016 – all rights reserved
The Journey From Source Code To Binary
• One may think that the language used for development 
will enforce performance level of the resulting binary. 
This is much less true than we may think
• Java uses JIT which transforms java byte-code into 
binary, with advanced optimizations.
• This transformation can be taken further: re-expressing 
the byte-code in another language, and recompile
• Java
• DotNet
Source
• JVM
• MSIL
Byte-
Code
• C/C++
• CUDA/C
Other
Source
• X86
• PTXAS
Assembly
© Altimesh 2016 – all rights reserved
Naive Approach
• Code written by image processing researcher developing 
the algorithm: no platform-dependent optimization
• Loop on Z,Y,X; then ii, jj, and n
• Performance expressed in seconds on a 512x512x512 grid 
of data, in single precision
• TOTAL S.A. is willing to have interactive feedback on 
parameter changes
Java Cuda (GTX Titan)
Naive 7500 s 85.0 s
© Altimesh 2016 – all rights reserved
Grouping Loops
• Some loops traverse same data and output some aggregate 
value
– Merge loops on n to avoid multiple loading of same data
– Calculate average, norm and correlation in single loop
• Extract loop invariants
– Avoid recalculating costly operations
• Inverse is less expensive than global load
• Inverse is more expensive than register load
Java Cuda (GTX Titan)
Naive 7500 s 85.0 s
Grouping Loops 6750 s 47.7 s
© Altimesh 2016 – all rights reserved
Processing Several Items at a Time
• Processing several Z-planes in single pass
– Further reduce data loading
– At the expense of code readability
• This further improves benefits from Loop grouping 
approach
Java Cuda (GTX Titan)
Naive 7500 s 85.0 s
Grouping Loops 6750 s 47.7 s
MultiZ 2650 s 11.5 s
© Altimesh 2016 – all rights reserved
Mapping Arrays On Registers
• GPUs have a big flexible register file: up to 255 
registers per context
• Changing loops
– Does not reduce lisibility of code, yet much closer to 
original code than MultiZ
– Could have been a first draft
• Using stackalloc for temporary buffers
– Easily expressed in Java
• Using NVRTC to unroll loops completely: loop range 
parameter is a filter parameter (3 of them) => on the 
fly compilation yields register-only code !!
© Altimesh 2016 – all rights reserved
Java & NVRTC Build and Run Chain
Java code
•Algorithm
•Hybridizer annotations
•Identify compile – time 
constants
JVM Bytecode
•Regular javac
Cuda code
•Hybridizer
•Maven plugin
Java code
•Client
•Use nvrtc and driver API via 
runtime library
•Call native code using JNA
•Work distribution (grid and 
block sizes)
NVRTC
•Parameters passed as Cuda
preprocessor macros
•-D<parameter name>=<value>
•Allows unrolling
•Compilation time is 0.6 ms in 
this case (Core i7 Haswell)
No compromise on performance
•Keeping flexibility on 
parameters values
•Compilation can be cached to 
reduce overhead
Compile time
Runtime
© Altimesh 2016 – all rights reserved
Mapping Arrays On Registers
Time in seconds Java Cuda (GTX Titan) NVRTC (compile time)
Naive 7500 s 85.0 s
Grouping Loops 6750 s 47.7 s
MultiZ 2650 s 11.5 s
Registrified N/A 2.84 s (+) 0.6 s
Core i7 – 4771 @ 3.5 GHz
Nvidia GTX Titan (Kepler)
JDK 1.7.0_40
Cuda 7.5
Memory transfer ~ 0,2 s
© Altimesh 2016 – all rights reserved
Combining MultiZ and Registrified
Time in seconds Java Cuda (GTX Titan) NVRTC (compile time)
Naive 7500 s 85.0 s
Grouping Loops 6750 s 47.7 s
MultiZ 2650 s 11.5 s
Registrified N/A 2.84 s (+) 0.6 s
MultiZ Registrified N/A 1.07 s (+) 0.7 s
Core i7 – 4771 @ 3.5 GHz
Nvidia GTX Titan (Kepler)
JDK 1.7.0_40
Cuda 7.5
Memory transfer ~ 0,2 s
© Altimesh 2016 – all rights reserved
On usage of intrinsic functions
Using intrinsic (__frsqrt_rn) No intrinsic (1 / sqrtf)
• Using intrinsic might lead to a stalling pipeline
– Using intrinsic for frsqrt does not adequately utilize the 
resources of the processor
– intuitive code and fastmath compilation performs better
© Altimesh 2016 – all rights reserved
How Close From Peak Usage ?
• GTX 980 - CUDA 7.5
© Altimesh 2016 – all rights reserved
How Close From Peak Usage ?
• GTX 980 - CUDA 7.5
• Pipe Utilization: 
– Arithmetic: 89,6%
© Altimesh 2016 – all rights reserved
How Close From Peak Usage ?
• GTX 980 - CUDA 7.5
• Pipe Utilization: 
– Arithmetic: 89,6%
• Next in line:
– Instruction fetch ?
© Altimesh 2016 – all rights reserved
DISCUSSION
How Close Are We From Optimum ?
© Altimesh 2016 – all rights reserved
Theoretical Peak ?
operation Count / grid size sample OP Per entry 
FLOP
GLOPS
<> (2w+1) 11 ADD 1 1,44
<>² (2w+1) 11 FMA 2 2,89
1/SQRT (2w+1) 11 RSQRT 15 21,67
correl 1/2 * (1+2*ws)² * (1+2*(shift)) 269,5 FMA 22 1557,60
OPTIMAL FLOPS
1583,60
In our sample, we need to perform 1.584 TFLOP at the minimum. 
© Altimesh 2016 – all rights reserved
Theoretical Peak ?
operation Count / grid size sample OP Per entry 
FLOP
GLOPS
<> (2w+1) 11 ADD 1 1,44
<>² (2w+1) 11 FMA 2 2,89
1/SQRT (2w+1) 11 RSQRT 15 21,67
correl 1/2 * (1+2*ws)² * (1+2*(shift)) 269,5 FMA 22 1557,60
OPTIMAL FLOPS
1583,60
OPTIMAL NO SYMETRY
3141,20
In our sample, we need to perform 1.584 TFLOP at the minimum. 
Moreover, if we don’t benefit from symmetry, we need 3.14 TFLOP.
1/2
Symmetry
© Altimesh 2016 – all rights reserved
Theoretical Peak ?
GTX Titan Jetson TX1 GTX 850m GTX 980
Best time (s) 1.07 7.075 2.86 0.809
Compute Capability 3.5 5.3 5.0 5.2
Ratio peak (no sym) 45 % 87 % 95 % 84 %
• Hardly possible to use symmetry
– Average of max is not commutative 
• Naïve implementation would require very high bandwidth 
and low latency memory
Kepler
© Altimesh 2016 – all rights reserved
Thank You
florent.duguet@altimesh.com
guillaume.de-roujoux@altimesh.com
http://www.altimesh.com


Image Processing in Java Running on GPU

https://hal.archives-ouvertes.fr/hal-02334255/file/s6314-florent-duguet-image-processing-gpu.pdf

Image Processing in Java Running on GPU

Abstract

Similar works

Full text

Available Versions

Archive Ouverte en Sciences de l'Information et de la Communication