Instruction set randomization (ISR) was initially proposed with the main goal
of countering code-injection attacks. However, ISR seems to have lost its
appeal since code-injection attacks became less attractive because protection
mechanisms such as data execution prevention (DEP) as well as code-reuse
attacks became more prevalent.
In this paper, we show that ISR can be extended to also protect against
code-reuse attacks while at the same time offering security guarantees similar
to those of software diversity, control-flow integrity, and information hiding.
We present Scylla, a scheme that deploys a new technique for in-place code
encryption to hide the code layout of a randomized binary, and restricts the
control flow to a benign execution path. This allows us to i) implicitly
restrict control-flow targets to basic block entries without requiring the
extraction of a control-flow graph, ii) achieve execution integrity within
legitimate basic blocks, and iii) hide the underlying code layout under
malicious read access to the program. Our analysis demonstrates that Scylla is
capable of preventing state-of-the-art attacks such as just-in-time
return-oriented programming (JIT-ROP) and crash-resistant oriented programming
(CROP). We extensively evaluate our prototype implementation of Scylla and show
feasible performance overhead. We also provide details on how this overhead can
be significantly reduced with dedicated hardware support