The Trilinos Project is an effort to facilitate the design, development,
integration and ongoing support of mathematical software libraries within an
object-oriented framework. It is intended for large-scale, complex multiphysics
engineering and scientific applications. Epetra is one of its basic packages.
It provides serial and parallel linear algebra capabilities. Before Trilinos
version 11.0, released in 2012, Epetra used the C++ int data-type for storing
global and local indices for degrees of freedom (DOFs). Since int is typically
32-bit, this limited the largest problem size to be smaller than approximately
two billion DOFs. This was true even if a distributed memory machine could
handle larger problems. We have added optional support for C++ long long
data-type, which is at least 64-bit wide, for global indices. To save memory,
maintain the speed of memory-bound operations, and reduce further changes to
the code, the local indices are still 32-bit. We document the changes required
to achieve this feature and how the new functionality can be used. We also
report on the lessons learned in modifying a mature and popular package from
various perspectives -- design goals, backward compatibility, engineering
decisions, C++ language features, effects on existing users and other packages,
and build integration