Although several thousands of exoplanets have now been detected and
characterized, observational biases have led to a paucity of long-period,
low-mass exoplanets with measured masses and a corresponding lag in our
understanding of such planets. In this paper we report the mass estimation and
characterization of the long-period exoplanet Kepler-538b. This planet orbits a
Sun-like star (V = 11.27) with M_* = 0.892 +/- (0.051, 0.035) M_sun and R_* =
0.8717 +/- (0.0064, 0.0061) R_sun. Kepler-538b is a 2.215 +/- (0.040, 0.034)
R_earth sub-Neptune with a period of P = 81.73778 +/- 0.00013 d. It is the only
known planet in the system. We collected radial velocity (RV) observations with
HIRES on Keck I and HARPS-N on the TNG. We characterized stellar activity by a
Gaussian process with a quasi-periodic kernel applied to our RV and cross
correlation function full width at half maximum (FWHM) observations. By
simultaneously modeling Kepler photometry, RV, and FWHM observations, we found
a semi-amplitude of K = 1.68 +/- (0.39, 0.38) m s^-1 and a planet mass of M_p =
10.6 +/- (2.5, 2.4) M_earth. Kepler-538b is the smallest planet beyond P = 50 d
with an RV mass measurement. The planet likely consists of a significant
fraction of ices (dominated by water ice), in addition to rocks/metals, and a
small amount of gas. Sophisticated modeling techniques such as those used in
this paper, combined with future spectrographs with ultra high-precision and
stability will be vital for yielding more mass measurements in this poorly
understood exoplanet regime. This in turn will improve our understanding of the
relationship between planet composition and insolation flux and how the rocky
to gaseous transition depends on planetary equilibrium temperature