Skip to main content
Article thumbnail
Location of Repository

On the Precision Attainable with Various Floating-Point Number Systems

By Richard P. Brenty


1 Introduction A real number x is usually approximated in a digital computer by an element fl(x) of a finite set F of "floating-point " numbers. We regard the elements of F as exactly representable real numbers, and take fl(x) as the floating-point number closest to x. The definition of "closest", rules for breaking ties, and the possibility of truncating instead of rounding are discussed later. We restrict our attention to binary computers in which floating-point numbers are represented in a word (or multiple word) of fixed length w bits, using some convenient (possibly redundant) code. Usually F is a set of numbers of the for

Topics: Index Terms, Base, floating-point arithmetic, radix, representation error, rms error, rounding error, simulation
Year: 2009
OAI identifier: oai:CiteSeerX.psu:
Provided by: CiteSeerX
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • (external link)
  • (external link)
  • Suggested articles

    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.