Skip to main content
Article thumbnail
Location of Repository

On the Precision Attainable with Various Floating-Point Number Systems

By Richard P. Brenty

Abstract

1 Introduction A real number x is usually approximated in a digital computer by an element fl(x) of a finite set F of "floating-point " numbers. We regard the elements of F as exactly representable real numbers, and take fl(x) as the floating-point number closest to x. The definition of "closest", rules for breaking ties, and the possibility of truncating instead of rounding are discussed later. We restrict our attention to binary computers in which floating-point numbers are represented in a word (or multiple word) of fixed length w bits, using some convenient (possibly redundant) code. Usually F is a set of numbers of the for

Topics: Index Terms, Base, floating-point arithmetic, radix, representation error, rms error, rounding error, simulation
Year: 2009
OAI identifier: oai:CiteSeerX.psu:10.1.1.136.1144
Provided by: CiteSeerX
Download PDF:
Sorry, we are unable to provide the full text but you may find it at the following location(s):
  • http://citeseerx.ist.psu.edu/v... (external link)
  • http://wwwmaths.anu.edu.au/~br... (external link)
  • Suggested articles


    To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.