Floating-point arithmetic

Floating-point arithmetic represents numbers with a fixed number of significant digits and a scale factor. Rounding and finite range make its results differ from exact real-number arithmetic.

IEEE 754: A technical standard specifying binary and decimal floating-point formats, operations, rounding, and exceptional values. It defines the formats and operation rules used by most modern processors.
Positional notation: A number representation in which digit values depend on their positions and powers of a base. Floating-point encodings extend positional notation with a movable scale.
Numerical analysis: The study of algorithms for numerical problems, including their accuracy, stability, and computational cost. It analyzes how finite precision affects computed solutions.
Catastrophic cancellation: Severe loss of relative accuracy when subtracting nearly equal approximate numbers. It is a particularly damaging form of cancellation in floating-point calculations.
Significand: The component of a floating-point number that stores its significant digits. Its finite width limits how many digits an operation can preserve.
Binary numeral system: A base-two numeral system that represents values using the digits zero and one. Binary formats store significands and exponents using bits.
Scientific computing: The use of computational methods to solve scientific and engineering problems. Simulations and measurements routinely rely on approximate numerical operations.
Overflow: A numerical condition in which a computed value exceeds the largest magnitude representable in a format. Finite exponent range can turn an otherwise valid operation into infinity or an exception.
Exponent: The component of a floating-point number that scales its significand by a base raised to a power. It provides range by shifting the represented number's scale.
Machine epsilon: A measure of the spacing between one and the next larger representable number in a given format. It quantifies precision near one, though spacing varies with magnitude.

Linked from 31 pages