Source: FP8 Formats for Deep Learning

NVIDIA/Arm/Intel paper specifying the E4M3 and E5M2 8-bit floating-point formats.

Contents

Summary

Proposes two FP8 encodings and shows they match 16-bit training quality across CNNs, RNNs and Transformers up to 175B parameters, and that FP8 post-training quantization works for models that resist INT8.

Key claims

  • E4M3: 4 exponent bits, 3 mantissa bits; extends range by dropping infinities and using one NaN pattern.
  • E5M2: 5 exponent bits, 2 mantissa bits; follows IEEE 754 special-value conventions (wider range, less precision).
  • FP8 training matched 16-bit result quality with unchanged hyperparameters.
  • FP8 PTQ worked for language models that resisted INT8 quantization.

Feeds

Quantization · KV cache

Suggest changes →