Math Fundamentals18 sections · 814 units
Open in Course

Precision and Rounding

(Floating point issues)

Geometric calculations often involve square roots and divisions, which produce floating-point numbers. Comparing floats directly can fail due to precision errors.

Instead of checking if a == b, check if |a - b| < epsilon, where epsilon is a small tolerance like 1e-9.

Some problems avoid floats entirely by working with squared distances. Instead of computing √(x² + y²), compare x² + y² directly.