machine-learning

On a fraud-scoring team (~450 RPS peak, ml.g5.2xlarge), inference components plus April 2026 capacity-aware instance pools cut endpoint provisioning failures from 6 retries to 0 — p99 held at 68 ms through a g5→g6 fallback event.

Cheapest Spot pool is often the unstable one. July 2026: price-capacity-optimized + attribute-based selection, stability scoring, interruption hygiene — before you chase $/hr.