Text models
Opus 5.5 vs. Fable 5.1 for long tasks
Measure reliability and spending across a multi-stage assignment.
Define checkpoints
Use the same starting state and deliverables. At each checkpoint, record which requirements passed and whether the model can resume from saved work.
Include the cost of failed stages
Track retries, human correction, elapsed time, and tokens. Compare the total cost of the completed deliverable.