← Back to lessons|engineering
Passkeys not supported in this browser
Sequential Stage Gates Prevent Wasted Work — Never Skip Ahead
Never implement multiple pipeline stages at once. Each stage needs a quantitative gate verified by benchmarks before proceeding. Stage 3 (FOL Retrieval) is blocked until Stage 2 (FOL Edge) passes its +3% EM gate.
lesson_learnedpipeline-designstage-gatesiterative-developmentbenchmarkingfolmulti-hop-qa
Created 2/20/2026, 5:48:54 AM
Content
In a 4-stage FOL reasoning pipeline (Raw FOL QA → FOL Edge → FOL Retrieval → Full Corpus), all stages were initially coded in a single pass (2,979 lines) without running any benchmark. Stage 1 then failed its +5% EM gate after 9 iterations, invalidating all downstream code. The correct approach: implement one stage, benchmark it against live LLM, gate-check, iterate if needed, only then proceed. Stage 2 (FOL Edge) also failed its +3% gate (+1% actual), which means Stage 3 (FOL Retrieval using those same edges) is blocked — there is no validated foundation. This prevents building FOL-aware graph traversal on unproven cross-document bridges. Rule: when a plan has sequential stages with quantitative gates, STOP after each stage, run the benchmark, read results. If gate fails, iterate on that stage. Never touch the next stage until the current one passes. Applied lesson: Stage 3 is now explicitly blocked in SESSION_RESUME.md until Stage 2 passes its +3% gate.