How I cut our multi-agent pipeline cost by 64% with cache forensics

The setup: 8 agents, $22.51 per run

Comis runs multi-agent pipelines where multiple AI agents collaborate on complex tasks. Our TradingAgents pipeline is a good example: 4 specialist analysts (fundamentals, sentiment, news, technical) run in parallel, feed into a bull-bear debate, then a trader makes a call, risk management reviews it, and a portfolio manager gives the final verdict.

That's 8 agents, each making 2-7 LLM calls, all running on Claude Opus 4.6. Without any caching, 4 million prompt tokens at $5/MTok input rate = $22.51 per run. We run this pipeline multiple times per day.

Anthropic's prompt caching is supposed to fix this. You pay a premium to write content to the cache ($6.25-10/MTok), but subsequent reads are dirt cheap ($0.50/MTok) - that's 10-20x cheaper. The theory is beautiful: write the system prompt once, read it on every subsequent call.

The reality was uglier. Our dashboard was showing a 1.19x read/write ratio. We were barely breaking even on the cache.

Down the rabbit hole

I started where any good debugging session starts: the logs. Comis has structured Pino logging throughout the pipeline, so I could trace exactly what was happening on each LLM call.

Turn 1: read=     0 write=65,318 ratio=0.0x  (cold start, expected)
Turn 2: read=92,232 write=43,268 ratio=2.1x  (decent)
Turn 3: read=53,828 write=85,352 ratio=0.6x  ← disaster

Turn 3 was the killer. 85,352 tokens of cache writes - that's more than the entire prompt. The cache was being completely destroyed and rebuilt. But why?

I dug into the context engine pipeline logs. And there it was:

Context engine pipeline complete:
  tokensMasked: 36,173
  thinkingBlocksRemoved: 6
  cacheFenceIndex: N/A

36,173 tokens were being masked by the observation masker. And cacheFenceIndex: N/A - there was no fence protecting the cached prefix.

Here's what was happening: the context engine has 8 layers that process messages before each LLM call. Four of those layers modify message content - stripping thinking blocks, stripping inline reasoning tags, evicting dead content, masking old tool results. When they modify messages that are part of the cached prefix, Anthropic's KV cache is invalidated. The entire prefix has to be re-cached. That's the 85K write.

The "aha" moment: three bugs, not one

What started as "the cache isn't working" turned into discovering three separate bugs, each building on the last. I think that's what makes this story interesting - the fix wasn't obvious, and each layer of the onion revealed a deeper problem.

Stored:  fence = 52 (post-CE space)
Trimmed: 99 messages removed from front
Adjusted: max(-1, 52 - 99) = -1 ← fence obliterated!

The results

Three fixes, shipped as quick tasks (220, 221, 222) over the course of one evening. Each one testable independently, each one with measurable impact on the next pipeline run.

What I learned

The architecture in a nutshell

If you're building multi-agent systems on top of Anthropic's API, here's the cache architecture that emerged from this debugging session:

The key insight: cache optimization in multi-agent systems isn't just about placing breakpoints. It's about protecting the prefix from your own processing pipeline. Every framework that processes messages before sending them to the API has this same problem - they just don't know it yet.

Ship, measure, iterate

The whole debugging session - from "why is the cache ratio 1.19x" to three shipped fixes with production validation - took one evening. Each fix was a focused quick task with atomic commits, diagnostic logs added before the fix and cleaned up after. The observability logs I added during debugging became the permanent monitoring system.

If you're running multi-agent pipelines on Anthropic's API and your cache ratio is below 2x, there's a good chance your context processing is silently invalidating the cache. Add cacheFenceIndex to your logs. You might be surprised what you find.

I thought I was done. I was wrong.

Three days later: the full forensic audit

The cache fence fixes brought our ratio from 1.19x to 2.90x. Decent. But I had a nagging feeling there was more to find. So on April 10th I did something I probably should have done from the start: I audited every single token.

I pulled the full daemon log (945 lines of structured JSON), all 13 workspace session JSONL files, all 12 sub-agent result JSONs, the graph run metadata, and cross-referenced everything against the Anthropic dashboard's hour-level token breakdown. Token by token.

Input:            72 vs 72       exact match
Output:        24,867 vs 24,867   exact match
Cache write 1h: 86,993              (38% of writes)
Cache write 5m:141,861              (62% of writes)
Cache read:    742,514 vs 742,514 exact match
Total:        996,307 vs 996,307  zero delta

Token accounting: perfect. Zero leaks. But that dashboard tooltip revealed something I didn't have in my logs: the per-TTL breakdown of cache writes. Anthropic was showing me exactly which tokens used 5-minute TTL vs 1-hour TTL. And that cracked open five more issues.

Five issues hiding behind "it works"

The pricing math nobody talks about

The audit led me to build a break-even model for Anthropic's cache. Here's the thing most people miss: 1-hour cache writes are significantly harder to justify than 5-minute ones.

5m writes pay for themselves if you read just 28% of what you wrote. That happens on the very next API call. But 1h writes need you to read more tokens than you wrote - which only happens for content that's read dozens of times (like tool definitions across a 42-call pipeline).

This is why the adaptive strategy works: start "short" for everything, then promote to "long" only for content you've proven gets high read-back. In our pipeline, tool definitions get read 40+ times (18.9x ROI on 1h writes). Conversation content gets read 1-3 times (net negative on 1h, break-even on 5m).

The full picture: 42 API calls, 996K tokens, $2.75

Here's what the full audit revealed across the complete TradingAgents session - 12 sub-agents analyzing MSFT, 42 API calls total:

What I actually learned (the sequel)

The plot twist: a 5-line fix worth more than all the others combined

A day after the full audit, I deployed all the fixes: adaptive retention, stagger increase, timing patches, breakpoint threshold tuning. I ran the pipeline and checked the Anthropic dashboard:

Prompt caching write (1h):  28,220  ← first time EVER
Prompt caching write (5m): 163,089
Prompt caching read:       790,336
Total:               1,003,941

For the first time in the project's history, 1-hour cache writes appeared. Before this fix, every single cache write across hundreds of pipeline runs had been at 5m. The "long" retention we'd been setting on tool breakpoints for weeks was being silently ignored.

Here's what I found: Anthropic's API requires cache breakpoint TTLs to be monotonically non-increasing across the request payload. The order is: system → tools → messages. Our SDK sets system: {type: "ephemeral"} (5m). After adaptive escalation, our code sets tools: {type: "ephemeral", ttl: "1h"}. That creates:

system(5m) → tools(1h) → messages(5m)
        ^^^^
        VIOLATION: 5m < 1h is an increase

The API silently downgrades the tool breakpoint to 5m. Our entire adaptive retention system - the escalation, the model tracking, the session-scoped warm state - was doing nothing at the API level. Every cache entry was 5m.

The fix: in our onPayload hook, when tool retention is "long", also upgrade the system blocks' cache_control TTL to match:

// 5 lines that unlocked the entire 1h cache feature
if (resolvedRetention === "long" && Array.isArray(params.system)) {
  for (const block of params.system) {
    if (block.cache_control) {
      block.cache_control = { type: "ephemeral", ttl: "1h" };
    }
  }
}

Now: system(1h) → tools(1h) → messages(5m). Monotonically non-increasing. The API honors the 1h TTL. Cache survives 6-minute pipeline gaps.

But then the second discovery: cache refreshes don't change the TTL of existing entries. Content written at 5m stays at 5m even if subsequent requests send ttl: "1h". The 1h only applies to the initial write. So the parent agent's cold start had to use 1h from the very first call - not 5m with a later escalation. One more line change: parent agents start at configRetention ("long") instead of "short". Sub-agents use mixed 5m/1h - the shared prefix inherits 1h from the first writer via CCPAT-03.

The result on the next pipeline run:

Prompt caching write (1h):       0  ← previous run's 1h cache reused!
Prompt caching write (5m): 154,796
Prompt caching read:       768,362
Total:                     947,721
Cost/MTok:                   $2.07  (vs $2.33 baseline)

Zero 1h writes. The previous run's 1h cache survived 11 minutes between runs and got reused entirely. The parent agent's first call cost $0.03 instead of $0.19. The post-pipeline execution - which previously re-wrote 42K tokens due to 5m cache expiry - now read 103K tokens from the surviving cache and only wrote 12K of genuinely new content.

The cache optimization never ends

The first round (cache fence, sub-agent breakpoints, index translation) took one evening and cut costs 44%. The full audit three days later found five more issues. And the day after that, the TTL monotonicity discovery - 5 lines of code that unlocked the entire 1h cache feature for the first time. Together: 64% cost reduction from $5.48 to $1.96 per pipeline run at steady state.

The lesson isn't "we found 10 cache bugs." It's that prompt caching in multi-agent systems is a continuous optimization problem, not a feature you turn on and forget. The API gives you the primitives. Your SDK adds its own cache markers. Your code adds more. And the API has silent constraints (like TTL monotonicity) that can make all of it do nothing. The only way to know is to audit every token - and then check the dashboard to see if the API actually did what you asked.