Agent Observability: The Datadog for Parallel Coding Agents Gap
The premise: parallel agents multiply the bill
Pulpo’s overview describes coding agents as “background workers — and a quota-and-cost multiplier,” and warns that “a weekly subscription allowance can vanish in an afternoon” when several run at once. Runaway spend is a recognized operational concern, not only a dashboard-design question.
Vendor usage pages do not necessarily give a team-wide view. Pulpo characterizes a vendor’s /usage page as “one account, one machine, one vendor, shown after the fact,” and says vendors do not aggregate spend across a user’s accounts. These are limitations as Pulpo describes them, not an independent survey of every vendor.
The claim under test is that “no clean OSS player exists” for observing and budgeting parallel coding agents. This post tests it against the documentation and repository pages in the research set: tool by tool, then Claude Code’s native OpenTelemetry route, a comparison table, and a budget-guard pattern built from documented primitives.
Testing the premise
The projects differ in depth. Some describe features in a README; others give a product claim or a listing. The notes below flag the difference.
Pulpo: metering, caps, and session worktrees
Covered: Pulpo says it reads exact token counts from agent session files, prices them with a rate table, and rolls them up “attributed per session and rolled up per repo.” Its docs describe per-session and per-schedule cost caps that “alert at 80% and stop at 100%, recorded as an intervention you can audit.” For parallel work it provides “per-session git worktrees so parallel agents never collide.” It describes itself as self-hosted, and events can go out by webhook: “your Grafana/Datadog/SIEM is the dashboard.” All of this is on its overview page, and its source is on GitHub.
Missing or unverified: The overview does not show a configuration key for its caps. Attribution is per session and per repo, not per worktree. Because its worktrees are per session, a session-to-worktree mapping may approximate per-worktree spend, but the docs do not present that as a feature. Codex reports an exact quota rather than a per-token cost. No licence appears in the sources reviewed, so “self-hosted” does not establish “open source.”
warden: Apache-2.0, deprecated, an opt-in pre-spawn gate
Covered: The warden README states it is “Licensed under the Apache License, Version 2.0” and links its LICENSE file. Write-type agents get their own git worktree by default. warden usage spend prices each agent’s model usage into dollars and rolls it up per agent, repo and day, with pricing documented for the Claude backend (other backends report tokens). Its budget gate is off by default. When enabled with budget_daily_usd or budget_weekly_usd and budget_gate: true, “a spawn that would push measured spend over the cap warns first (re-run with --force to proceed).” It applies to new spawns.
Missing or unverified: The README says warden is deprecated: “It is no longer developed, and new work happens in wdx.” The sources do not show per-worktree cost. The repository showed 1,569 commits at our snapshot, a repository signal only.
cezar: MIT, per-task worktrees, shared budgets
Covered: cezar’s README is marked MIT (“MIT © Patryk Lewczuk”) and calls itself “Free and open source.” Every task gets its own git worktree, and the README says a running task shows “agent text, tool calls, tokens and cost.” It states that “cezar limits a parent to four children in flight; when a parent has a budget, its children share that budget.” Its Usage & cost view compares reported spend “across projects,” with a project breakdown. The repository showed 1,883 commits at our snapshot.
Missing or unverified: The README describes a shared budget but no stop when it is exceeded. Cost is shown live per task and rolled up by project; the README does not say cost is recorded per worktree.
AgentGrid: MIT, a stop alert, not a documented stop
Covered: AgentGrid’s README is MIT-licensed. It describes a dashboard that runs Claude Code, Gemini CLI and Codex in parallel, with “Git Worktree Isolation” giving each agent “an independent working branch,” and a “Real-time display of all agents’ status, cost, and logs.” Its config sets alertThreshold (18.0 USD, “Show a warning banner”) and stopThreshold (20.0 USD, “Show a stop alert when this amount (USD) is reached”).
Missing or unverified: The README describes a stop alert, not a process stop. The repository listing showed 7 commits at snapshot.
amux: a budget-kill tip, MIT with a Commons Clause
Covered: The amux guide, from the project’s site, describes per-session token spend and gives the tip “Set cost caps: Configure amux to kill sessions that exceed a per-task budget.” It registers each agent to a worktree directory (amux register agent-$i --dir ../myproject-agent-$i). The guide calls amux one of “the main open-source options,” and its own comparison table lists the licence as “MIT + Commons Clause.” Author’s note: the Commons Clause restricts commercial resale, and it is not generally treated as open source. That is not stated in the sources.
Missing or unverified: The guide gives no configuration for the budget kill, so how it works cannot be confirmed.
Skoga: named agents and a live budget display
Covered: Skoga describes one sandboxed worktree per agent and “token spend against your daily budget, plotted live.”
Missing or unverified: The page shows a budget display but does not say it enforces a cap or stops an agent, and it gives no per-worktree cost. It is listed as “Private beta” and “coming soon” for macOS, which is not evidence of generally available software.
ccusage and local cost readers
These appear only in Pulpo’s comparison table, described as single-machine, read-only cost displays. That is a secondhand description, not an independently reviewed feature set.
The narrowed verdict
The literal claim that “no clean OSS player exists” is not supported as stated. cezar (MIT) gives each task its own worktree and live cost. amux (MIT + Commons Clause) advises a per-task budget kill. warden (Apache-2.0) gives per-agent spend and an opt-in gate, and the tool itself is deprecated.
The narrower finding is this: no source in this set documents one tool that combines (a) an OSI-open licence, (b) a stop with a shown configuration, and (c) cost recorded per worktree. Pulpo documents a stop at 100% with no licence stated. cezar has (a) and live per-task cost, with no stop described. amux has (a) with a caveat and an unconfigured stop. warden has (a) and a pre-spawn gate, which is opt-in; the tool itself is deprecated.
Claude Code’s vendor-native OpenTelemetry route
Before adding another product, Claude Code’s monitoring documentation describes exporting metrics into a user’s own telemetry stack: metrics, then an exporter, then your alerting. The documented metrics include:
| Metric | Documented unit |
|---|---|
claude_code.session.count |
None |
claude_code.lines_of_code.count |
None |
claude_code.pull_request.count |
None |
claude_code.commit.count |
None |
claude_code.cost.usage |
USD |
claude_code.token.usage |
tokens |
claude_code.code_edit_tool.decision |
None |
claude_code.active_time.total |
s |
The documentation names these attributes for segmentation: session.id, ccr.session.id, skill.name, plugin.name, agent.name, model, query_source, user.account_uuid, user.account_id, app.version, and app.entrypoint. The model attribute is available on claude_code.token.usage, claude_code.cost.usage, and claude_code.lines_of_code.count.
Cardinality can be managed with documented environment variables:
| Environment variable | Included attribute(s) | Default |
|---|---|---|
OTEL_METRICS_INCLUDE_SESSION_ID |
session.id, ccr.session.id |
true |
OTEL_METRICS_INCLUDE_VERSION |
app.version |
false |
OTEL_METRICS_INCLUDE_ACCOUNT_UUID |
user.account_uuid, user.account_id |
true |
OTEL_METRICS_INCLUDE_ENTRYPOINT |
app.entrypoint |
false |
The docs also describe OTEL_RESOURCE_ATTRIBUTES for custom labels, with an example of department=engineering,team.id=platform,cost_center=eng-123. Those keys are attached to metric datapoints unless OTEL_METRICS_INCLUDE_RESOURCE_ATTRIBUTES=false, and each one “becomes a label on every metric series, so high-cardinality values increase storage cost in your metrics backend.” The docs also describe vcs.* repository attributes, such as vcs.repository.name. They describe the repository, not a worktree. Both the OTEL_METRICS_INCLUDE_RESOURCE_ATTRIBUTES variable and the vcs.* attributes require Claude Code v2.1.269 or later.
The documentation says that when prometheus is the only exporter listed in OTEL_METRICS_EXPORTER, the USD, tokens, and s units are omitted. Its cost warning is direct: “Cost metrics are approximations.” For official billing data it points to Claude Code’s organization cost documentation. Streaming responses count toward cost and token metrics exactly once, including through a gateway or proxy behind ANTHROPIC_BASE_URL. A third-party walkthrough from Dash0 covers the same export route; it is not vendor documentation.
There are attribution caveats. Session counts can rise without adding an active user when one person starts many sessions. Per-model commit breakdowns require a join on session.id, because one session can span multiple models; the docs recommend filtering the cost or token side to query_source = "main" so auxiliary or subagent requests are not assigned to the wrong model.
Claude Code’s docs suggest alerts for cost spikes, unusual token consumption, and high session volume from specific users. The documentation reviewed describes alerting, not budget enforcement. That is a statement about the documentation, not a claim that enforcement is impossible.
What the sources establish side by side
| Tool | Link | Parallel worktrees? | Cost tracking | Budget cap/stop | Attribution granularity | Licence / status |
|---|---|---|---|---|---|---|
| Pulpo | Pulpo | Yes; per-session worktrees | Session-file tokens, rate table | Alert at 80%, stop at 100%; no config shown | Per session, rolled up per repo | Self-hosted; licence not stated |
| Claude Code monitoring | Documentation | Not documented | Approximate cost metric | Alerting described; enforcement not documented | Session, account, skill, agent, model; custom labels via env | Vendor feature |
| warden | Repository | Yes; write agents by default | Per agent, repo and day; Claude pricing | Opt-in gate, off by default; warns before spawn, --force overrides |
Per agent | Apache-2.0; deprecated for wdx |
| cezar | Repository | Yes; a worktree per task | Live per-task tokens and cost; project rollups | Shared parent budget; stop not described | Per task display; project rollups | MIT |
| AgentGrid | Repository | Yes; isolated branch per agent | Cost meter | Stop alert threshold only | Per agent display | MIT |
| amux | Guide | Yes; agent registered to a worktree directory | Per-session token spend | Tip: kill sessions over a per-task budget; no config | Per session | MIT + Commons Clause |
| Skoga | Skoga | Yes; one sandboxed worktree per agent | Token spend against daily budget | Display only | Named agents; per-worktree cost not stated | Private beta; coming soon (macOS) |
“Not documented” or “not stated” means the reviewed source does not establish the point. No row combines a stop with a shown configuration, per-worktree cost and an OSI-open licence.
A budget-guard pattern you can assemble
These steps combine documented primitives. Where a step needs a join or design choice no source states, it is labelled as an inference.
- Meter at the source. Use Claude Code’s OTel metrics, or a runner that reads agent session files directly, as Pulpo does. These are two documented routes, not a claim that they produce identical data.
- Segment by session or agent. Claude Code exposes
session.id,agent.name,skill.nameandmodel. Choose the dimensions your workflow can reliably associate with each run. - Tag each worktree. Inference: set
OTEL_RESOURCE_ATTRIBUTESper run with a worktree identifier. Because a per-worktree label is a high-cardinality value, weigh its storage cost, or send it on the resource block only withOTEL_METRICS_INCLUDE_RESOURCE_ATTRIBUTES=false. The docs document custom resource attributes but do not describe worktree tagging. - Approximate worktree spend with a session-to-worktree mapping. Pulpo documents per-session worktrees and cost; amux registers each agent to a worktree directory. Inference: joining these records approximates per-worktree spend. No reviewed source presents this as a built-in cost-attribution feature.
- Send events to the existing stack. Claude Code documents OTel metrics. Pulpo documents webhooks as plain POST events to destinations such as Grafana, Datadog or a SIEM. The sources do not establish a particular dashboard configuration.
- Alert on documented patterns; choose thresholds for your system. Claude Code suggests alerts for cost spikes, unusual token use and high session volume. Pulpo documents one concrete example: alert at 80%, stop at 100%.
- Put enforcement where the session runs. Pulpo’s stated principle is that “only the thing actually running the session can stop a runaway before the wall.” Treat a dashboard alert and a runner-level stop as separate controls.
FAQ
Is there really no OSS “Datadog for agents”?
The sources do not support “no clean OSS player” as stated. cezar is MIT and gives each task a worktree with live cost. amux is MIT with a Commons Clause and advises a per-task budget kill. warden is Apache-2.0 with per-agent spend, but its gate is opt-in and it is deprecated. Pulpo documents a self-hosted meter and a stop at 100%, with no licence stated. What the sources do not show is one tool that combines a stop with a shown configuration, per-worktree cost and an OSI-open licence.
Are Claude Code’s cost numbers billing-accurate?
The documentation says, “Cost metrics are approximations,” and points to its organization cost documentation for official billing data. Streaming responses are counted once in cost and token metrics, including through a gateway or proxy. Claude Code monitoring docs
Can the OTel route stop a runaway?
The Claude Code documentation reviewed describes metrics and alert suggestions, not budget enforcement. That does not establish that enforcement is impossible; the reviewed source does not document it.
Can I get cost per worktree today?
No reviewed source documents cost recorded per worktree as a built-in view. Three routes approximate it: Pulpo’s per-session cost rollups with session worktrees, cezar’s live per-task cost display with per-task worktrees, and a worktree label set through OTEL_RESOURCE_ATTRIBUTES (an inference). Each needs a mapping you maintain.
What about Codex?
Pulpo’s documentation says Codex reports an exact quota rather than a per-token cost. That qualification is specific to Pulpo’s described metering.
Is Skoga usable now?
The page describes it as “Private beta” and “coming soon” for macOS. Those are the availability signals in the reviewed material, not an independent status check.
What is ccusage?
In this evidence set, ccusage is described only secondhand, in Pulpo’s comparison table, as a single-machine, read-only cost display. Its capabilities were not independently verified.
Why can’t a vendor dashboard do this?
Pulpo describes vendor /usage pages as one account, one machine, one vendor, shown after the fact, and says vendors do not aggregate spend across a user’s accounts. That is Pulpo’s explanation of the limitation, not a verified statement about every vendor’s current product.
The Bottom Line
The gap is real, but it is not dashboarding. Claude Code’s OTel metrics and Pulpo’s webhooks feed an existing observability stack. What remains unclear is attribution and enforcement. cezar and amux put cost and budgets at the task level, but the sources do not show a configurable stop tied to a worktree in an OSI-licensed tool. Pulpo documents a stop with no licence stated, warden’s gate is opt-in and warns before a spawn, AgentGrid’s threshold is a display alert, and Claude Code’s docs describe alerting rather than enforcement.
Restated: the sources show partial matches across several tools, and none combines a stop with a shown configuration, per-worktree cost and an OSI-open licence. That is a finding about this evidence set, not proof that no such tool exists.
How This Guide Was Built
This guide is based on official documentation, public repository pages, and product pages. The author did not run these tools hands-on; there are no benchmarks or screenshots. Repository and product pages were saved in full, except Pulpo, which was read from its overview page only. The Claude Code documentation and the warden README were read through keyword excerpts, which are verbatim substrings of the saved pages. Commit counts are point-in-time signals. Readers should verify licences, availability and current features before adopting a tool.
📖 Related Reads
- NoCode Insider — AI workflow automation with no-code tools, agents, and APIs
- ToolBrain — tool reviews, LLM comparisons, and AI workflow guides
- CodeIntel Log — code quality, debugging, and software engineering benchmarks
Cross-links automatically generated from NiteAgent.
← Back to all posts


