Lead-gen's pipeline LangGraph is — by construction — a five-phase sequence. Each phase becomes one span; fan-out phases nest a worker span per company id. Every LLM call inside a node is its own child span via the LangSmith auto-instrumentation. Phase nodes return a StageReport (processed, created, errors[], duration_ms) which becomes the phase span's attribute bag.

Three graphs predate create_react_agent: they parse {"tool": ..., "args": ...} JSON from the LLM and execute Python tools directly. LangChain's auto-tracer can't see the tool dispatch step in this pattern. The fix: wrap each loop in agent_run_span (parent chain run) and each tool call in tool_call_span. Both are no-ops when LANGSMITH_TRACING is not true — strictly additive.

Two context managers, both strict no-ops when LangSmith is disabled. Caller pattern: wrap the whole loop in agent_run_span; wrap each tool dispatch in tool_call_span and pipe the result (or exception) through the yielded finish(result=…, error=…) callback.

@contextmanager
def agent_run_span(name: str, *, metadata=None) -> Iterator[Run | None]:
    """Wraps an LLM↔tool loop as one parent chain run. The caller calls
    run.end(outputs={answer, steps, total_tokens, total_cost_usd}) at
    loop exit so the trace overview answers 'how many tokens / dollars
    did this agent burn' without having to sum the per-call children."""

@contextmanager
def tool_call_span(tool: str, args: dict, *, attempt: int = 1):
    """Wraps a single tool dispatch as a tool run. Yields
    finish(result=…, error=…). On exit, writes outputs + latency_ms
    metadata; on error, sets level=ERROR + outputs.error. The attempt
    counter is how retries surface in the trace."""

# caller-side (admin_chat_graph.chat):
with agent_run_span("agent_run:admin_chat", metadata={"max_steps": MAX_STEPS}) as run:
    for step in range(MAX_STEPS):
        result, tel = await ainvoke_json_with_telemetry(llm, msgs)
        acc_tokens += tel["total_tokens"]; acc_cost += tel["cost_usd"]
        if "answer" in result:
            if run is not None:
                run.end(outputs={"answer": result["answer"], "steps": step + 1,
                                 "total_tokens": acc_tokens, "total_cost_usd": acc_cost})
            return {"response": result["answer"]}
        with tool_call_span(tool, args, attempt=step + 1) as finish:
            try:
                observation = await _dispatch(tool, args); finish(result=observation[:2000])
            except Exception as exc:
                finish(error=exc); raise

The LangSmith API key lives in Cloudflare Secrets Store, not in wrangler secret putplain-text secrets. Three steps to land it in os.environ["LANGSMITH_API_KEY"]inside the Python container:

# create / rotate (uses stored OAuth — drop CLOUDFLARE_API_TOKEN)
env -u CLOUDFLARE_API_TOKEN \
  npx wrangler secrets-store secret create ec928f4771fb4577a607a0b122e8087e \
    --name LANGSMITH_API_KEY --value "lsv2_pt_…" --scopes workers --remote

// backend/core/wrangler.jsonc — secrets_store_secrets[]
{
  "binding":     "LANGSMITH_API_KEY",
  "store_id":    "ec928f4771fb4577a607a0b122e8087e",
  "secret_name": "LANGSMITH_API_KEY"
}

// non-secret config flows via vars{}
"LANGSMITH_TRACING":  "true",
"LANGSMITH_ENDPOINT": "https://api.smith.langchain.com",
"LANGSMITH_PROJECT":  "lead-gen"

// backend/core/src/index.js — CoreContainer#start()
async start(startOptions, waitOptions) {
  const [hfToken, langsmithKey] = await Promise.all([
    this.env.HF_TOKEN?.get()          ?? Promise.resolve(""),
    this.env.LANGSMITH_API_KEY?.get() ?? Promise.resolve(""),
  ]);
  return super.start({
    ...startOptions,
    envVars: {
      ...this.envVars,
      LANGSMITH_API_KEY: langsmithKey ?? "",
      /* … */
    },
  }, waitOptions);
}

Deploy gotcha: wrangler deploy with CLOUDFLARE_API_TOKEN set errors with 10021 Secrets store binding authorization failed — the token lacks secrets_store:read. Fix: env -u CLOUDFLARE_API_TOKEN wrangler deploy so wrangler falls back to the stored OAuth (which has full scope).

These are the seven signals an interviewer is looking for, mapped to where they actually live in this codebase. Nothing here is aspirational — the wrapper is on by default on every LLM call.

One trace per pipeline run. The root span carries the run id; phase spans carry StageReport as attributes; fan-out phases nest one worker span per id; every LLM call gets its own leaf span with token/cost/latency.

pipeline_run (trace root)
├─ phase: discover  (run_discover)
│  └─ llm: company_discovery_graph  [model, tokens, latency, cost]
├─ phase: enrich  (enrich_queue → Send fanout → enrich_collect)
│  ├─ worker: enrich_one  [company_id=42]
│  │   └─ llm: company_enrichment_graph  [model, tokens, latency, cost]
│  ├─ worker: enrich_one  [company_id=43]
│  │   └─ llm: company_enrichment_graph  ...
│  └─ ... (one worker span per id)
├─ phase: contacts  (contacts_queue → Send fanout → contacts_collect)
│  ├─ worker: contact_one  [company_id=42]
│  │   ├─ llm: contact_discovery_graph
│  │   └─ llm: contact_enrich_graph
│  └─ ...
├─ phase: qa  (run_qa → run_qa_critic)
│  ├─ sql: row_counts
│  └─ llm: critic  [model, tokens, latency, cost]
└─ phase: outreach  (outreach_queue → interrupt → outreach_run)
   ├─ checkpoint: human_gate  [decision=approve]
   └─ worker: outreach_run  [contact_id=314]
       └─ llm: email_outreach_graph  [model, tokens, latency, cost]

A real admin_chat run answering "how many companies are in the database?" — three loop turns, two tool calls, one root agent_run_span. LLM children are captured automatically by LangChain's tracer; tool children are captured by tool_call_span.

agent_run:admin_chat  (chain — agent_run_span)
│  metadata: { max_steps: 6, prompt_chars: 41 }
│
├─ ChatOpenAI deepseek-v4-pro                                       ← LangChain auto-trace
│   prompt:        "User question: how many companies are in the database?"
│   completion:    "{\"tool\": \"inspect_schema\", \"args\": {\"table\": \"companies\"}}"
│   model:         deepseek-v4-pro
│   input_tokens:  412   output_tokens: 38   total: 450
│   latency_ms:    1820
│
├─ tool:inspect_schema  (tool — tool_call_span)
│   inputs:    { args: { table: "companies" }, attempt: 1 }
│   outputs:   { result: "id: int\nname: text\nslug: text\n…" }
│   metadata:  { tool: "inspect_schema", attempt: 1, latency_ms: 83 }
│
├─ ChatOpenAI deepseek-v4-pro
│   completion:    "{\"tool\": \"count_rows\", \"args\": {\"table\": \"companies\"}}"
│   input_tokens:  680   output_tokens: 24   total: 704
│   latency_ms:    1340
│
├─ tool:count_rows
│   inputs:    { args: { table: "companies" }, attempt: 2 }
│   outputs:   { result: "companies: 2607 rows" }
│   metadata:  { tool: "count_rows", attempt: 2, latency_ms: 145 }
│
├─ ChatOpenAI deepseek-v4-pro
│   completion:    "{\"answer\": \"There are 2607 companies in the database.\"}"
│   input_tokens:  720   output_tokens: 22   total: 742
│   latency_ms:    980
│
└─ outputs: {
      answer:            "There are 2607 companies in the database.",
      steps:             3,
      total_tokens:      1896,
      total_cost_usd:    0.00041
   }

The shape below is what one enrich_one worker emits into graph_meta.telemetry, and what gets rolled up into product_intel_runs.total_cost_usd at run completion. Values are illustrative; the keys and types are the real ones from ainvoke_json_with_telemetry.

// graph_meta.telemetry["enrich.enrich_one[company_id=42]"]
{
  "model": "deepseek-v4-pro",
  "calls": 3,
  "input_tokens": 4180,
  "output_tokens": 612,
  "total_tokens": 4792,
  "cost_usd": 0.001847,
  "latency_ms": 2840,
  "retries": 0
}

// Rolled up into product_intel_runs at end of run:
{
  "id": "0193-...-runid",
  "lg_run_id": "01HX...",
  "lg_thread_id": "01HX...",
  "total_cost_usd": 0.1834,
  "progress": {
    "discover":  { "duration_ms": 1820, "processed": 1,  "created": 14 },
    "enrich":    { "duration_ms": 18430, "processed": 14, "errors": [] },
    "contacts":  { "duration_ms": 24110, "processed": 14, "created": 38 },
    "qa":        { "duration_ms": 3120, "critic_pass_rate": 0.86 },
    "outreach":  { "duration_ms": 6240, "drafts": 12, "human_decision": "approve" }
  }
}

What the parent agent_run_span emits as outputs at loop exit, plus one child tool_call_span. Token + cost totals come from ainvoke_json_with_telemetry (the same path the pipeline graph uses) and are accumulated across loop turns before being written on the parent.

// parent agent_run_span outputs at loop exit:
{
  "answer":         "There are 2607 companies in the database.",
  "steps":          3,
  "total_tokens":   1896,
  "total_cost_usd": 0.00041
}

// one child tool_call_span (inspect_schema):
{
  "name":     "tool:inspect_schema",
  "run_type": "tool",
  "inputs":   { "args": { "table": "companies" }, "attempt": 1 },
  "outputs":  { "result": "id: int\nname: text\nslug: text\n…" },
  "metadata": { "tool": "inspect_schema", "attempt": 1, "latency_ms": 83 }
}

// error-path tool_call_span (forced bad SQL — query_db):
{
  "name":     "tool:query_db",
  "run_type": "tool",
  "inputs":   { "args": { "sql": "SELECT count(*) FROM nonexistent" }, "attempt": 4 },
  "outputs":  { "error": "Error: relation \"nonexistent\" does not exist" },
  "level":    "ERROR",
  "metadata": { "tool": "query_db", "attempt": 4, "latency_ms": 31 }
}

A user says "your agent gave a wrong answer". Five steps, in order, to localize the failure and decide what to fix. Each maps to a real artifact in this codebase — the trace, the DB pointer, the worker logs, the boot probe. No prompt-tweaking on a hunch.

Seven recurring shapes from real wrong-answer incidents. The fingerprint column tells you what to look for in the trace; the fix column is the actual remediation, not generic advice. Hallucination and tool failure are the two ends of the spectrum — most rows diagnose which side of that line you're on.

What's deliberately not built yet. Calling these out is part of a good trace-design answer — observability is never finished, and an interviewer learns more from the gap list than from the feature list.