Case study

Document intake you can score — not only trace

When agents file documents into Drive and open tasks, a green chat turn is not the same as a completed intake job.

This case study walks a typical document-intake path. You will see what breaks when “done” lives only in chat. You will see how the filing stack runs in plain steps. You will see what Witdem shows on a real demo run — outcome, goal, and spend on one execution identity.

The problem

Documents arrive faster than people can sort them. Inboxes fill with PDFs, scans, and mail attachments. Shared drives and ticket queues grow at the same time.

Someone still has to choose a folder. Someone still has to name the file. Someone still has to decide who owns the next step.

Teams then lose the thread across tools. The chat assistant may say it got the file. Drive may hold a copy on a path nobody agreed on. A task may exist in another tool — or it may not.

Policy asks a sharper question. Did the file land where it must? Were the follow-ups the process needs actually opened?

Without an explicit product goal on the run, “the model answered” stands in for “intake worked.” That false finish is costly. People re-check Drive by hand. They reopen chat history. They argue from screenshots instead of from one scored run.

What this means in practice

Picture one intake job from start to finish. A person brings a document into chat. The stack files it and may open follow-up tasks. You then ask one plain question: did this run finish the intake job you named?

First look at the chat turn and the tool calls. They can look healthy. That only says the path ran.

Then look at the systems of record. Is the file in the right Drive place? Are the tasks present when the process needs them? If not, the job failed for the business — even when chat looks fine.

Put cost on the same run identity. Tokens and time matter more when you know whether the goal passed. A cheap miss and an expensive miss need different reviews. Both need the goal marked clearly.

Write the success rule once next to the workflow. Every reviewer should score the same run the same way. That is document intake you can score, not only scroll.

False success on intake

Intake fails quietly when runtime looks fine. Tools can return. The chat turn can complete. The filing job can still miss the bar you care about.

Ops may trust the green chat path. Product still sees an incomplete intake job. Without a named goal on the run, those two views never meet in one review.

A finished turn is not the same as a successful filing job. Spans show the path taken. They do not prove the Drive write or the tasks the process required.

A product goal is a short shared rule for “done.” Evidence is the structured fields that prove pass or fail. Keep both beside the run you already inspect.

The pipeline

A common intake stack looks like this in product terms. Each step has one job. None of them alone answers “was intake done?”

Onyx is the human-facing chat boundary. Someone drops or points at a document. The assistant accepts it into the conversation. That is where the person thinks the job started.

witdem-proxy sits between that chat surface and the processing stack. It carries the request into the next stage. It is also a natural place to attach observability for the full execution identity.

Haystack runs the processing pipeline. It extracts structure from the artifact. It classifies the document type. It prepares what “filed correctly” means for this case.

An LLM step then decides labels, routing, or filing choices the pipeline needs. The model is one step in the job. It is not the system of record.

Google Drive and Tasks hold the side effects the business depends on. Drive stores the filed artifact. Tasks open follow-ups for review, approval, or exceptions. Those tools are where “done” must show up in the real world.

Witdem sits on the observability side of that path. It does not replace Onyx, the proxy, Haystack, the model, Drive, or Tasks. It answers whether a given execution finished the declared intake job, what it spent, and whether extra work showed up — next to the same run you would otherwise only see as spans or chat.

What Witdem shows on one run

Operators should not rebuild success from three UIs. One execution identity should carry runtime health, the application outcome, the product goal, and measured spend when evidence exists.

The public demo Document intake screens show that join. Open an execution at a glance. You see how the job resolved. You see whether the product goal was achieved. You see elapsed time, tokens, and measured cost on the same view.

Two sibling demo executions both achieve the product goal. They resolve differently. One ends as Stored With Tasks — filing plus follow-up tasks. The other ends as Stored — filing alone. That difference is visible without opening every chat turn.

The all-executions list keeps those outcomes on the same cards operators already scan. Business result and product goal sit together on each Document intake row. You can compare sibling runs in one list instead of stitching chat, Drive, and tickets by hand.

Concrete insights

These notes come from Document intake runs on demo.witdem.com/runs. One example is execution 55be9c22116c4573ac09d0eea13e99e0 (Sep 14, 2026). They are demo facts. They are not invented customer KPIs.

Runtime and goal share one identity. That intake run is completed. The business result is Stored With Tasks. The product goal is Achieved. A green pipeline alone would not prove the filing-plus-tasks bar. The scored run does.

Outcomes differ on similar paths. Sibling Document intake rows show Stored and Stored With Tasks. Both show goal achieved. Operators can see whether success meant “filed only” or “filed and follow-up tasks created.”

Spend rides the same run. The Stored With Tasks screen reports 7.4 s, 263 tokens, and measured cost $0.00007635 beside the outcome. Cost questions attach to the job result, not to a separate sheet.

Retries are visible fields. That example shows 0 total attempts / 0 extra and no recovery. When retries or recovery appear on other runs, they sit next to outcome and cost. They do not live only in chat history or buried tool logs.

• completed + Stored With Tasks + goal Achieved on one execution
• Sibling demo outcomes: Stored vs Stored With Tasks
• 7.4 s · 263 tokens · $0.00007635 measured on the scored run
• 0 attempts / 0 extra · no recovery on that path

Zoom out from a single run. Goal performance shows how declared goals land across the portfolio. The screen includes Document filed among those goals. You see achieved versus needs attention without leaving the product-outcome layer.

What changes

Builders declare what success means next to the workflow. Filing rules and task expectations become product goals you can inspect. They stop living only as tribal knowledge in a chat thread.

Operators stop rebuilding “did intake work?” from chat, Drive, and tickets. They open one run. They read the application result, the goal state, and the spend. They escalate when the job missed the bar or the evidence is incomplete.

That is the shift this case study is about. Document intake becomes an agentic workflow you can score. It is not only a conversation you can scroll. For the broader frame — tracing, eval, cost, and product outcomes on one run — see the pillar guide. Framework pages cover Haystack and LangGraph patterns.

FAQ

Does a green chat turn mean intake succeeded? No. Green means the assistant path finished. Success means the product goal you named held for that run — for example filed as required, with tasks when the process needs them.

What steps sit in this intake stack? In product terms: Onyx accepts the document, witdem-proxy carries it into processing, Haystack structures the job, an LLM step helps decide, and Google Drive plus Tasks hold the real-world side effects.

Does Witdem replace Drive or the chat assistant? No. Witdem sits on the observability side. It joins outcome, goal, and spend on the same execution identity you already review.

What do the demo screens show? Execution-at-a-glance cards with Stored or Stored With Tasks, goal achieved, time, tokens, and cost; an all-executions list with those fields on each row; and Goal performance including Document filed.

Where can I check the demo facts? Open Document intake runs on demo.witdem.com/runs. The numbers and labels on this page match those screens.

Related

Agentic workflow observability · Haystack observability · LangGraph observability · LangChain observability · OpenAI Agents observability