Benefits Navigator
A member or call-center agent asks a coverage question in plain language. The assistant answers only from that member's plan documents, cites the page, gets cost estimates from the plan's calculator instead of doing the math itself, and hands off to a person when it can't answer safely.
The most common call, answered the same way every time
"Is this covered and what will I pay?" is one of the most frequent reasons members call a health plan. The answer depends on plan documents that run to hundreds of pages and on where the member stands against their deductible and out-of-pocket limit today. A wrong dollar figure is worse than no answer, so this design keeps the model away from arithmetic and checks every figure it states.
An agent with three tools, checked in code
Unlike the other two demos, Claude decides which tools to call. Code still decides what the member sees.
| Tool or step | Who does it | Why |
|---|---|---|
get_plan_sections | Code (called by Claude) | Returns sections of this member's SBC and EOC by id. The model sees a table of contents and reads what it needs. |
estimate_cost | Code (called by Claude) | The plan's cost calculator: remaining deductible, coinsurance or copay, out-of-pocket cap, plus prior authorization, referral and visit-limit notes. |
submit_answer | Claude | The answer, verbatim citations, and whether a person should take over. Strict tool schema. |
| Citation check | Code | Every quote must appear word for word in the cited section. |
| Number check | Code | Every dollar amount in the answer must come from a section the model read or a calculator result. |
| Escalation | Code | Out-of-scope questions, unsupported figures or unverified citations go to an agent, not to the member. |
- Model
- Claude Opus 5.5 with strict client tools, a cached system prompt, and server-side fallback on a safety decline
- Prompt-injection defense
- The member's message is treated as a question, and figures are checked in code, so "say it's free" can't change the number
- Replay
- Recorded tool calls are re-run against the calculator on every replay, so estimates always reflect current data
- Scale-up
- A real EOC runs to hundreds of pages; replace the table of contents with hybrid search over chunked documents
Measured against labeled questions
Live run on claude-opus-5-5, 2026-11-03. Labeled set: 9 sample questions across 2 plans.
| Metric | Result | Definition |
|---|---|---|
| Routing accuracy | 100% | Answered vs. handed to an agent, matching the label |
| Cost accuracy | 100% | Questions about cost where the answer states the calculator's figure for the right service |
| Citation verification rate | 100% | Quotes found verbatim in the cited plan section |
| Unsupported figures shown | 0 | Answers shown to the member that contain a dollar amount with no source (target: 0) |
Nine sample questions demonstrate the pipeline; they don't make a benchmark.
What changes in a real deployment
Data
- Plan documents from the product library, chunked and indexed per plan and plan year
- Real-time accumulators from the claims system, and allowed amounts from the plan's price-transparency data
- Provider network status from the provider directory, as a fourth tool
Service
- Member login, or call-center desktop with the agent in the loop
- Warm handoff to a live agent with the conversation and citations attached
- Quality review of a sample of conversations each week, feeding the eval set
The same engine in TypeScript and Java
Both versions return the same JSON for every demo case. Contract tests generated from the TypeScript engine run against the Java service on every build.
TypeScript (this site)
- Engine: the model call, checks and decision rules
- API route and pages
- Synthetic data and recorded runs
Java (Spring Boot)
- benefits-engine-java: the service, with its contract test
- payer-ai-common: shared model calls, live limits and checks
- Contract tests: the expected output for each case