From memory · Five questions a day
Five questions a day, asked by everyone, answered from memory.
Member-facing staff at a national membership body were answering roughly five questions a day each, the same five, about eligibility rules, fee waivers and the documentation required for renewals.
The answers were correct, mostly. They came from the heads of three long-tenured staff who had read the policy enough times to recite it.
The Knowledge Lead had been quietly mapping which questions came up, who answered them, and how often the answer drifted between people. The drift worried her most.
Build · 19 days
The policy passage first, with the citation beside the answer.
An internal assistant the staff ask in plain language. It retrieves the relevant policy passage from the SharePoint library and shows the citation beside the answer. The model summarises; it does not interpret.
Answers come with the source paragraph as a click-through. If retrieval finds nothing close enough, the assistant says so and routes the question to the Knowledge Lead’s queue rather than guessing.
The SharePoint policy library is indexed into pgvector, and Anthropic Claude writes grounded summaries against the retrieved passages. n8n syncs SharePoint nightly. Nexus, Raava’s orchestrator, coordinates the index refresh, the query handler and the unanswered-question queue.
Built in 19 days as a pilot, then six weeks of iteration with the Knowledge Lead and three member-facing staff.
Staff member asks
Can a renewal fee be waived after a hardship claim?
Staff member asks
Do lapsed members keep their event credits?
Closest passages in the policy library
Close enough
Source: Fee waivers policy, section 4.2
A renewal fee may be waived once in any two-year period where the member has an approved hardship claim.
Summary
Yes, once every two years, if the hardship claim is approved.
Nothing in the policy library is close enough to answer this.
Sent to the Knowledge Lead’s queue
No answer guessed
A member question. A staff member asks about a fee waiver in plain language.
A staff member asks about a fee waiver in plain language.
The assistant retrieves the closest passage from the SharePoint policy library.
The source paragraph shows first, with the plain-language summary under it.
When retrieval finds no close passage, the assistant says so and sends the question to the Knowledge Lead.
Model Anthropic ClaudeRetrieval pgvectorGlue n8nSource SharePoint syncCoordination Nexus, Raava’s orchestratorPrivacy Citation-grounded answers only
Threshold · Six weeks
Six weeks teaching the assistant to admit it did not know.
The early version would summarise from a tangentially related passage and read fluent, and wrong.
We tightened the similarity threshold, added an explicit hand-off prompt for low-confidence retrievals, and changed the answer template so the source paragraph showed first and the summary second.
In use · Window not on the record
What the Knowledge Lead trusted before she shared it.
Once the assistant routed unanswered questions back to the Knowledge Lead, she had a clean signal of what the policy library was missing. Twelve passages got rewritten in the first month, not because the model failed, but because the policy itself had been ambiguous and the staff had been quietly papering over it.
Numbers verified by the Knowledge Lead against the call-handling export and her quarterly consistency audit. Anonymised under our standard case-study disclosure: vertical and outcome are real; organisation name, location, and policy-library tenant are not.
94%
of answers carrying a clickable citation back to the source policy paragraph
Each square is one answer in a hundred; no time frame is on the record.
- Checked by
- The Knowledge Lead
Nothing from before the assistant to compare it with, and no time frame.
−60%
time to first answer on member queries
- Baseline
- The prior quarter’s average call-handling time
- Checked by
- The Knowledge Lead, against the call-handling export
The weeks it was measured over are not on the record.
−11%
drift between staff on the same five questions
- Checked by
- The Knowledge Lead’s answer-consistency audit
The drift it fell from, and over what period, are not on the record.
“The assistant is helpful, but the real win was finding out which of our policies were quietly broken. We could not see the gaps until we had to answer them in writing.”
Curious whether your team has the same five questions?
A free 45-minute audit. We look at the questions your team answers most, where the source policy lives, and where the drift is creeping in. You get a written follow-up within 48 hours, whether we’d be a fit or not.
