Skip to main content

Case study · A national membership body · Knowledge Management

Answers show the policy paragraph they came from, or say they do not know.

Result

−60%

time to first answer on member queries

Measured, incomplete
Baseline
The prior quarter’s average call-handling time
Checked by
The Knowledge Lead, against the call-handling export

The weeks it was measured over are not on the record.

Member-facing staff answered roughly five questions a day each, the same five, from the memory of three long-tenured colleagues, and the answers drifted. An internal assistant now answers in plain language with the source policy paragraph beside it, and routes what it cannot find to the Knowledge Lead.

A staff member draws one archive box from a long shelf of identical grey boxes, a faint violet trail following it out.

A business like yours?

Organisation
A national membership body
Queries
Around 250 a week
Systems
A SharePoint policy library

Explainer · about 20 s · silent

Read the transcript
  1. At a national membership body, staff answered the same five questions every day from the memory of three colleagues, and the answers drifted.
  2. An internal assistant retrieves the policy passage and shows the citation beside its answer.
  3. When nothing close enough is found, it says so and routes the question to the Knowledge Lead.
  4. Time to a first answer fell 60% against the prior quarter’s call-handling time, checked by the Knowledge Lead; the window is not on the record.

From memory · Five questions a day

Five questions a day, asked by everyone, answered from memory.

Member-facing staff at a national membership body were answering roughly five questions a day each, the same five, about eligibility rules, fee waivers and the documentation required for renewals.

The answers were correct, mostly. They came from the heads of three long-tenured staff who had read the policy enough times to recite it.

The Knowledge Lead had been quietly mapping which questions came up, who answered them, and how often the answer drifted between people. The drift worried her most.

Build · 19 days

The policy passage first, with the citation beside the answer.

An internal assistant the staff ask in plain language. It retrieves the relevant policy passage from the SharePoint library and shows the citation beside the answer. The model summarises; it does not interpret.

Answers come with the source paragraph as a click-through. If retrieval finds nothing close enough, the assistant says so and routes the question to the Knowledge Lead’s queue rather than guessing.

The SharePoint policy library is indexed into pgvector, and Anthropic Claude writes grounded summaries against the retrieved passages. n8n syncs SharePoint nightly. Nexus, Raava’s orchestrator, coordinates the index refresh, the query handler and the unanswered-question queue.

Built in 19 days as a pilot, then six weeks of iteration with the Knowledge Lead and three member-facing staff.

A member question. A staff member asks about a fee waiver in plain language.

  1. A staff member asks about a fee waiver in plain language.

  2. The assistant retrieves the closest passage from the SharePoint policy library.

  3. The source paragraph shows first, with the plain-language summary under it.

  4. When retrieval finds no close passage, the assistant says so and sends the question to the Knowledge Lead.

Model Anthropic ClaudeRetrieval pgvectorGlue n8nSource SharePoint syncCoordination Nexus, Raava’s orchestratorPrivacy Citation-grounded answers only

Threshold · Six weeks

Six weeks teaching the assistant to admit it did not know.

The early version would summarise from a tangentially related passage and read fluent, and wrong.

We tightened the similarity threshold, added an explicit hand-off prompt for low-confidence retrievals, and changed the answer template so the source paragraph showed first and the summary second.

In use · Window not on the record

What the Knowledge Lead trusted before she shared it.

Once the assistant routed unanswered questions back to the Knowledge Lead, she had a clean signal of what the policy library was missing. Twelve passages got rewritten in the first month, not because the model failed, but because the policy itself had been ambiguous and the staff had been quietly papering over it.

Numbers verified by the Knowledge Lead against the call-handling export and her quarterly consistency audit. Anonymised under our standard case-study disclosure: vertical and outcome are real; organisation name, location, and policy-library tenant are not.

94%

of answers carrying a clickable citation back to the source policy paragraph

Measured, incomplete
Checked by
The Knowledge Lead

Nothing from before the assistant to compare it with, and no time frame.

−60%

time to first answer on member queries

Measured, incomplete
Baseline
The prior quarter’s average call-handling time
Checked by
The Knowledge Lead, against the call-handling export

The weeks it was measured over are not on the record.

−11%

drift between staff on the same five questions

Measured, incomplete
Checked by
The Knowledge Lead’s answer-consistency audit

The drift it fell from, and over what period, are not on the record.

“The assistant is helpful, but the real win was finding out which of our policies were quietly broken. We could not see the gaps until we had to answer them in writing.”

Knowledge Lead · National membership body · AU (anonymised at the client’s request)

Curious whether your team has the same five questions?

A free 45-minute audit. We look at the questions your team answers most, where the source policy lives, and where the drift is creeping in. You get a written follow-up within 48 hours, whether we’d be a fit or not.