Case study · Knowledge management · Enterprise compliance team · 90 days
Same policy question, asked by two different roles, answered from the document each role is allowed to see.
An enterprise compliance team was running two parallel policy libraries because the people who set the policies and the people who acted on them could not share documents safely. An internal assistant now answers in plain language against each role’s permitted library, cites the clause, and flags any answer drawn from a policy older than 18 months so the reader sees the staleness before they act on it.
Chapter 01 · The problem
Two parallel libraries, and nobody checking the dates.
The compliance team ran what they called “the binder”, a shared drive with several hundred policy documents covering hiring, conduct, financial controls, and the regulated parts of their day-to-day operation. The administrative side of the team owned the documents and kept them current. The executive side of the team consumed the documents and was expected to make decisions against them. The two sides were supposed to be using the same source of truth, except they were not allowed to. The administrative library contained operational templates and the documented exceptions to the published policies, neither of which the executive side could read on regulatory grounds. The executive library was a subset, deliberately. The cost of running two libraries was that nobody had a clean view of which version of a given policy each side was actually answering from. A handful of people had read enough of the binder to answer questions without checking. The team relied on them. Their answers were almost always correct on the substance and almost never current on the date. The compliance lead had been quietly tracking which policies got cited most in internal questions, then cross-checking those policies against their stated review intervals. The pattern that worried her was not the wrong answers, it was the right answers drawn from a document that had not been reviewed in years and was probably no longer the operative version. The team would have had no way of knowing. Search was on. Nobody used it. The team had tried a generic search index and a separate internal wiki, and the same pattern recurred each time: the documents were findable, but the reader had no way to verify which document was current without opening every candidate file and reading the footer. The two-library segregation was technically enforced in document permissions but was easy to defeat by emailing a document across the boundary, which had happened. The team needed a tool that would refuse to cross the boundary itself, not just refuse to display the document.
Same handful of questions, asked across every team meeting
Chapter 02 · The approach
One assistant per role. Every answer carries the clause and the date the clause was last reviewed.
The fix was a single assistant that knew which role was asking and retrieved only from the policy documents that role was permitted to read. The role-segregation was enforced at the database layer in Postgres row-level security, not in the application code, so a misconfigured query simply returned no rows rather than leaking a document across the boundary. The retrieval pipeline summarised against the retrieved passages and returned the answer with the source clause inline. The model was instructed to refuse rather than guess: when retrieval found nothing in the role’s permitted library, the assistant said so explicitly and offered to route the question to the administrative side rather than answering from a tangentially related passage. Every uploaded policy carried a review date as part of its ingest metadata. When the assistant answered, it surfaced the answer, the source clause, and a visible flag if the source policy had passed its 18-month currency threshold. The flag was not a soft warning, it was a banner the reader had to acknowledge before the answer panel rendered. Documents without an extractable review date defaulted to the upload date and were flagged for manual review in the administrator queue, so the worst case was a false positive rather than a quiet stale answer. The compliance lead’s prior worry, right answers drawn from documents nobody had checked, became a question the reader could answer at the moment of asking. The pipeline ingested PDFs through an n8n workflow that extracted text, ran metadata extraction for review-date detection, generated the embedding, and wrote the document with its role assignment to the database. The administrator interface let the policy owners reassign role permissions after upload, recover from ingest failures, and review the documents that had been flagged for missing review dates. The whole thing ran self-hosted on the team’s existing Supabase footprint, with the model provider abstracted behind the n8n workflow layer so the choice of provider stayed configurable as the team’s procurement preferences shifted.
What we built
A role-segregated retrieval system with PostgreSQL row-level security enforcing the policy library boundary at the database layer, an n8n ingest pipeline that extracts review-date metadata from uploaded PDFs, a chat interface that answers in plain language with the source clause inline, and a currency banner that fires on every answer drawn from a policy past its 18-month review threshold.
Build time · 90 days · MVP delivered with two roles and the staleness flag operational; first 30 days were retrieval and ingest, next 30 were the role-boundary hardening, final 30 were the currency-flag UX and the administrator queue for missing dates.
Chapter 03 · The outcome
Three numbers the compliance lead wanted before she signed off the rollout.
of answers carry a visible currency flag when the source policy is past its 18-month review threshold, surfaced before the answer panel renders.
of answers return with the source clause inline, so the reader can verify the wording without opening the source document.
- 0
across the validation suite of role-mismatched queries, with the boundary enforced at the database layer in Postgres row-level security rather than in application code.
Numbers verified against the project’s success-metric definitions and the rollout review with the compliance lead. Anonymised under the standard case-study disclosure: vertical, architectural choices, and the currency-flag behaviour are real; the specific industry, the team’s name, the named department descriptors, and the model-provider choice have been generalised to functional descriptors. The 18-month threshold is the team’s own published review cadence, not an invention. The “100% currency-flag coverage” and “100% citation inline” claims describe pipeline-level behaviour rather than user-survey results.
Chapter 04 · What we learned
The retrieval was the easy part. The staleness flag was the feature that changed how the team read the answers.
The temptation with a compliance-oriented retrieval build is to keep adding features to the answer panel until the reader has every possible piece of context. We cut the answer UI back to the source clause and the staleness flag, with the flag designed as the most visually prominent element on the page. Designing for the flag the team needed to act on, rather than for every metric the model could display, was the load-bearing call.
Row-level security at the database layer felt over-engineered for a two-role system. We did it anyway because the cost of a cross-role leak in a compliance setting is reputational, not just functional. The architectural payoff arrives when the brief eventually grows: two-role systems become three-role systems faster than the spec suggests they will, and the boundary already enforced where the data lives means adding the next role is a policy update rather than a refactor of the retrieval pipeline.
The 18-month threshold was the team’s existing policy review cadence, not a number we invented. Once the flag was visible on every answer, the team got a free audit of which policies were silently overdue. The compliance lead could finally see which documents were being read most often and were also the most stale, and policy maintenance shifted from a calendar-driven exercise to a usage-driven one. The retrieval system was the surface. The audit was the actual win.
Settled handoff rate · Audit as side-effect of the flag
Chapter 05 · In the client’s words
The thing that changed was not that the team got faster answers. It was that they stopped quietly trusting documents nobody had checked in years. The flag did that, more than the retrieval did.
Compliance Operations Lead, AU enterprise compliance team (anonymised at the team’s request)
Have a policy library nobody is sure is still current?
A 45-minute audit. We look at the documents your team relies on, where the currency dates live, and whether anyone is checking them at the moment of answer. You leave with a one-page memo whether or not we are the right fit.