Skip to main content

Case study · An enterprise compliance team · Knowledge Management

Each role is answered from its own library, with the clause and its review date.

Result

100%

of answers from a policy past its 18-month review threshold carry the currency flag, shown before the answer panel renders

Tested behaviour

Pipeline behaviour checked against the project’s success-metric definitions, not a usage result.

The team kept two policy libraries, one each side of a regulatory line, and trusted answers often came from documents nobody had reviewed in years. One assistant now answers each role only from the library that role may read, with the source clause inline and a banner when the policy is past its 18-month review.

A staff member opens one drawer in a bank of grey filing cabinets, violet light spilling from inside.

A business like yours?

Team
An enterprise compliance team
Library
Several hundred policy documents
Systems
Postgres on a self-hosted Supabase footprint

Explainer · about 25 s · silent

Read the transcript
  1. An enterprise compliance team kept two policy libraries, and trusted answers often came from documents nobody had reviewed in years.
  2. One assistant now answers each role only from the library that role may read, with the boundary enforced in the database.
  3. Every answer carries its source clause, and a policy past its 18-month review shows a banner first.
  4. Every answer from an overdue policy carries the flag: pipeline behaviour checked at the rollout review, not a usage result.

Two libraries · Before the build

The team ran two parallel libraries, and nobody checked the dates.

The compliance team ran what they called “the binder”: a shared drive with several hundred policy documents covering hiring, conduct, financial controls and the regulated parts of their day-to-day operation. The administrative side owned the documents and kept them current; the executive side was expected to make decisions against them.

They were not allowed to share one source of truth. The administrative library held operational templates and the documented exceptions to the published policies, which the executive side could not read on regulatory grounds, so the executive library was a deliberate subset. Nobody had a clean view of which version of a policy each side was answering from.

A handful of people had read enough of the binder to answer without checking. Their answers were almost always correct on the substance and almost never current on the date. What worried the compliance lead was right answers drawn from documents that had not been reviewed in years.

Search existed, and nobody used it: the documents were findable, but the reader could not tell which was current without opening every candidate and reading the footer. The split between the libraries was enforced in document permissions, and it had been defeated by emailing a document across it.

Retrieval and ingest · Days 1 to 30

One assistant that knows which role is asking.

The first 30 days went on retrieval and ingest. An n8n workflow takes each uploaded PDF, extracts the text, reads the review date from its metadata, generates the embedding, and writes the document to the database with its role assignment.

The assistant answers in plain language against the retrieved passages, with the source clause inline. It is instructed to refuse rather than guess: when nothing in the role’s permitted library matches, it says so and offers to route the question to the administrative side.

It runs self-hosted on the team’s existing Supabase footprint, with the model provider behind the n8n workflow layer so the choice stays configurable as the team’s procurement preferences shift.

A policy is uploaded. Its review date is read at ingest and stored with its role assignment.

  1. Its review date is read at ingest and stored with its role assignment.

  2. A reader on the executive side asks a policy question in plain language.

  3. Row-level security returns only the documents that role may read.

  4. The answer comes back with the source clause in the panel.

  5. An answer from an overdue policy waits behind a banner the reader acknowledges.

Model Claude (configurable per workflow)Retrieval pgvector on PostgresRole boundary Postgres row-level securityIngest n8n metadata + embedding pipelineHosting Self-hosted Supabase footprintCurrency flag Banner on every answer past the 18-month threshold

Role boundary · Days 31 to 60

The boundary lives in the database, not in the application.

The next 30 days hardened the role boundary. It is enforced in Postgres row-level security, not in the application code, so a misconfigured query returns no rows rather than leaking a document across the boundary.

Row-level security felt over-engineered for a two-role system. We did it anyway: in a compliance setting a cross-role leak costs reputation, not just function. Two-role systems become three-role systems faster than the spec suggests, and with the boundary already where the data lives, the next role is a policy update rather than a rework of the retrieval pipeline.

Currency flag · Days 61 to 90

A banner the reader acknowledges before the answer appears.

The final 30 days went on the currency flag and the administrator queue. Every policy carries its review date from ingest, and when an answer draws on a policy past the team’s 18-month review threshold, a banner the reader has to acknowledge comes up before the answer panel renders.

Documents without an extractable review date default to the upload date and are flagged for manual review in the administrator queue, so the worst case is a false positive rather than a quiet stale answer. The same interface lets policy owners reassign role permissions and recover failed ingests.

We cut the answer panel back to the source clause and the flag, and made the flag the most prominent thing on it. The 18-month threshold is the team’s own published review cadence, not a number we invented.

“The thing that changed was not that the team got faster answers. It was that they stopped quietly trusting documents nobody had checked in years. The flag did that, more than the retrieval did.”

Compliance Operations Lead, AU enterprise compliance team (anonymised at the team’s request)

Sign-off · Rollout review

What the compliance lead wanted before she signed off the rollout.

Each figure here describes how the pipeline behaves, checked against the project’s success-metric definitions and the rollout review with the compliance lead. None is a usage or survey result.

Once the flag showed on every answer, the team could see which policies were silently overdue, and which of the most-read documents were also the most stale. Policy maintenance moved from a calendar-driven exercise to a usage-driven one.

Anonymised under the standard case-study disclosure: vertical, architectural choices and the currency-flag behaviour are real; the industry, the team’s name, the department descriptors and the model-provider choice have been generalised.

0

cross-role leaks across the validation suite of role-mismatched queries

Tested behaviour

What the validation suite checked before rollout, with the boundary held in Postgres row-level security.

100%

of answers from a policy past its 18-month review threshold carry the currency flag, shown before the answer panel renders

Tested behaviour

Pipeline behaviour checked against the project’s success-metric definitions, not a usage result.

100%

of answers return with the source clause inline, so the reader can check the wording without opening the document

Tested behaviour

How every answer is built, not a survey of readers.

Have a policy library nobody is sure is still current?

A 45-minute audit. We look at the documents your team relies on, where the currency dates live, and whether anyone is checking them at the moment of answer. You get a written follow-up within 48 hours whether or not we are the right fit.