RAG & PERMISSIONS / 02

A document’s permissions must travel with it.

By Sachin Gupta7 min read

A photocopy does not acquire new readers.

Portrait of Sachin Gupta rendered in binary

The records clerk at Sachin, our fictional company, sends the Engineering incident note to the copy room. The note is long, so the clerk wants two useful extracts instead of the whole folder.

The copier returns two loose sheets. Neither carries the Engineering label from the folder. Has the information become available to Support?

Of course not. Copying the paper changed its shape, not who may read it. Our software needs to preserve that fact explicitly.

In Part 1, Maya’s Support badge denied her access to D2. Here, you will make that decision survive the journey from one document to several searchable pieces.

AT THE RECORDS DESK · FICTIONAL COMPANY SACHIN

What should leave the copy room with every sheet?

1One labeled folder

D2 belongs to Sachin. Its label grants Engineering access.

Folder label → source permission metadata

2Two labeled copies

D2.1 and D2.2 each carry the company, grants, parent ID, and version.

Copy operation → chunk ingestion

3The same decision

Arun may use both. Maya may use neither. A missing label stops ingestion.

Preserved labels → preserved authorization

The text becomes smaller. Its audience stays the same.

The labels are part of the record

A chunk is a piece of a document used as a retrieval unit. Metadata is structured information stored alongside its text. An access control list, or ACL, records grants to people or groups. The index is the searchable collection of these records.

Our copy room maps to this representation:

Scroll sideways to see every column.

In the storyIn the recordExample
Company buildingtenantIdsachin
Original folderparentIdD2
Individual sheetidD2.1
Groups on the labelallowedGroups["engineering"]
Named readersallowedUsers[]
Edition of the permission stampaclVersion1

The permission rule remains same company AND (a direct user grant OR a matching group). This deliberately small policy has no explicit deny rules, nested groups, or inherited folder hierarchy. Those would need their own defined evaluation rules.

Storing permission identifiers is only one part of the mechanism. Microsoft’s security-filter documentation distinguishes matching those identifiers from authenticating the person presenting them. A string saying engineering does not prove Engineering membership. Microsoft security filters.

Two trusted inputs meet at the check

Imagine one desk issues badges and another maintains folder labels. The records clerk trusts those desks, not a visitor’s handwritten replacement label.

In a real application, an authorized ingestion process reads permission metadata from the source system. It does not infer access from document prose. Separately, a verified session supplies the asker’s identity and current group membership. A request body that says groups: ["engineering"] is not that verification.

Those are two integrations to build: the source-permission adapter and the identity adapter. This browser lesson supplies fictional fixtures for both. Enforcement belongs in trusted application code, with access denied unless the policy grants it. OWASP authorization guidance.

Follow D2 through the copy room

Our example splits D2 into two passages: one about sign-in timeouts and one about the session refresh fix. Both retain tenantId: "sachin", parentId: "D2", Engineering access, no direct user grants, and policy version 1.

Maya is denied both pieces. Arun is allowed both. A chunk ID changes which piece we found; it does not change the grant.

This example assumes the whole document has one policy. If sections have different permissions, identify those boundaries before chunking. Joining a restricted paragraph to a broadly readable paragraph and applying the broader label would change who can receive the restricted text.

Predict: if a copied sheet loses its group label, should the ingestion process treat it as public, deny it, or reject the incomplete record? Try the missing-label case.

YOUR TURN · CHANGE ONE INPUT

Send D2 through the copy room.

Fictional records. Browser simulation. No live model call.

SOURCE FOLDER

D2

Engineering incident note

tenant: sachin
groups: ["engineering"]
users: []
D2.1× Deny

Sign-in requests time out under peak load.

Parent D2 · Sachin · policy v1

Maya: No matching permission

D2.2× Deny

The identity team is fixing the session refresh path.

Parent D2 · Sachin · policy v1

Maya: No matching permission

0 of 2 copies allowed for Maya. Each copy is evaluated using its inherited company and grants.

Inspect the input, rule, and output

Input / state

{
  "principal": {
    "id": "maya",
    "name": "Maya",
    "role": "Support",
    "tenantId": "sachin",
    "groups": [
      "support"
    ]
  },
  "source": {
    "id": "D2",
    "title": "Engineering incident note",
    "team": "Engineering",
    "tenantId": "sachin",
    "allowedGroups": [
      "engineering"
    ],
    "allowedUsers": [],
    "match": 98,
    "fact": "Sign-in requests time out under peak load. The identity team is fixing the session refresh path."
  },
  "paragraphs": [
    "Sign-in requests time out under peak load.",
    "The identity team is fixing the session refresh path."
  ]
}

Decision rule

validate permission metadata
copy tenant + grants onto each chunk
allow = sameCompany && (directGrant || groupGrant)

Output

{
  "chunks": [
    {
      "id": "D2.1",
      "title": "Engineering incident note",
      "team": "Engineering",
      "tenantId": "sachin",
      "allowedGroups": [
        "engineering"
      ],
      "allowedUsers": [],
      "match": 98,
      "fact": "Sign-in requests time out under peak load.",
      "parentId": "D2",
      "aclVersion": 1
    },
    {
      "id": "D2.2",
      "title": "Engineering incident note",
      "team": "Engineering",
      "tenantId": "sachin",
      "allowedGroups": [
        "engineering"
      ],
      "allowedUsers": [],
      "match": 98,
      "fact": "The identity team is fixing the session refresh path.",
      "parentId": "D2",
      "aclVersion": 1
    }
  ],
  "decisions": [
    {
      "id": "D2.1",
      "hasIdentity": true,
      "sameCompany": true,
      "directGrant": false,
      "groupGrant": false,
      "matchingGroups": [],
      "allowed": false,
      "reason": "No matching permission"
    },
    {
      "id": "D2.2",
      "hasIdentity": true,
      "sameCompany": true,
      "directGrant": false,
      "groupGrant": false,
      "matchingGroups": [],
      "allowed": false,
      "reason": "No matching permission"
    }
  ]
}

This view runs the same deterministic functions as the downloadable example. The short rule above summarizes the operation; the download contains the complete implementation.

Missing is different from empty

An empty list is an explicit value. allowedGroups: [] means this list grants access to no groups. A direct user grant can still allow a named person. With both grant lists empty, our rule allows nobody.

A missing list means the record is incomplete. The ingestion example rejects it. It does not guess whether the connector meant “everyone,” “nobody,” or “the fetch failed.” That distinction makes a broken metadata fetch visible instead of silently widening access.

The same rule applies to malformed lists and a missing company identifier. A source connector must deliver valid policy metadata before these chunks enter the usable collection.

Build the copy operation

The download contains ingestDocument, the permission rule from Part 1, and assertions for both readers. It copies the grant arrays so changing a chunk’s array does not accidentally mutate the parent folder’s array.

FROM THE PICTURE TO A RUNNING EXAMPLE

Run the same model yourself.

Node.js 20 or newer. No packages or API keys. Save the file, open a terminal in its directory, and run the command below.

Download part 2 · JavaScript ↓
node sachin-permissions-2.mjs

EXPECTED OUTPUT

D2.1: Maya deny; Arun allow
D2.2: Maya deny; Arun allow
Missing labels: rejected
Ingestion checks passed.
Read the complete source and assertions
// Sachin RAG permissions, part 2. Node.js 20+. Fictional fixtures; fixed answers.
// @ts-check
/** Fictional teaching fixtures. This simulation is not an authorization boundary.
 * @typedef {{id: string, name: string, role: string, tenantId: string, groups: readonly string[]}} Principal
 * @typedef {{id: string, title: string, team: string, tenantId: string, allowedGroups: readonly string[], allowedUsers: readonly string[], match: number, fact: string}} Evidence
 */

/** @type {readonly Principal[]} */
export const PEOPLE = [
  { id: "maya", name: "Maya", role: "Support", tenantId: "sachin", groups: ["support"] },
  { id: "arun", name: "Arun", role: "Engineering", tenantId: "sachin", groups: ["engineering"] },
];

export const QUESTION = "Why is the Orion launch delayed?";

/** @type {readonly Evidence[]} */
export const DOCUMENTS = [
  {
    id: "D1", title: "Customer launch update", team: "Shared", tenantId: "sachin",
    allowedGroups: ["support", "engineering"], allowedUsers: [], match: 87,
    fact: "Orion’s launch has moved to 14 October while the team completes reliability checks.",
  },
  {
    id: "D2", title: "Engineering incident note", team: "Engineering", tenantId: "sachin",
    allowedGroups: ["engineering"], allowedUsers: [], match: 98,
    fact: "Sign-in requests time out under peak load. The identity team is fixing the session refresh path.",
  },
  {
    id: "D3", title: "Customer support FAQ", team: "Support", tenantId: "sachin",
    allowedGroups: ["support"], allowedUsers: [], match: 82,
    fact: "Existing workspaces stay available. Customers do not need to take any action.",
  },
  {
    id: "D4", title: "Launch cost review", team: "Finance", tenantId: "sachin",
    allowedGroups: ["finance"], allowedUsers: [], match: 91,
    fact: "The delay adds $24,000 to the internal launch budget.",
  },
];

// @example:policy
/** Identity must already come from a trusted session; this function only checks permission.
 * @param {Principal | null} user @param {Evidence} chunk */
export function checkAccess(user, chunk) {
  const hasIdentity = user !== null;
  const sameCompany = hasIdentity && user.tenantId === chunk.tenantId;
  const directGrant = hasIdentity && chunk.allowedUsers.includes(user.id);
  const matchingGroups = user?.groups.filter(group => chunk.allowedGroups.includes(group)) ?? [];
  const groupGrant = matchingGroups.length > 0;
  const allowed = sameCompany && (directGrant || groupGrant);
  const reason = !hasIdentity ? "No verified identity"
    : !sameCompany ? "Different company"
    : directGrant ? "Direct user grant"
    : groupGrant ? `${matchingGroups.join(", ")} group grants access`
    : "No matching permission";

  return { hasIdentity, sameCompany, directGrant, groupGrant, matchingGroups, allowed, reason };
}
// @example:end-policy

/** @param {Principal | null} user @param {readonly Evidence[]} documents */
export function retrieveEvidence(user, documents = DOCUMENTS) {
  return documents
    .filter(chunk => checkAccess(user, chunk).allowed)
    .sort((a, b) => b.match - a.match);
}

/** @param {Principal | null} user @param {readonly Evidence[]} documents */
export function buildAnswer(user, documents = DOCUMENTS) {
  const evidence = retrieveEvidence(user, documents);
  const context = evidence.map(chunk => `[${chunk.id}] ${chunk.title}\n${chunk.fact}`).join("\n\n");
  // These are example request messages. This lesson makes no model call.
  const messages = evidence.length ? [
    { role: "system", content: "Use the supplied evidence to answer the question. Cite each source ID. If the evidence is insufficient, say so." },
    { role: "user", content: `Question: ${QUESTION}\n\nPermitted context:\n${context}` },
  ] : [];

  return {
    evidence,
    context,
    messages,
    // Assemble fixed facts to make the experiment reproducible, without a model.
    sentences: evidence.map(({ id, fact }) => ({ sourceId: id, text: fact })),
    emptyMessage: evidence.length ? null : "I don’t have accessible evidence to answer this question.",
  };
}


/** Fictional, deterministic teaching models shared by the diagrams and downloads.
 * @typedef {import('./rag-permissions.mjs').Principal} Principal
 * @typedef {import('./rag-permissions.mjs').Evidence} Evidence
 */

export function hasPolicy(record) {
  const ids = value => Array.isArray(value) && value.every(id => typeof id === "string" && id.length > 0);
  return typeof record?.tenantId === "string" && record.tenantId.length > 0
    && ids(record.allowedGroups) && ids(record.allowedUsers);
}

export function ingestDocument(document, paragraphs, aclVersion = 1) {
  if (!hasPolicy(document)) throw new Error("Missing or invalid permission metadata");
  if (!Number.isInteger(aclVersion) || aclVersion < 1) throw new Error("Invalid policy version");
  if (!Array.isArray(paragraphs) || paragraphs.some(text => typeof text !== "string" || !text.trim())) {
    throw new Error("Chunks must contain nonempty text");
  }
  return paragraphs.map((fact, index) => ({
    ...document, id: `${document.id}.${index + 1}`, parentId: document.id,
    fact, aclVersion, allowedGroups: [...document.allowedGroups], allowedUsers: [...document.allowedUsers],
  }));
}

// Ten fictional passages extend the same four documents. Scores are illustrative, not vector distances.
/** @type {Array<[string, number, number, string]>} */
const rankingRows = [
  ["D2", 1, 98, "Sign-in requests time out under peak load."],
  ["D4", 1, 96, "The delay adds $24,000 to the internal launch budget."],
  ["D1", 1, 94, "Orion’s launch has moved to 14 October."],
  ["D2", 2, 92, "The identity team is fixing the session refresh path."],
  ["D3", 1, 90, "Existing workspaces stay available."],
  ["D1", 2, 88, "The team is completing reliability checks."],
  ["D3", 2, 86, "Customers do not need to take any action."],
  ["D1", 3, 84, "The launch update will be revised after the next readiness review."],
  ["D4", 2, 82, "Finance will reconcile launch costs after release."],
  ["D3", 3, 80, "Support will share the next approved launch update."],
];
export const RANKED_CHUNKS = rankingRows.map(([parentId, ordinal, match, fact]) => {
  const parent = DOCUMENTS.find(document => document.id === parentId);
  if (!parent) throw new Error("Unknown parent document");
  return { ...parent, id: `${parentId}.${ordinal}`, parentId, match, fact, aclVersion: 1 };
});

/** Exact ranked-list experiment; it does not simulate an ANN graph. */
export function compareRetrieval(user = PEOPLE[0], k = 5, fetchCount = 5) {
  const ranked = [...RANKED_CHUNKS].sort((a, b) => b.match - a.match);
  const isAllowed = chunk => hasPolicy(chunk) && checkAccess(user, chunk).allowed;
  const eligible = ranked.filter(isAllowed);
  const ideal = eligible.slice(0, k);
  const fetched = ranked.slice(0, fetchCount);
  const after = fetched.filter(isAllowed).slice(0, k);
  const recovered = after.filter(chunk => ideal.some(target => target.id === chunk.id)).length;
  return { ranked, eligible, ideal, fetched, after, missed: ideal.filter(chunk => !after.some(hit => hit.id === chunk.id)),
    recall: ideal.length ? recovered / ideal.length : null };
}

/** Each stage is a new snapshot; moving backwards never mutates the fixtures. */
export function revocationSnapshot(stage = 0, enforceCurrent = true) {
  const original = DOCUMENTS[1];
  const source = { ...original, allowedGroups: stage >= 1 ? [] : [...original.allowedGroups], aclVersion: stage >= 1 ? 2 : 1 };
  const indexed = stage >= 2 ? { ...source } : { ...original, aclVersion: 1 };
  const indexAllows = checkAccess(PEOPLE[1], indexed).allowed;
  const currentAllows = checkAccess(PEOPLE[1], source).allowed;
  const oldCacheExists = stage < 3;
  const cacheVersion = 1;
  const cacheUsable = oldCacheExists && cacheVersion === source.aclVersion && currentAllows;
  return { source, indexed, indexAllows, currentAllows, oldCacheExists, cacheUsable,
    contextIncludesD2: indexAllows && (!enforceCurrent || currentAllows),
    cacheVersion, policyVersion: source.aclVersion };
}

/** Source IDs are looked up and authorized again on every source request. */
export function readSource(user, id, documents = DOCUMENTS) {
  const document = documents.find(item => item.id === id && item.tenantId === user?.tenantId);
  if (!document || !hasPolicy(document) || !checkAccess(user, document).allowed) {
    return { status: 404, body: { message: "Source unavailable" } };
  }
  return { status: 200, body: { id: document.id, title: document.title, text: document.fact } };
}

export function inspectSurface(personId = "maya", surface = "citation", enforce = true) {
  const person = PEOPLE.find(user => user.id === personId) ?? PEOPLE[0];
  const permitted = buildAnswer(person);
  const restricted = DOCUMENTS[1];
  const safe = surface === "citation" ? readSource(person, "D2")
    : surface === "preview" ? { status: 200, body: permitted.evidence.map(({ id, title }) => ({ id, title })) }
    : surface === "shared-answer" ? { status: 200, body: permitted.sentences }
    : { status: 200, body: { requestId: "demo-1", outcome: "allowed", returnedIds: permitted.evidence.map(chunk => chunk.id) } };
  const unsafe = surface === "citation" ? { status: 200, body: { id: restricted.id, title: restricted.title, text: restricted.fact } }
    : surface === "preview" ? { status: 200, body: DOCUMENTS.map(({ id, title }) => ({ id, title })) }
    : surface === "shared-answer" ? { status: 200, body: buildAnswer(PEOPLE[1]).sentences }
    : { status: 200, body: { question: QUESTION, candidates: DOCUMENTS, sessionToken: "fictional-demo-token" } };
  return { person, response: enforce ? safe : unsafe, protected: enforce };
}

export function createServiceState() {
  return {
    // Public exercise credentials, not secrets or a real login system.
    sessions: { "support-demo": structuredClone(PEOPLE[0]), "engineering-demo": structuredClone(PEOPLE[1]),
      "other-tenant-demo": { ...structuredClone(PEOPLE[0]), tenantId: "other" } },
    documents: structuredClone(DOCUMENTS), policyVersion: 1, policyAvailable: true,
    cache: new Map(), audit: [],
  };
}

export function revokeD2(state) {
  const document = state.documents.find(item => item.id === "D2");
  document.allowedGroups = [];
  document.allowedUsers = [];
  state.policyVersion += 1;
}

export function prepareServiceRequest(state, token, body = {}) {
  const user = Object.hasOwn(state.sessions, token) ? structuredClone(state.sessions[token]) : null;
  const base = { user, sessionKey: token, evidence: [], version: state.policyVersion, key: "", query: "", trace: ["Resolve identity from the server’s session table"] };
  if (!user) return { ...base, error: 401, message: "Sign in required" };
  if (typeof body?.question !== "string" || body.question.trim() !== QUESTION) {
    return { ...base, error: 400, message: "This exercise supports the Orion launch question only" };
  }
  if (!state.policyAvailable) return { ...base, error: 503, message: "Permission check unavailable" };
  if (state.documents.some(document => !hasPolicy(document))) {
    return { ...base, error: 503, message: "Permission metadata unavailable" };
  }
  // Body-supplied groups and tenantId never participate in the decision.
  const evidence = buildAnswer(user, state.documents).evidence;
  const key = JSON.stringify(["fixed-generator-v1", state.policyVersion, user.tenantId, user.id, [...user.groups].sort(), QUESTION]);
  return { ...base, error: 0, message: "", evidence, query: QUESTION, key,
    trace: [...base.trace, "Ignore client permission claims", "Check tenant and current document grants", "Prepare permitted evidence"] };
}

export function finishServiceRequest(state, plan) {
  const reply = (status, body, cacheHit = false) => {
    // This audit record stays on the server. No token, prompt, title, or document body.
    state.audit.push({ requestId: `request-${state.audit.length + 1}`, status,
      policyVersion: state.policyVersion, returnedIds: status === 200 ? (body.sources ?? []).map(source => source.id) : [] });
    return { status, body, cacheHit, trace: plan.trace };
  };
  if (plan.error) return reply(plan.error, { message: plan.message });
  if (!state.policyAvailable) return reply(503, { message: "Permission check unavailable" });
  if (plan.version !== state.policyVersion) return reply(409, { message: "Permissions changed; retry the request" });
  // Recheck current membership too. The version check covers policy changes through revokeD2.
  const current = Object.hasOwn(state.sessions, plan.sessionKey) ? state.sessions[plan.sessionKey] : null;
  if (!current || current.id !== plan.user.id || current.tenantId !== plan.user.tenantId
    || JSON.stringify([...current.groups].sort()) !== JSON.stringify([...plan.user.groups].sort())
    || plan.evidence.some(chunk => {
    const source = state.documents.find(document => document.id === chunk.id);
    return !source || !hasPolicy(source) || !checkAccess(current, source).allowed;
  })) return reply(409, { message: "Permissions changed; retry the request" });
  if (!plan.evidence.length) return reply(200, { answer: "I don’t have accessible evidence to answer this question.", sources: [], modelCalled: false });
  const cached = state.cache.get(plan.key);
  // Compare dependencies too: a missed revision update must not restore an excluded source.
  const selectedIds = plan.evidence.map(chunk => chunk.id);
  if (cached && JSON.stringify(cached.sources.map(source => source.id)) === JSON.stringify(selectedIds)) {
    return reply(200, structuredClone(cached), true);
  }
  // Deterministic generation seam: a real model integration needs separate output evaluation.
  const response = { answer: plan.evidence.map(chunk => `${chunk.fact} [${chunk.id}]`).join("\n\n"),
    sources: plan.evidence.map(({ id, title }) => ({ id, title })), modelCalled: false };
  state.cache.set(plan.key, structuredClone(response));
  return reply(200, response);
}

export function runServiceScenario(scenario = "maya") {
  const state = createServiceState();
  let token = scenario === "arun" || scenario === "in-flight" || scenario === "revoked" ? "engineering-demo" : "support-demo";
  if (scenario === "missing-session") token = "unknown";
  if (scenario === "other-tenant") token = "other-tenant-demo";
  if (scenario === "outage") state.policyAvailable = false;
  const body = { question: QUESTION, ...(scenario === "forged-groups" ? { groups: ["engineering"], tenantId: "other" } : {}) };
  if (scenario === "revoked") {
    finishServiceRequest(state, prepareServiceRequest(state, token, body));
    revokeD2(state);
  }
  const plan = prepareServiceRequest(state, token, body);
  if (scenario === "in-flight") revokeD2(state);
  const response = finishServiceRequest(state, plan);
  return { body, plan, response, audit: state.audit, cacheEntries: state.cache.size, policyVersion: state.policyVersion };
}

import assert from "node:assert/strict";
const chunks = ingestDocument(DOCUMENTS[1], ["Sign-in requests time out under peak load.", "The identity team is fixing the session refresh path."]);
for (const chunk of chunks) {
  assert.equal(checkAccess(PEOPLE[0], chunk).allowed, false);
  assert.equal(checkAccess(PEOPLE[1], chunk).allowed, true);
  assert.equal(chunk.parentId, "D2");
  assert.equal(chunk.aclVersion, 1);
  console.log(chunk.id + ": Maya deny; Arun allow");
}
assert.throws(() => ingestDocument({ ...DOCUMENTS[1], allowedGroups: undefined }, ["A passage"]), /permission metadata/);
console.log("Missing labels: rejected");
console.log("Ingestion checks passed.");

Run the file, then add allowedUsers: ["maya"] to the source document passed to ingestDocument. Both copies should now allow Maya. The company check still applies: a person with the same ID in a different company must remain denied.

The function accepts already selected text passages. Parsing files, choosing useful chunk boundaries, retrying connector failures, and maintaining stable IDs across re-ingestion are separate ingestion work. The permission invariant stays the same across those choices: copying an extract must not create a new grant.

A stamped version is not a freshness check

Version 1 tells us which policy a copy carries. It cannot prove that version 1 is still current. In Part 4, we will compare a stale copy with a changed source policy.

First, our correctly labeled sheets need to be found. Continue to Part 3: find the best evidence this person may read.

Related

Tagged