Why You Should Scan Documents for Secrets Before Indexing

Somewhere in your wiki, an engineer pasted a production API key into a troubleshooting page and moved on with their day. Run that page through a retrieval pipeline and the key becomes a chunk, then an embedding, then a possible answer. This article argues that you should scan documents for secrets before they reach the chunker, and shows a small ingestion step that redacts or quarantines what it finds.

Your wiki is already a secrets store

Debugging happens in tickets, chats and wiki pages, so that is where credentials get pasted. GitGuardian's State of Secrets Sprawl 2025 reports that 6.1% of Jira tickets exposed credentials and 2.4% of corporate Slack channels contained leaked secrets. It also rated 38% of incidents in collaboration and project tools as highly critical or urgent, against 31% for source control.

Those are the places RAG (retrieval-augmented generation) projects love to index, because they hold the answers people actually want. Nobody browses 40,000 tickets, but anyone can ask a chatbot "what's the staging database password?"

Indexing also makes a copy. The secret now lives in chunk text, the vector store, its snapshots, and probably your prompt logs and traces. Rotating the original key does not clean those copies. OWASP files this class of problem under LLM02:2025, Sensitive Information Disclosure.

Why scan documents for secrets before chunking

Three reasons, in rough order of how much they hurt:

• Chunking can hide a key. A splitter that cuts a long token or a multi-line private key in half leaves two fragments that no pattern matches. The whole document still matches.
• Embedding is a copy you can't cheaply take back. Once text is embedded, it sits in the index, the backups and the caches. Not writing it costs one function call.
• The scan is cheap and local. It is pattern matching on text, so it runs before you pay for a single embedding call.

Add a gitleaks step to ingestion

Gitleaks is an open-source scanner. Its dir command scans directories or files, so it works on plain text exported from a wiki, not only on git repos. Install it with brew install gitleaks, or use the ghcr.io/gitleaks/gitleaks:latest Docker image.

The function below writes extracted text to a temp directory, scans it once, and returns findings per document. It asks gitleaks to use exit code 2 for findings, so any other non-zero code means the scanner itself broke. In that case the pipeline stops instead of indexing unscanned text.

import json
import subprocess
import tempfile
from pathlib import Path

LEAKS_FOUND = 2  # exit code we ask gitleaks to use when it finds something


def scan_batch(docs: dict[str, str]) -> dict[str, list[dict]]:
    # docs maps a filename-safe doc id (a content hash works) to extracted plain text
    with tempfile.TemporaryDirectory() as tmp:
        src = Path(tmp) / "src"
        src.mkdir()
        for doc_id, text in docs.items():
            (src / f"{doc_id}.txt").write_text(text, encoding="utf-8")

        report = Path(tmp) / "report.json"
        proc = subprocess.run(
            [
                "gitleaks", "dir", str(src), "--no-banner",
                "--exit-code", str(LEAKS_FOUND),
                "--report-format", "json",
                "--report-path", str(report),
            ],
            capture_output=True,
            text=True,
        )
        if proc.returncode == 0:
            return {}
        if proc.returncode != LEAKS_FOUND:
            raise RuntimeError(f"gitleaks exited {proc.returncode}; refusing to index")
        findings = json.loads(report.read_text(encoding="utf-8"))

    by_doc: dict[str, list[dict]] = {}
    for f in findings:
        by_doc.setdefault(Path(f["File"]).stem, []).append(f)
    return by_doc

The temp directory, and the report with it, is deleted when the block exits. Keep it that way, because the findings carry the raw secrets.

Redact or quarantine what it finds

A document with one or two hits can usually be cleaned. A document with many is usually a config dump or a credentials page, and no text edit makes it trustworthy. Those go to a human.

import logging

log = logging.getLogger("ingest")
QUARANTINE_AT = 5  # this many hits in one document: a human decides


def redact(text: str, hits: list[dict]) -> str:
    # longest first, so a short secret never breaks up a longer one
    for secret in sorted({h["Secret"] for h in hits}, key=len, reverse=True):
        text = text.replace(secret, "[REDACTED]")
    return text


def prepare_for_index(docs: dict[str, str]) -> dict[str, str]:
    findings = scan_batch(docs)
    safe: dict[str, str] = {}
    for doc_id, text in docs.items():
        hits = findings.get(doc_id, [])
        if not hits:
            safe[doc_id] = text
            continue
        rules = sorted({h["RuleID"] for h in hits})  # log rule ids, never the secret
        if len(hits) >= QUARANTINE_AT:
            log.warning("quarantined %s: %d hits, rules=%s", doc_id, len(hits), rules)
            continue
        cleaned = redact(text, hits)
        if scan_batch({doc_id: cleaned}):  # still dirty after redaction
            log.warning("quarantined %s after redaction, rules=%s", doc_id, rules)
            continue
        log.info("redacted %s: %d hits, rules=%s", doc_id, len(hits), rules)
        safe[doc_id] = cleaned
    return safe


# in the pipeline, between text extraction and chunking
for doc_id, text in prepare_for_index(extracted).items():
    chunks = split_into_chunks(text)  # your existing chunker
    index.upsert(doc_id, chunks)      # your existing embed and store step

The rescan after redaction is the part people skip. It proves the cleaned text is clean instead of assuming it.

But access control already covers this, right?

Partly, and you need it. Access control decides who can open a document. It doesn't know that a page shared with the whole engineering org contains a credential meant for three people. Retrieval is very good at finding the one page nobody ever opened. Chunk-level permissions in a vector store are also one more thing that can drift out of sync with the source.

The scan covers the case where permissions are right and the content is wrong. It does not replace rotation: redacting the index doesn't fix the wiki page. Send each finding's owner a ticket to rotate the key.

Gotchas and alternatives

detect-secrets is another open-source option. detect-secrets scan docs/ --all-files > .secrets.baseline scans files that git doesn't track. Its docs say it only implements heuristics for obvious cases and misses multi-line secrets and passwords like login = "hunter2". Treat any scanner as a filter, not a guarantee.

• The report holds real secrets. Keep it in a temp directory and never log it.
• Big files can slip through. Gitleaks skips files larger than --max-target-megabytes, so set the flag explicitly and split huge exports.
• Scan extracted text. Run it after PDF and OCR conversion, on the text you will actually chunk.
• Rescan on every re-index. Wikis keep getting edited, and new keys keep getting pasted.
• Start in report-only mode. Log what would be quarantined for a week to see your false-positive rate, then tune with a config file via --config.
• Plan for a successor. The gitleaks README says it is feature complete and gets security patches only, with its author's focus now on Betterleaks. Pin the version and keep the scanner behind your own function so swapping is cheap.

Start with an audit

Export one wiki space as text and run this:

gitleaks dir ./wiki-export --no-banner --report-format json --report-path wiki-findings.json

Count findings by rule, rotate the live ones, then delete the report because it describes real secrets. After that, put the function above in front of your chunker.

A retrieval system is only as safe as the least careful page it indexes, so the scan belongs in the ingestion pipeline, not in a hope that nobody asks the right question.