Saqr: grounded RAG over Saudi cybersecurity frameworks

Why I built it

Claims about KSA regulation are easy to make and hard to verify. Saqr makes my knowledge testable: ask a question and it shows the official sources used to ground its response. A plausible but wrong regulatory answer carries real consequences, so I designed Saqr to be checkable, not merely convincing.

What it does

Ask a question in Arabic or English. An alias map translates common Arabic phrasings into the corpus’s English keyword space. A length-weighted keyword scorer ranks corpus entries. The top three go to Claude as [SOURCE 1] through [SOURCE 3] blocks, with a grounding prompt that bans em-dashes, sycophancy, and made-up statistics. No API key? The top match is returned verbatim. Run the CLI or MCP tools and you see exactly which entry answered the question.

Grounded, visibly

Every corpus entry cites its official source: the NCA controls library, the SAMA Rulebook, CST’s regulatory framework, Aramco’s Third Party Cybersecurity Standard (SACS-210, the number Aramco’s CCC program cites today), SDAIA’s PDPL text, and the ISO/IEC catalogue. The chatbot renders those citations under each answer as clickable source chips, in both languages. That’s retrieval-level attribution: the answer is grounded in the entries shown, and you can click through and check the claim against the regulator’s own page. A lint gate in CI enforces the citation schema, verifies every URL resolves to the official domain, and screens all client-facing text against an injection blocklist, so the corpus can’t quietly drift away from its sources.

Design decisions

Deterministic retrieval over embeddings. Keyword scoring is reproducible and auditable: the same question always retrieves the same entries, and the eval suite can pin that behavior in CI. Saudi regulator documents are dense, and general-purpose embeddings misread them constantly, especially cross-framework questions (“do ECC and CCC overlap here?”). A small corpus written by a practitioner gets higher precision for the same effort, and comparisons are first-class entries rather than lucky paragraph adjacencies.

Arabic first, byte-level pinned. Arabic ranking depends on UTF-8 byte-length weighting; a characterization test pins that semantics so a refactor can’t silently re-rank Arabic queries. The alias map handles real phrasings: hamza variants, definite-article assimilation, Latin acronyms embedded in Arabic sentences.

The LLM layer is optional and fenced. The generator sees only the retrieved entries, its HTML output is rebuilt through a whitelist sanitizer, and the corpus itself is screened against prompt-injection tokens at lint time. Without a key, Saqr degrades to curated answers verbatim.

Corpus as code. Entry IDs are frozen in a lock file. Schema, style rules, citation shape, and URL reachability are all enforced by lint in CI. Content changes are diffs you can review, not database edits.

Measured, not claimed

The retrieval eval suite holds 137 questions (68 English, 69 Arabic) covering every corpus entry, with a committed baseline that CI refuses to regress. Over a corpus this small, high hit@1 (currently 0.956 English / 0.971 Arabic, MRR 0.971) is the floor, not the achievement: the point of the suite is that a refactor cannot silently re-rank Arabic queries, and that every known miss is documented with a root cause. One caveat, straight from the repo’s eval docs: the Arabic figure measures alias coverage over this question set, not open-domain Arabic understanding; the known misses are documented with root causes. The full test suite (unit, characterization, snapshot, smoke, eval, and the MCP server) runs in CI on every push.

Saqr in action

Coverage

• NCAECC, CCC, CSCC, DCC, TCC, OTCC, OSMACC, SCyWF
• SAMACSF, ITGF, BCM
• CSTCRF
• Saudi AramcoSACS-210 / CCC (contractor)
• SDAIAPDPL
• InternationalISO 27001
• Cross-frameworkECC↔ISO, ECC↔CCC, ECC↔CSCC, SAMA↔NCA, PDPL↔DCC, Aramco↔NCA
• Practitioner adviceWhere to start · maturity · audit · third party

Stack & openness

PHP 8.2 or newer, tested through 8.3. Apache-2.0 with a patent grant. The corpus is plain JSON, so anyone reading the repo can see exactly what Saqr knows and where each claim comes from. It also ships as an MCP server, so Claude Desktop or Cursor can query the corpus as tools. The stack has no database or embeddings-service dependency and no vendor lock-in. The Anthropic key is optional, read from an environment variable, never hard-coded.