Compliance Scanning

1. What it does

Aegis scans a landing page's live copy against CAP advertising rules and records the result against the route. A pass is one of the four approvals that gate route activation — the AI Evaluator tick, stored as sign_off_ai.

Scanning runs through Passmark. It returns real CAP rule ids, a risk level per finding, the offending line and suggested wording — not free-form prose.

The in-house Compliance AI worker was retired on 2026-09-05. It scanned copy with an LLM and passed almost everything, which meant a page could collect an AI-Evaluator sign-off it had not earned. Passmark is now the only engine.

Passmark is its own product now

Passmark started as the capcode scanner inside Aegis and was spun out into a standalone application with its own repository and its own home at passmark.io. It is no longer a worker in this repo — if you go looking for compliance-ai/src/, that is why it is not there.

Aegis is a client of it, over the public API. There is no shared database and no service binding; the integration is one HTTP call:

POST https://passmark.io/api/v1/scans
Authorization: Bearer $PASSMARK_API_KEY
{ "url": "https://…", "rulebook_id": "…", "share": false }

shared/passmark.js is the whole adapter. It maps Passmark's response onto the shape Aegis already stored, which is why the route handler barely changed when the engine did.

Two consequences of it being a separate service worth knowing:

  • There is no fallback. If the scan cannot run, scanUrl throws and the caller returns 502 rather than recording a scan that did not happen. The old fallback answered passed: true, violations: [] on pages Passmark fails — so a billing problem would have been stored as a compliance pass and would have flipped sign_off_ai.
  • Quota is not a verdict. A 402 or 429 is reported as a quota problem, never as a page failing its CAP check.

2. How a scan runs

  • Triggered from Admin → the route → Compliance, or by the cap-scan endpoint.
  • Passmark scans the live URL, not pasted HTML. The page must be publicly reachable — an unpublished Shopify preview cannot be scanned at all.
  • If the route has a Preview URL set, that URL is scanned instead of the computed live one, the same as everywhere else in Aegis.
  • Results are written to ai_cap_audits and shown in the Compliance tab, with a link to the full Passmark report.

3. What "passed" means — read this before trusting a tick

Two different verdicts are recorded, deliberately:

Field Meaning
verdict Passmark's own answer — did the page pass its rulebook?
passed What Aegis's gate decided, using the configured pass policy

PASSMARK_PASS_POLICY chooses between them:

Policy A page passes when
verdict (default) Passmark itself says the page passed
no-high-risk No finding is high or critical risk, even if Passmark failed it

They disagree in practice: Passmark fails pages that the older "no high-risk violation" rule passes. That is a policy choice, not a bug — and it governs sign_off_ai, so it decides whether a route can be activated. The UI shows both, so "Passmark says FAILED, gate says pass" is visible rather than hidden.

4. When a scan cannot run

A failed scan is not a failed page. If Passmark cannot be reached, the quota is exhausted, or the API key is rejected, the endpoint returns 502, writes nothing to ai_cap_audits, and leaves sign_off_ai exactly as it was.

There is no fallback engine, on purpose. The old fallback answered passed: true, violations: [] on pages Passmark fails, so an exhausted quota would have been recorded as a compliance pass. A scan that did not happen must not change an approval.

Check Settings → Health for Passmark connectivity, quota and key status. The error message on screen names which of the three it was.

5. Reports and history

  • Report links are minted at scan time and cannot be recovered later, so Aegis asks for one by default. Set passmark_share_reports = 'false' in app_settings to opt out — after which older reports stay reachable and new ones do not.
  • ai_cap_audits holds the history. Both the current and the retired engine wrote the same shape, so historic rows still render.
  • A human can override a verdict; that is recorded in routes.ai_overridden and the vote/feedback endpoints, and it does not silently rewrite the scan.

6. Settings

Setting Where Purpose
PASSMARK_API_KEY Worker secret Auth. Missing = no scanning at all, reported as a FAIL in Health
PASSMARK_RULEBOOK_ID Worker var Which rulebook to scan against
PASSMARK_PASS_POLICY Worker var verdict or no-high-risk — see §3
passmark_share_reports app_settings 'false' opts out of shareable report links

A leftover compliance_provider = 'compliance-ai' row in app_settings now does nothing — the worker it pointed at is gone. Health flags it if present, so nobody believes they have switched engines. Delete the row.