Documentation
POC Verification Checklist
Project-Agent / poc/POC_CHECKLIST.md
This checklist maps each item to a thing the evaluator can run or click. The expectation is that a security/GRC reviewer can tick most of these inside the POC stack without escalating questions back to ARX.
Where a row maps to a SIG-Lite question we link the question number; items not covered here are answered in sig-lite-responses.md.
> Convention: every command runs from the repo root with the POC stack > already up (./poc/bootstrap.sh completed successfully). Replace > <email> / <password> with the admin creds printed at bootstrap time.
---
1. Tenant isolation (SIG E.1.1, E.1.4)
- [ ] Two-org cross-tenant test
Create a second org via POST /v1/admin/orgs (admin token); attempt to read its data with the first org's JWT — must return empty/403. ```bash curl -H "Authorization: Bearer <org1_token>" \ http://localhost:8000/v1/agents | jq '.[].org_id'
All ids must equal the org1 UUID. None from org2.
```
- [ ] RLS policy enumeration
``bash docker compose -f poc/docker-compose.poc.yml exec postgres \ psql -U postgres -d postgres -c "\\d+ users" | grep -i "Policies:" ` Must list at least one policy referencing current_setting('request.jwt.claims', true)::json->>'org_id'`.
2. Audit-log immutability — INV-001 (SIG H.1.x)
- [ ] UPDATE blocked by trigger
``bash docker compose -f poc/docker-compose.poc.yml exec postgres \ psql -U postgres -d postgres \ -c "UPDATE audit_log SET status='tampered' WHERE id IS NOT NULL;" ` Must error: ERROR: audit_log is immutable: UPDATE is not allowed`.
- [ ] DELETE blocked by trigger
Same as above with DELETE FROM audit_log; — same error.
- [ ] Triggers present after a fresh restore
python scripts/dr/integrity_check.py "$DATABASE_URL" returns audit_log_immutability_triggers PASS.
3. Encryption posture (SIG G.1.x)
- [ ] No dev encryption key in production
``bash docker compose -f poc/docker-compose.poc.yml exec api \ python -c "from app.config import get_settings; s=get_settings(); \ assert 'dev-encryption-key' not in s.encryption_key; print('ok')" ``
- [ ] Connector credentials encrypted at rest
``bash docker compose -f poc/docker-compose.poc.yml exec postgres \ psql -U postgres -d postgres \ -c "SELECT credentials_encrypted IS NOT NULL FROM connector_configs LIMIT 1;" ``
- [ ] Vault is the secret store
Hit Vault UI at http://localhost:8200; secrets/* paths exist under the configured mount.
4. SAML signature validation — Item 2 (SIG E.2.x)
- [ ] Valid signed assertion accepted
pytest arxsec-api/tests/test_saml_signature.py::test_valid_signed_response_verifies
- [ ] Tampered assertion rejected
pytest arxsec-api/tests/test_saml_signature.py::test_tampered_assertion_fails
- [ ] Replay rejected
pytest arxsec-api/tests/test_saml_signature.py::test_replay_protection
- [ ] End-to-end JIT against a real IdP
Follow poc/USER_ONBOARDING.md §3 to wire an Okta dev tenant; first SSO login should land a fresh public.users row with sso_provider='saml'.
5. SCIM 2.0 provisioning (SIG E.4.x)
- [ ] User CRUD via SCIM
./poc/scripts/scim-demo.sh walks ServiceProviderConfig → POST → GET → PATCH(active=false) → DELETE end-to-end using the bearer token that bootstrap.sh persists to poc/.env.poc. Equivalent manual curls and Okta wiring steps live in poc/USER_ONBOARDING.md §2.
6. Rate limiter — Items 3 & 10 (SIG D.1.x, K.1.x)
- [ ] Lua bucket limits to plan capacity
pytest arxsec-api/tests/test_rate_limiter_lua.py
- [ ] Writes fail closed when Redis is down
pytest arxsec-api/tests/test_rate_limiter_failclosed.py::test_writes_fail_closed_when_redis_raises
- [ ] **
/health/*exempt from limiter**
```bash for i in $(seq 1 200); do curl -s -o /dev/null -w '%{http_code}\n' \ http://localhost:8000/health/components; done | sort -u
Must show only 200 (or 503 if a dep is down) — never 429.
```
7. Retention + GDPR — Item 5 (SIG L.1.x, R.1.x)
- [ ] Retention guardrails enforced
pytest arxsec-api/tests/test_retention.py
- [ ] Deletion request anonymises the user record
```bash
Submit + run the worker manually
curl -X POST http://localhost:8000/v1/data-deletion \ -H "Authorization: Bearer <admin_token>" \ -d '{"target_email":"poc-deployer@example.com","reason":"DSAR"}' docker compose -f poc/docker-compose.poc.yml exec worker \ python -c "from app.runtime.scheduler import process_pending_deletion_requests; \ print(process_pending_deletion_requests())" docker compose -f poc/docker-compose.poc.yml exec postgres \ psql -U postgres -d postgres \ -c "SELECT email FROM users WHERE id = (SELECT target_user_id FROM deletion_requests LIMIT 1);"
Must show: deleted-<uuid>@redacted.local
```
8. Observability — Item 12 (SIG K.1.x, M.1.x)
- [ ] /metrics returns Prometheus format with bounded cardinality
``bash curl http://localhost:8000/metrics | grep arxsec_http_requests_total | head ``
- [ ] PII scrubber drops sensitive keys
pytest arxsec-api/tests/test_observability.py::test_scrubber_redacts_sensitive_keys
- [ ] Production refuses to boot without DD_API_KEY
pytest arxsec-api/tests/test_observability.py::test_production_requires_datadog
9. Backups + DR — Item 11 (SIG O.1.x)
- [ ] Integrity check passes on a clean restore
python scripts/dr/integrity_check.py "$DATABASE_URL" --expect-data-in orgs users
- [ ] DR runbook exists with RPO/RTO targets
docs/ops/disaster-recovery.md — RPO=24h / RTO=4h documented.
10. Trust + disclosure (SIG B.1.x)
- [ ] Trust page renders at
http://localhost:3000/trust. - [ ]
/.well-known/security.txtpresent and RFC 9116 valid
``bash curl http://localhost:3000/.well-known/security.txt | head ``
- [ ] Vulnerability-disclosure policy at
docs/security/vulnerability-disclosure.md. - [ ] Incident-response policy at
docs/security/incident-response.md.
11. Compliance package generation (SIG H.2.x)
- [ ] Generate a SOC 2 evidence PDF
``bash curl -X POST http://localhost:8000/v1/compliance/packages \ -H "Authorization: Bearer <admin_token>" \ -d '{"agent_id":"<agent_id>","framework":"soc2"}' \ -o evidence.pdf file evidence.pdf # PDF document ``
12. Billing entitlements — Item 9
- [ ] Subscription endpoint reflects POC plan
curl http://localhost:8000/v1/billing/subscription -H "Authorization: Bearer <admin_token>" plan: business, is_active: true, entitlements populated.
- [ ] Webhook signature required
pytest arxsec-api/tests/test_billing.py
---
When all items are ticked, the evaluator should be confident the platform behaves as documented. Anything that fails belongs in a written follow-up in poc/issues.md so the sales engineer can address it before the next review.