Source Control Plane Migration Analysis

This note explains how to adopt the new source control plane in /Users/mr.frierson/Project-Agent/arxsec-api without disrupting older sync paths.

Goal

Move new Atlas-facing source onboarding and recurring sync operations onto the shared source control plane:

• GET /v1/source-registry/catalog
• POST /v1/source-registry
• POST /v1/source-registry/{source_id}/sync-now
• GET /v1/source-registry/{source_id}/jobs
• GET /v1/source-registry/{source_id}/health
• POST /v1/source-registry/jobs/{job_id}/retry

Keep legacy one-off or domain-specific sync surfaces only where they still represent distinct business workflows.

Canonical New Surface

The new source platform is implemented in:

• source_control_plane.py
• source_control_plane.py
• scheduler.py
• base.py
• engine.py

It already provides:

• source registry metadata
• adapter catalog and checkpoint strategy visibility
• health and freshness state
• queued and scheduled sync jobs
• job history
• retry controls and failure categorization
• checkpoint persistence
• Atlas memory ingest forwarding

This should be the default onboarding path for new evidence sources.

Legacy Surface Inventory

Quick classification

| Surface | Primary purpose | Recommendation | Notes | | --- | --- | --- | --- | | /v1/source-registry/* | evidence-source onboarding, scheduling, health, checkpoints, retries | adopt as default | canonical source ingest and operator plane | | /v1/workforce/hris/* | workforce cache refresh and manager rebinding | keep separate for now | side-effecting workforce workflow, not just ingest | | /v1/matrix-sync/* | reconciliation, dry-run conflict detection, writeback | keep separate | not an Atlas evidence-ingest surface | | /v1/governance/connectors/sync-status | governance visibility | read-only collateral | not the operator system of record |

1. Workforce HRIS sync

Primary files:

• hris.py
• hris_sync.py

Endpoints:

• GET /v1/workforce/hris/status
• GET /v1/workforce/hris/cache
• POST /v1/workforce/hris/sync

What it does:

• reads tenant HRIS config from customer_hris_config
• refreshes hris_employee_cache
• rebinds manager relationships for workforce agents and cohort attachments

Recommendation:

• stay separate for now as an HR/workforce domain workflow
• migrate source onboarding and observability to the source control plane over time

Reason:

The HRIS endpoint is not just evidence ingest. It also performs workforce-specific side effects such as manager rebinding. That makes it a poor short-term candidate for direct replacement by generic source ingestion.

Migration direction:

• add the HRIS connector to source_registry for evidence ingest and freshness/health visibility
• continue using POST /v1/workforce/hris/sync for workforce cache + rebinding until those side effects are explicitly ported
• treat HRIS as dual-surface during rollout:
• source_registry for Atlas-facing source monitoring and evidence ingest
• /v1/workforce/hris/sync for workforce cache correctness and manager rebinding

2. Matrix sync

Primary files:

• matrix_sync.py
• matrix_sync.py

Endpoints:

• GET /v1/matrix-sync/config
• PUT /v1/matrix-sync/config/{provider}
• POST /v1/matrix-sync/dry-run
• POST /v1/matrix-sync/run
• GET /v1/matrix-sync/conflicts

What it does:

• bidirectional escalation-matrix sync
• conflict handling
• provider-specific writeback behavior

Recommendation:

• stay separate

Reason:

Matrix sync is not an Atlas evidence-ingest workflow. It is a synchronization and reconciliation plane with its own dry-run, conflict, and writeback semantics. It should not be forced into the source control plane unless the source platform later expands into a broader reconciliation/control plane.

Migration note:

• do not rewrite matrix sync onto the source control plane
• if matrix-linked evidence is needed in Atlas, add a separate evidence-source entry and keep matrix writeback separate

3. Governance connector sync-status

Primary file:

• governance.py

Endpoint:

• GET /v1/governance/connectors/sync-status

Recommendation:

• keep as read-only collateral
• do not treat as the primary operator surface

Reason:

This is a governance/status view, not the source ingestion system of record.

Migration Decisions

Should migrate onto source control plane

• new Atlas evidence sources
• document systems
• collaboration systems
• project/workflow systems
• KPI/event evidence feeds
• connector freshness and health monitoring
• queued/scheduled source job execution
• checkpointed incremental ingestion

Should stay separate for now

• HRIS workforce cache refresh and manager rebinding
• matrix sync and escalation conflict resolution
• purely governance-facing status summaries

Should be deprecated as the primary onboarding path

• manual per-domain evidence sync logic for sources already covered by source_registry
• ad hoc operator guidance that points people to legacy sync endpoints for Atlas evidence ingest

Legacy entrypoint map

Use this table during implementation reviews when deciding where new source work should land.

| Existing entrypoint | Keep / migrate / deprecate | Why | | --- | --- | --- | | POST /v1/source-registry/{source_id}/sync-now | keep | canonical forced-run surface | | POST /v1/source-registry/jobs/{job_id}/retry | keep | canonical retry surface | | POST /v1/workforce/hris/sync | keep for now | manager rebinding and workforce cache side effects | | GET /v1/workforce/hris/status | keep for now | workforce-specific status view | | POST /v1/matrix-sync/run | keep separate | reconciliation and provider writeback | | POST /v1/matrix-sync/dry-run | keep separate | preflight conflict analysis, not ingest | | GET /v1/governance/connectors/sync-status | keep as secondary | governance rollup, not operator control plane |

Rollout Sequence

1. Use source control plane for all new evidence-source onboarding.
2. Mirror existing HRIS providers into source_registry for visibility and evidence ingest, but keep workforce rebinding on the legacy endpoint.
3. Keep matrix sync separate.
4. Shift operator documentation and runbooks to the source control plane first.
5. Train operators to use source job history, checkpoints, and retry controls before de-emphasizing older evidence-sync habits.
6. Only after operational stability is proven, decide whether HRIS side effects should be folded into the new plane.

Adoption checklist

Before declaring a source family migrated, confirm all of the following:

• the connector exists in SOURCE_SYNC_ADAPTERS
• the source can be created and scheduled in /v1/source-registry
• the operator can see health, freshness, checkpoint state, and recent jobs
• forced sync and retry behavior are documented in the runbook
• stale/failing alert expectations are understood by the owning team
• any remaining legacy endpoint is explicitly documented as domain-specific, not the default ingest path

Deprecation notes by family

HRIS

• Do not deprecate /v1/workforce/hris/sync yet.
• De-emphasize it for new Atlas evidence onboarding.
• If an operator asks “how do I add Workday as a source for Atlas,” send them to the source control plane first.

Matrix sync

• Do not deprecate matrix sync.
• Do not route new matrix conflict or writeback work through the source control plane.
• Keep its ownership and runbooks separate from source ingest operations.

New evidence connectors

• Any new document, collaboration, project, or KPI/event feed should start in source_registry.
• Avoid building one-off polling endpoints unless they provide side effects the source control plane does not own.

Deprecation Guidance

Use these rules during rollout:

• if the workflow exists to ingest source records for Atlas, prefer source_registry
• if the workflow exists to mutate workforce bindings or reconcile escalation policy, keep the legacy surface
• do not add new one-off source polling endpoints when source_registry can handle the connector

Team Recommendation

For the next phase, treat the source control plane as:

• the system of record for evidence-source inventory
• the operator surface for sync health, checkpoints, and retries
• the adoption target for new connectors

Treat legacy HRIS and matrix flows as:

• domain-specific compatibility surfaces
• not the long-term default for new source onboarding

Recommended rollout ownership

• Platform/backend: source control plane adapters, scheduler, and observability
• Workforce/domain owners: HRIS cache correctness and manager rebinding
• Governance/integration owners: matrix sync reconciliation and conflict workflows
• Operator/solutions teams: source onboarding, forced runs, job history inspection, and retry handling through the source control plane

Related docs

• Source Control Plane Runbook
• source_control_plane.py
• hris.py
• matrix_sync.py