​

Data Flows (AI Agent Quick Trace)

This file is for fast debugging and change planning. Each flow includes the key files and the minimum execution path.

​

1) Startup + Initialization

Entry:

• backend/copilot.py -> @app.on_event("startup")

Flow:

1. FastAPI app starts (backend/copilot.py).
2. DB bootstrap/migration path runs (backend/app/db/db_setup.py):
   • create_database_if_not_exists (prod)
   • create_copilot_user_if_not_exists (prod)
   • apply_migrations
3. Object storage buckets are ensured (backend/app/data_store/data_store_setup.py:create_buckets).
4. Seed/reference data runs:
   • connectors (add_connectors -> backend/app/db/db_populate.py)
   • roles
   • available integrations/network connectors
5. Admin + scheduler users ensured.
6. APScheduler initialized and started (backend/app/schedulers/scheduler.py).

​

2) Auth Request Flow

Primary token endpoint:

• POST /api/auth/token in backend/app/auth/routes/auth.py

Flow:

1. Frontend sign-in form submits credentials (frontend/src/components/auth/SignIn.vue).
2. API wrapper sends form-data to /auth/token (frontend/src/api/endpoints/auth.ts).
3. Backend authenticates user (AuthHandler.authenticate_user in backend/app/auth/utils.py).
4. JWT is created with role scope(s) (encode_token in backend/app/auth/utils.py).
5. Frontend stores token in auth store (frontend/src/stores/auth.ts).
6. Axios interceptor adds Authorization: Bearer <token> on later calls (frontend/src/api/httpClient.ts).
7. Protected backend routes validate token/scope via AuthHandler.get_current_user or require_any_scope.

​

3) Scheduler Job Execution

Core scheduler files:

• backend/app/schedulers/scheduler.py
• backend/app/schedulers/routes/scheduler.py

Flow:

1. Startup calls init_scheduler.
2. initialize_job_metadata ensures known jobs exist in DB (JobMetadata).
3. schedule_enabled_jobs loads enabled jobs and registers interval triggers.
4. At run-time APScheduler calls mapped functions (get_function_by_name).
5. Example job invoke_alert_creation_collect:
   • runs alert auto-create route logic (backend/app/schedulers/services/invoke_alert_creation.py)
   • updates JobMetadata.last_success.
6. Manual operations (/api/scheduler/...) can run/pause/update/delete jobs.

​

4) Connector Verify + Use

Verify dispatch path:

• POST /api/connectors/verify/{id} -> backend/app/connectors/routes.py
• dispatch map in backend/app/connectors/services.py:get_connector_service

Flow:

1. Frontend calls verify (frontend/src/api/endpoints/connectors.ts).
2. Backend fetches connector row by ID, builds response model.
3. Connector name is mapped to a service class in service_map.
4. Service class calls connector-specific verifier in backend/app/connectors/<service>/utils/universal.py.
5. DB updates connector_verified + connector_last_updated.

Use path (runtime connector client):

1. Feature route/service calls a connector client factory in utils/universal.py.
2. Factory pulls credentials via get_connector_info_from_db (backend/app/connectors/utils.py).
3. Downstream API requests run with those connector settings.

​

5) Alert -> Case

Alert creation and case linking paths:

• Auto/manual alert creation routes: backend/app/incidents/routes/incident_alert.py
• Case creation routes: backend/app/incidents/routes/db_operations.py
• Case creation service: backend/app/incidents/services/db_operations.py

Flow:

1. Alert is ingested/created (/incident_alert/create/manual or /incident_alert/create/auto).
2. Analyst (or workflow) calls /incident_management/case/from-alert.
3. Backend creates Case using alert fields (create_case_from_alert).
4. Backend creates join record in CaseAlertLink (create_case_alert_link).
5. Case now references the originating alert for SOC workflows and reporting.

​

6) Artifact Upload to MinIO

Two common paths:

• Generic upload: /api/agent_data_store/upload (backend/app/data_store/data_store_routes.py)
• Velociraptor collection upload: backend/app/connectors/velociraptor/services/artifacts.py

Velociraptor-specific flow:

1. Collection job runs and gets flow_id.
2. fetch_file_from_filestore downloads zipped results locally.
3. upload_agent_artifact_file uploads file to MinIO bucket velociraptor-artifacts with key agent_id/flow_id/file.zip (backend/app/data_store/data_store_operations.py).
4. Metadata is stored in AgentDataStore table.
5. UI/API can list/download/delete via backend/app/data_store/data_store_routes.py.