Skip to content

Feat/encrypted connector secrets#5

Open
Grandvizir wants to merge 11 commits intomainfrom
feat/encrypted-connector-secrets
Open

Feat/encrypted connector secrets#5
Grandvizir wants to merge 11 commits intomainfrom
feat/encrypted-connector-secrets

Conversation

@Grandvizir
Copy link
Collaborator

Summary

This PR adds per-tenant encrypted storage for connector secrets (API keys, client secrets, tokens), so users can enter
credentials directly in the UI instead of relying on server-side environment variables.

Each tenant's secrets are encrypted with a unique Fernet key derived via HKDF-SHA256 from a master key + tenant UUID,
ensuring cryptographic isolation between organizations. Existing connectors using credential_ref (env var names)
continue to work unchanged.

Changes

Encryption layer

  • New connectors/crypto.py module with encrypt_secret() / decrypt_secret() using HKDF key derivation (info prefix
    docuscore-connector-secret-v1: for future key rotation)
  • FIELD_ENCRYPTION_KEY setting (falls back to SECRET_KEY when not set)
  • Added cryptography>=42.0 dependency

Model

  • New encrypted_secret TextField on ConnectorConfig with set_secret() / get_secret() methods
  • get_secret() resolution order: encrypted DB value > credential_ref env var lookup > empty string
  • Schema migration 0003_add_encrypted_secret

Connector updates

  • sharepoint.py and confluence.py now receive the decrypted secret directly instead of calling os.environ.get()
  • ingestion/pipeline.py calls get_secret() at init time

UI

  • Password input field added to connector creation form (standalone page + modal)
  • Security banner on the connector list page with expandable technical details (Fernet, HKDF, tenant isolation, env
    var compatibility)
  • Compact inline security banner above the secret field in both create forms

Migration path for existing deployments

  • New manage.py migrate_connector_secrets management command:
    • Dry-run by default (previews what would be migrated)
    • --apply reads env var values and encrypts them into the DB
    • --clear-ref optionally removes credential_ref after migration
    • Skips connectors with missing env vars or already-encrypted secrets
    • Safe to run multiple times (idempotent)

Documentation

  • Updated: CHANGELOG, README, SECURITY, deployment guide, project structure, technical reference, ingestion docs
  • Added upgrade instructions for existing production deployments

Test plan

  • pytest tests/test_connector_crypto.py — 11 tests (roundtrip, empty string, cross-tenant ciphertext, wrong-tenant
    rejection, SECRET_KEY fallback, corrupted ciphertext, model set/get, env var fallback, encrypted preferred over env,
    cross-tenant isolation, no-secret empty return)
  • pytest tests/test_migrate_connector_secrets.py — 6 tests (dry-run empty, dry-run shows candidates, apply encrypts,
    apply+clear-ref, skips unset env vars, skips already-encrypted)
  • pytest tests/test_connectors_views.py — 14 existing tests still pass
  • Manual: create a connector via UI with a secret value, verify it's encrypted in DB, trigger a sync and confirm the
    connector authenticates correctly

Related Issues

Closes #

…tility

Add HKDF-based Fernet encryption that derives a unique key per tenant
from a master key, enabling cryptographic isolation of connector secrets.
Falls back to SECRET_KEY when not set, but a dedicated key is
recommended for production deployments.
The model now supports per-tenant encrypted secrets with graceful
fallback to the existing credential_ref env var lookup.
…okup

Connectors now receive the already-decrypted credential value from
get_secret(), removing the os.environ.get() indirection at init time.
- Pipeline passes get_secret() to connectors instead of raw credential_ref
- Create view encrypts secret_value from POST into the model
- Templates add a password field for direct secret entry with env var fallback
Tests cover encrypt/decrypt roundtrip, empty strings, cross-tenant
isolation, wrong-tenant rejection, env var fallback, and model methods.
Update CHANGELOG, README, SECURITY, deployment guide, project structure,
technical reference, and ingestion docs to cover the new encrypted secret
feature, FIELD_ENCRYPTION_KEY setting, and credential resolution order.
Provides a safe migration path for existing deployments: reads env var
values referenced by credential_ref and encrypts them into the new
encrypted_secret field. Supports dry-run, --apply, and --clear-ref modes.
- List page: full-width banner with expandable technical details
  (Fernet, HKDF-SHA256, tenant isolation, env var compatibility)
- Create page + modal: compact inline banner above the secret field
- Uses emerald green accent to convey trust/security, with subtle
  gradient background, shield-check icon, and smooth expand animation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant