Add comprehensive secrets and credentials management guide

[dimitri-yatsenko]

Summary

Adds comprehensive guide for secure configuration management, addressing a critical gap identified in the cohesion review.

Problem

From cohesion review (COHESION-REVIEW.md Issue #3, High Priority):

• No authoritative guidance on .secrets/ directory structure and usage
• Configuration priority order not clearly documented
• Scattered information across configure-database.md and configure-storage.md
• Users unclear how to handle credentials across different environments

Solution

New guide: how-to/manage-secrets.md (500+ lines)

Content Organization

Overview:

• Clear separation: non-sensitive (datajoint.json) vs sensitive (.secrets/) vs dynamic (env vars)
• Configuration priority table (programmatic > env > secrets > config > defaults)

Database Credentials:

• Option 1: Secrets directory (recommended for development)
• Option 2: Environment variables (recommended for production)
• Option 3: Programmatic configuration
• Complete examples for each

Object Storage Credentials:

• File storage (no credentials needed)
• S3/MinIO with secrets directory structure
• Environment variable alternatives
• Multiple store configuration examples

Environment Variable Reference:

• Complete tables for database settings (DJ_HOST, DJ_USER, DJ_PASS, etc.)
• Store credentials pattern (DJ_STORES__ACCESS_KEY, etc.)

Security Best Practices:

• Development environment setup (mkdir, chmod, .gitignore)
• Production environment (vault integration examples)
• CI/CD (GitHub Actions, GitLab CI examples)
• Docker containers (env vars and volume mounts)
• Cloud platforms (AWS Lambda, Google Cloud Functions, Azure Functions)

Common Patterns:

• Local development with secrets directory
• Production with environment variables
• Multi-environment setup (.secrets/datajoint.dev.json, etc.)

Troubleshooting:

• Secrets not loading (check priority)
• Permission errors (chmod fixes)
• Environment variables not taking effect
• Accidentally committed secrets (remediation steps)

Configuration Templates:

• Minimal development setup
• Production with S3 storage

Structure

how-to/
├── installation.md
├── manage-secrets.md          ← NEW (comprehensive guide)
├── configure-database.md       (complementary: connection specifics)
├── configure-storage.md        (complementary: storage specifics)
└── ...

Navigation Update

Added to how-to/index.md under Setup section:

## Setup
- [Installation](installation.md)
- [Manage Secrets and Credentials](manage-secrets.md) ← NEW
- [Configure Database Connection](configure-database.md)
- [Configure Object Storage](configure-storage.md)
- [Use the Command-Line Interface](use-cli.md)

User Impact

Before (Confusion)

• "Where do I put my database password?"
• "What's the difference between datajoint.json and .secrets/?"
• "How do I configure credentials for Docker/CI/production?"
• Information scattered across multiple docs

After (Clarity)

• Single authoritative guide for all credentials management
• Clear environment-specific patterns (dev, prod, CI/CD, Docker, cloud)
• Complete examples for every scenario
• Security best practices in one place

Cross-References

The new guide complements existing docs:

• References configure-database.md for database-specific settings
• References configure-storage.md for storage-specific settings
• Referenced by installation.md for initial setup
• Linked from Configuration Reference

Related

• Addresses COHESION-REVIEW.md Issue Update styling #3 (High Priority)
• Part of comprehensive documentation cohesion improvements
• Complements PR Fix terminology: Replace 'inline' with 'in-table' storage #109 (terminology), PR Clarify DataJoint 2.0 pre-release status in installation guide #111 (installation clarity)

[dimitri-yatsenko]

Consolidated into #119 - Documentation Cohesion Review: Comprehensive Improvements for DataJoint 2.0