boostsecurityio · lindycoder · Dec 4, 2025 · Dec 2, 2025 · Dec 2, 2025
diff --git a/README.md b/README.md
@@ -17,3 +17,25 @@
 ### [CodeQL](scanners/boostsecurityio/codeql)
 
 ### [GitLeaks](scanners/boostsecurityio/gitleaks)
+
+## Scanner Testing
+
+The registry includes automated testing for scanners. When a scanner is modified in a pull request, tests are automatically run across multiple CI/CD providers.
+
+### Overview
+
+- **Test Definition**: Each scanner can define tests in a `tests.yaml` file
+- **Automatic Detection**: Modified scanners are detected and tested on PR
+- **Multi-Provider**: Tests run on GitHub Actions, GitLab CI, Azure DevOps, and Bitbucket
+
+### Documentation
+
+- [Setting Up Tests](docs/setup-tests.md) - How to write tests for your scanner
+- [Authentication Strategy](docs/authentication-strategy.md) - OAuth2/OIDC token architecture
+
+#### Test Runner Setup (per CI/CD provider)
+
+- [GitHub Actions](docs/setup-github.md)
+- [GitLab CI](docs/setup-gitlab.md)
+- [Azure DevOps](docs/setup-azure-devops.md)
+- [Bitbucket Pipelines](docs/setup-bitbucket.md)
diff --git a/docs/authentication-strategy.md b/docs/authentication-strategy.md
@@ -0,0 +1,109 @@
+# CI/CD Test Runner Authentication Strategy
+
+## Executive Summary
+
+Migration from long-lived user tokens to short-lived OAuth2/OIDC tokens across all CI/CD platforms for the scanner registry test orchestration system.
+
+---
+
+## Authentication Solution
+
+| Platform         | Auth Method               | Token Lifetime | User-Independent |
+|------------------|---------------------------|----------------|------------------|
+| **GitHub**       | GitHub App                | 1 hour         | ✅                |
+| **GitLab**       | OAuth2 Application        | 2 hours        | ✅                |
+| **Azure DevOps** | OIDC (Federated Identity) | ~1 hour        | ✅                |
+| **Bitbucket**    | OAuth2 Consumer           | 2 hours        | ✅                |
+
+### Architecture Flow
+
+```
+Scanner Registry PR Created
+         ↓
+GitHub Actions Workflow Triggered
+         ↓
+┌────────────────────────────────────┐
+│  Token Generation (in GH Actions)  │
+│  - GitHub: Official Action         │
+│  - GitLab: OAuth2 API call         │
+│  - Azure: OIDC (no secrets)        │
+│  - Bitbucket: OAuth2 API call      │
+└────────────────────────────────────┘
+         ↓
+Short-lived tokens (1-2 hours)
+         ↓
+Passed to test-action CLI
+         ↓
+Trigger test pipelines on each platform
+         ↓
+Tokens expire automatically
+```
+
+### Key Security Improvements
+
+- ✅ **Short-lived tokens** - Auto-expire in 1-2 hours (vs. indefinite)
+- ✅ **No manual rotation** - Tokens generated fresh on each run
+- ✅ **Application-based** - Not tied to user accounts
+- ✅ **Principle of least privilege** - Minimal scopes (trigger pipelines + read status)
+- ✅ **Secrets isolation** - Client credentials stored only in GitHub Actions, never in test-action
+- ✅ **Zero secrets for Azure** - OIDC federated identity eliminates client secret storage entirely
+- ✅ **Industry standard** - OAuth2/OIDC protocols used by all platforms
+
+---
+
+## Rejected Alternatives
+
+**Managed Identities (Azure)** - Only works if test-action runs inside Azure infrastructure; not applicable for GitHub Actions execution.
+
+**Service Principal with Client Secret (Azure)** - Requires storing `AZURE_CLIENT_SECRET` in GitHub Actions; OIDC federated identity is more secure as it eliminates secret storage entirely.
+
+**Repository/Workspace Access Tokens (Bitbucket)** - Cannot trigger or read pipeline status; insufficient permissions for use case.
+
+**OIDC for GitHub API** - OIDC is for external services authenticating to GitHub, not for GitHub Actions triggering other GitHub workflows; GitHub Apps are the correct solution.
+
+---
+
+## Implementation Requirements
+
+### One-Time Setup (per platform)
+
+1. **GitHub**: Register GitHub App with `contents: read`, `actions: write` permissions
+2. **GitLab**: Create OAuth2 Application with `api` scope
+3. **Azure DevOps**: Register Microsoft Entra ID application, add federated credential for GitHub Actions OIDC, grant Build (Read & Execute) to Azure DevOps project
+4. **Bitbucket**: Create OAuth2 Consumer with `pipeline`, `pipeline:write`, `repository` scopes
+
+### Secrets Configuration
+
+Store client credentials in GitHub Actions repository secrets:
+- `GH_APP_ID`, `GH_APP_PRIVATE_KEY`
+- `GITLAB_CLIENT_ID`, `GITLAB_CLIENT_SECRET`
+- `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` (no client secret - uses OIDC; no subscription needed for Azure DevOps API)
+- `BITBUCKET_CLIENT_ID`, `BITBUCKET_CLIENT_SECRET`
+
+### Code Changes
+
+- **GitHub Actions workflow**: Add token generation steps (GitHub App action, OAuth API calls, azure/login with OIDC)
+- **GitHub Actions permissions**: Add `id-token: write` permission for Azure OIDC
+- **test-action CLI**: Accept `--{platform}-token` arguments instead of client credentials
+- **Providers**: Use provided tokens directly instead of generating them
+
+---
+
+## Security Benefits Summary
+
+| Metric          | Before          | After               | Improvement            |
+|-----------------|-----------------|---------------------|------------------------|
+| Token Lifetime  | Indefinite      | 1-2 hours           | **99%+ reduction**     |
+| Manual Rotation | Required        | None                | **Zero maintenance**   |
+| User Dependency | Yes (4 users)   | No (4 apps)         | **Zero bus factor**    |
+| Token Scope     | Over-privileged | Minimal             | **Least privilege**    |
+| Audit Trail     | User actions    | Application actions | **Better attribution** |
+
+---
+
+## Compliance & Standards
+
+- ✅ **OAuth 2.0 / OpenID Connect** - Industry standard protocols (RFC 6749, RFC 7519)
+- ✅ **Zero Trust** - Short-lived credentials, continuous verification
+- ✅ **NIST SP 800-63B** - Authenticator assurance level via cryptographic proof
+- ✅ **SOC 2 / ISO 27001** - Automated credential lifecycle management
diff --git a/docs/setup-azure-devops.md b/docs/setup-azure-devops.md
@@ -0,0 +1,135 @@
+# Azure DevOps Test Runner Setup
+
+## 1. Create Test Runner Repository
+
+1. Create a new repository named `scan-test-runner-azure-devops` in your Azure DevOps project
+2. Add the pipeline configuration at `azure-pipelines.yml`
+3. Create a pipeline from the YAML file
+
+---
+
+## 2. Register Microsoft Entra ID Application
+
+1. Navigate to Azure Portal:
+   **Microsoft Entra ID → App registrations → New registration**
+
+   Direct link: https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/CreateApplicationBlade
+
+2. Configure the application:
+
+   | Field                       | Value                                          |
+   |-----------------------------|------------------------------------------------|
+   | **Name**                    | `BoostSecurity.io Scan Test Runner`            |
+   | **Supported account types** | Accounts in this organizational directory only |
+   | **Redirect URI**            | Leave blank                                    |
+
+3. Click **Register**
+
+4. Note the **Application (client) ID** and **Directory (tenant) ID** from the overview page
+
+---
+
+## 3. Add Federated Credential for GitHub Actions
+
+1. In the app registration, go to:
+   **Certificates & secrets → Federated credentials → Add credential**
+
+2. Select **GitHub Actions deploying Azure resources**
+
+3. Configure the federated credential:
+   Need to be done for both
+   - boostsecurityio/dev-registry
+   - boost-community/scanner-registry
+
+   | Field                  | Value               |
+   |------------------------|---------------------|
+   | **Organization**       | <org name>          |
+   | **Repository**         | <repo name>         |
+   | **Entity type**        | Pull Request        |
+   | **Name**               | `github-actions-pr` |
+
+4. Click **Add**
+
+---
+
+## 4. Grant Permissions in Azure DevOps
+
+1. Navigate to your Azure DevOps organization:
+   **Organization Settings → Users → Add users**
+
+   Or: `https://dev.azure.com/{ORG}/_settings/users`
+
+2. Add the service principal:
+   - Search for the app name: `BoostSecurity.io Scan Test Runner`
+   - Access level: **Basic**
+   - Add to project: Select your project
+
+3. Navigate to project permissions:
+   **Project Settings → Permissions → {Your Project} Team → Members → Add**
+
+4. Add the service principal with **Build Administrator** role (or create a custom role with Build: Read & Execute)
+
+---
+
+## 5. Configure Secrets on Scanner Registry Repository
+
+Navigate to the scanner registry repository (GitHub):
+**Settings → Secrets and variables → Actions → New repository secret**
+
+| Secret Name                       | Value                               |
+|-----------------------------------|-------------------------------------|
+| `BOOST_SCAN_RUNNER_ADO_TENANT_ID` | Directory (tenant) ID from step 2   |
+| `BOOST_SCAN_RUNNER_ADO_CLIENT_ID` | Application (client) ID from step 2 |
+
+
+---
+
+## 6. Usage in GitHub Actions Workflow
+
+```yaml
+permissions:
+  id-token: write  # Required for OIDC
+  contents: read
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Azure Login (OIDC)
+        uses: azure/login@v2
+        with:
+          client-id: ${{ secrets.AZURE_CLIENT_ID }}
+          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
+          allow-no-subscriptions: true
+
+      - name: Get Azure DevOps Token
+        id: azure-token
+        run: |
+          token=$(az account get-access-token \
+            --resource 499b84ac-1321-427f-aa17-267ca6975798 \
+            --query accessToken -o tsv)
+          echo "token=$token" >> $GITHUB_OUTPUT
+          echo "::add-mask::$token"
+
+      - name: Run test-action
+        ...
+```
+
+---
+
+## 7. Token Details
+
+| Property             | Value                                |
+|----------------------|--------------------------------------|
+| **Lifetime**         | ~1 hour                              |
+| **Refresh**          | New token generated per workflow run |
+| **Secrets Required** | None (OIDC federation)               |
+
+---
+
+## References
+
+- [Configure OIDC for GitHub Actions and Azure](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-azure)
+- [Workload Identity Federation](https://learn.microsoft.com/en-us/entra/workload-id/workload-identity-federation)
+- [Azure DevOps REST API Authentication](https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/service-principal-managed-identity)
+- [azure/login GitHub Action](https://github.com/Azure/login)
diff --git a/docs/setup-bitbucket.md b/docs/setup-bitbucket.md
@@ -0,0 +1,97 @@
+# Bitbucket Test Runner Setup
+
+## 1. Create Test Runner Repository
+
+1. Create a new repository named `scan-test-runner-bitbucket-pipelines` in your Bitbucket workspace
+2. Add the pipeline configuration at `bitbucket-pipelines.yml`
+3. Enable Pipelines: **Repository settings → Pipelines → Settings → Enable Pipelines**
+
+---
+
+## 2. Create OAuth2 Consumer
+
+1. Navigate to your workspace settings:
+   **Workspace → Settings → OAuth consumers → Add consumer**
+
+   Direct link: `https://bitbucket.org/{WORKSPACE}/workspace/settings/api`
+
+2. Configure the OAuth consumer:
+
+   | Field                          | Value                                                |
+   |--------------------------------|------------------------------------------------------|
+   | **Name**                       | `BoostSecurity.io Scan Test Runner`                  |
+   | **Callback URL**               | `http://localhost` (not used for client credentials) |
+   | **This is a private consumer** | ✅ Checked                                            |
+
+3. Set **Permissions**:
+
+   | Permission       | Access |
+   |------------------|--------|
+   | **Repositories** | Read   |
+   | **Pipelines**    | Read   |
+   | **Pipelines**    | Write  |
+
+4. Click **Save**
+
+5. Note the **Key** (client ID) and **Secret** - the secret is only shown once!
+
+---
+
+## 3. Configure Secrets on Scanner Registry Repository
+
+Navigate to the scanner registry repository (GitHub):
+**Settings → Secrets and variables → Actions → New repository secret**
+
+| Secret Name                                 | Value              |
+|---------------------------------------------|--------------------|
+| `BOOST_SCAN_RUNNER_BITBUCKET_CLIENT_ID`     | Key from step 2    |
+| `BOOST_SCAN_RUNNER_BITBUCKET_CLIENT_SECRET` | Secret from step 2 |
+
+
+---
+
+## 4. Usage in GitHub Actions Workflow
+
+```yaml
+- name: Generate Bitbucket OAuth Token
+  id: bitbucket-token
+  run: |
+    response=$(curl -s -X POST \
+      "https://bitbucket.org/site/oauth2/access_token" \
+      -u "${{ secrets.BITBUCKET_CLIENT_ID }}:${{ secrets.BITBUCKET_CLIENT_SECRET }}" \
+      -d "grant_type=client_credentials")
+
+    token=$(echo "$response" | jq -r '.access_token')
+    echo "token=$token" >> $GITHUB_OUTPUT
+    echo "::add-mask::$token"
+
+- name: Run test-action
+  ...
+```
+
+---
+
+## 5. Token Details
+
+| Property     | Value                                      |
+|--------------|--------------------------------------------|
+| **Lifetime** | 2 hours                                    |
+| **Refresh**  | New token generated per workflow run       |
+| **Scopes**   | `repository`, `pipeline`, `pipeline:write` |
+
+---
+
+## 6. API Endpoints
+
+| Operation               | Endpoint                                                                             |
+|-------------------------|--------------------------------------------------------------------------------------|
+| **Trigger Pipeline**    | `POST https://api.bitbucket.org/2.0/repositories/{workspace}/{repo}/pipelines/`      |
+| **Get Pipeline Status** | `GET https://api.bitbucket.org/2.0/repositories/{workspace}/{repo}/pipelines/{uuid}` |
+
+---
+
+## References
+
+- [Bitbucket OAuth2](https://support.atlassian.com/bitbucket-cloud/docs/use-oauth-on-bitbucket-cloud/)
+- [OAuth2 Client Credentials Grant](https://support.atlassian.com/bitbucket-cloud/docs/use-oauth-on-bitbucket-cloud/#Client-credentials-Grant--4-LO-)
+- [Bitbucket Pipelines API](https://developer.atlassian.com/cloud/bitbucket/rest/api-group-pipelines/)