diff --git a/README.md b/README.md index c6679e59..4e2bb875 100644 --- a/README.md +++ b/README.md @@ -17,3 +17,25 @@ ### [CodeQL](scanners/boostsecurityio/codeql) ### [GitLeaks](scanners/boostsecurityio/gitleaks) + +## Scanner Testing + +The registry includes automated testing for scanners. When a scanner is modified in a pull request, tests are automatically run across multiple CI/CD providers. + +### Overview + +- **Test Definition**: Each scanner can define tests in a `tests.yaml` file +- **Automatic Detection**: Modified scanners are detected and tested on PR +- **Multi-Provider**: Tests run on GitHub Actions, GitLab CI, Azure DevOps, and Bitbucket + +### Documentation + +- [Setting Up Tests](docs/setup-tests.md) - How to write tests for your scanner +- [Authentication Strategy](docs/authentication-strategy.md) - OAuth2/OIDC token architecture + +#### Test Runner Setup (per CI/CD provider) + +- [GitHub Actions](docs/setup-github.md) +- [GitLab CI](docs/setup-gitlab.md) +- [Azure DevOps](docs/setup-azure-devops.md) +- [Bitbucket Pipelines](docs/setup-bitbucket.md) diff --git a/docs/authentication-strategy.md b/docs/authentication-strategy.md new file mode 100644 index 00000000..392003a9 --- /dev/null +++ b/docs/authentication-strategy.md @@ -0,0 +1,109 @@ +# CI/CD Test Runner Authentication Strategy + +## Executive Summary + +Migration from long-lived user tokens to short-lived OAuth2/OIDC tokens across all CI/CD platforms for the scanner registry test orchestration system. + +--- + +## Authentication Solution + +| Platform | Auth Method | Token Lifetime | User-Independent | +|------------------|---------------------------|----------------|------------------| +| **GitHub** | GitHub App | 1 hour | ✅ | +| **GitLab** | OAuth2 Application | 2 hours | ✅ | +| **Azure DevOps** | OIDC (Federated Identity) | ~1 hour | ✅ | +| **Bitbucket** | OAuth2 Consumer | 2 hours | ✅ | + +### Architecture Flow + +``` +Scanner Registry PR Created + ↓ +GitHub Actions Workflow Triggered + ↓ +┌────────────────────────────────────┐ +│ Token Generation (in GH Actions) │ +│ - GitHub: Official Action │ +│ - GitLab: OAuth2 API call │ +│ - Azure: OIDC (no secrets) │ +│ - Bitbucket: OAuth2 API call │ +└────────────────────────────────────┘ + ↓ +Short-lived tokens (1-2 hours) + ↓ +Passed to test-action CLI + ↓ +Trigger test pipelines on each platform + ↓ +Tokens expire automatically +``` + +### Key Security Improvements + +- ✅ **Short-lived tokens** - Auto-expire in 1-2 hours (vs. indefinite) +- ✅ **No manual rotation** - Tokens generated fresh on each run +- ✅ **Application-based** - Not tied to user accounts +- ✅ **Principle of least privilege** - Minimal scopes (trigger pipelines + read status) +- ✅ **Secrets isolation** - Client credentials stored only in GitHub Actions, never in test-action +- ✅ **Zero secrets for Azure** - OIDC federated identity eliminates client secret storage entirely +- ✅ **Industry standard** - OAuth2/OIDC protocols used by all platforms + +--- + +## Rejected Alternatives + +**Managed Identities (Azure)** - Only works if test-action runs inside Azure infrastructure; not applicable for GitHub Actions execution. + +**Service Principal with Client Secret (Azure)** - Requires storing `AZURE_CLIENT_SECRET` in GitHub Actions; OIDC federated identity is more secure as it eliminates secret storage entirely. + +**Repository/Workspace Access Tokens (Bitbucket)** - Cannot trigger or read pipeline status; insufficient permissions for use case. + +**OIDC for GitHub API** - OIDC is for external services authenticating to GitHub, not for GitHub Actions triggering other GitHub workflows; GitHub Apps are the correct solution. + +--- + +## Implementation Requirements + +### One-Time Setup (per platform) + +1. **GitHub**: Register GitHub App with `contents: read`, `actions: write` permissions +2. **GitLab**: Create OAuth2 Application with `api` scope +3. **Azure DevOps**: Register Microsoft Entra ID application, add federated credential for GitHub Actions OIDC, grant Build (Read & Execute) to Azure DevOps project +4. **Bitbucket**: Create OAuth2 Consumer with `pipeline`, `pipeline:write`, `repository` scopes + +### Secrets Configuration + +Store client credentials in GitHub Actions repository secrets: +- `GH_APP_ID`, `GH_APP_PRIVATE_KEY` +- `GITLAB_CLIENT_ID`, `GITLAB_CLIENT_SECRET` +- `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` (no client secret - uses OIDC; no subscription needed for Azure DevOps API) +- `BITBUCKET_CLIENT_ID`, `BITBUCKET_CLIENT_SECRET` + +### Code Changes + +- **GitHub Actions workflow**: Add token generation steps (GitHub App action, OAuth API calls, azure/login with OIDC) +- **GitHub Actions permissions**: Add `id-token: write` permission for Azure OIDC +- **test-action CLI**: Accept `--{platform}-token` arguments instead of client credentials +- **Providers**: Use provided tokens directly instead of generating them + +--- + +## Security Benefits Summary + +| Metric | Before | After | Improvement | +|-----------------|-----------------|---------------------|------------------------| +| Token Lifetime | Indefinite | 1-2 hours | **99%+ reduction** | +| Manual Rotation | Required | None | **Zero maintenance** | +| User Dependency | Yes (4 users) | No (4 apps) | **Zero bus factor** | +| Token Scope | Over-privileged | Minimal | **Least privilege** | +| Audit Trail | User actions | Application actions | **Better attribution** | + +--- + +## Compliance & Standards + +- ✅ **OAuth 2.0 / OpenID Connect** - Industry standard protocols (RFC 6749, RFC 7519) +- ✅ **Zero Trust** - Short-lived credentials, continuous verification +- ✅ **NIST SP 800-63B** - Authenticator assurance level via cryptographic proof +- ✅ **SOC 2 / ISO 27001** - Automated credential lifecycle management diff --git a/docs/setup-azure-devops.md b/docs/setup-azure-devops.md new file mode 100644 index 00000000..9815cab5 --- /dev/null +++ b/docs/setup-azure-devops.md @@ -0,0 +1,135 @@ +# Azure DevOps Test Runner Setup + +## 1. Create Test Runner Repository + +1. Create a new repository named `scan-test-runner-azure-devops` in your Azure DevOps project +2. Add the pipeline configuration at `azure-pipelines.yml` +3. Create a pipeline from the YAML file + +--- + +## 2. Register Microsoft Entra ID Application + +1. Navigate to Azure Portal: + **Microsoft Entra ID → App registrations → New registration** + + Direct link: https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/CreateApplicationBlade + +2. Configure the application: + + | Field | Value | + |-----------------------------|------------------------------------------------| + | **Name** | `BoostSecurity.io Scan Test Runner` | + | **Supported account types** | Accounts in this organizational directory only | + | **Redirect URI** | Leave blank | + +3. Click **Register** + +4. Note the **Application (client) ID** and **Directory (tenant) ID** from the overview page + +--- + +## 3. Add Federated Credential for GitHub Actions + +1. In the app registration, go to: + **Certificates & secrets → Federated credentials → Add credential** + +2. Select **GitHub Actions deploying Azure resources** + +3. Configure the federated credential: + Need to be done for both + - boostsecurityio/dev-registry + - boost-community/scanner-registry + + | Field | Value | + |------------------------|---------------------| + | **Organization** | | + | **Repository** | | + | **Entity type** | Pull Request | + | **Name** | `github-actions-pr` | + +4. Click **Add** + +--- + +## 4. Grant Permissions in Azure DevOps + +1. Navigate to your Azure DevOps organization: + **Organization Settings → Users → Add users** + + Or: `https://dev.azure.com/{ORG}/_settings/users` + +2. Add the service principal: + - Search for the app name: `BoostSecurity.io Scan Test Runner` + - Access level: **Basic** + - Add to project: Select your project + +3. Navigate to project permissions: + **Project Settings → Permissions → {Your Project} Team → Members → Add** + +4. Add the service principal with **Build Administrator** role (or create a custom role with Build: Read & Execute) + +--- + +## 5. Configure Secrets on Scanner Registry Repository + +Navigate to the scanner registry repository (GitHub): +**Settings → Secrets and variables → Actions → New repository secret** + +| Secret Name | Value | +|-----------------------------------|-------------------------------------| +| `BOOST_SCAN_RUNNER_ADO_TENANT_ID` | Directory (tenant) ID from step 2 | +| `BOOST_SCAN_RUNNER_ADO_CLIENT_ID` | Application (client) ID from step 2 | + + +--- + +## 6. Usage in GitHub Actions Workflow + +```yaml +permissions: + id-token: write # Required for OIDC + contents: read + +jobs: + test: + runs-on: ubuntu-latest + steps: + - name: Azure Login (OIDC) + uses: azure/login@v2 + with: + client-id: ${{ secrets.AZURE_CLIENT_ID }} + tenant-id: ${{ secrets.AZURE_TENANT_ID }} + allow-no-subscriptions: true + + - name: Get Azure DevOps Token + id: azure-token + run: | + token=$(az account get-access-token \ + --resource 499b84ac-1321-427f-aa17-267ca6975798 \ + --query accessToken -o tsv) + echo "token=$token" >> $GITHUB_OUTPUT + echo "::add-mask::$token" + + - name: Run test-action + ... +``` + +--- + +## 7. Token Details + +| Property | Value | +|----------------------|--------------------------------------| +| **Lifetime** | ~1 hour | +| **Refresh** | New token generated per workflow run | +| **Secrets Required** | None (OIDC federation) | + +--- + +## References + +- [Configure OIDC for GitHub Actions and Azure](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-azure) +- [Workload Identity Federation](https://learn.microsoft.com/en-us/entra/workload-id/workload-identity-federation) +- [Azure DevOps REST API Authentication](https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/service-principal-managed-identity) +- [azure/login GitHub Action](https://github.com/Azure/login) diff --git a/docs/setup-bitbucket.md b/docs/setup-bitbucket.md new file mode 100644 index 00000000..13c2c7f2 --- /dev/null +++ b/docs/setup-bitbucket.md @@ -0,0 +1,97 @@ +# Bitbucket Test Runner Setup + +## 1. Create Test Runner Repository + +1. Create a new repository named `scan-test-runner-bitbucket-pipelines` in your Bitbucket workspace +2. Add the pipeline configuration at `bitbucket-pipelines.yml` +3. Enable Pipelines: **Repository settings → Pipelines → Settings → Enable Pipelines** + +--- + +## 2. Create OAuth2 Consumer + +1. Navigate to your workspace settings: + **Workspace → Settings → OAuth consumers → Add consumer** + + Direct link: `https://bitbucket.org/{WORKSPACE}/workspace/settings/api` + +2. Configure the OAuth consumer: + + | Field | Value | + |--------------------------------|------------------------------------------------------| + | **Name** | `BoostSecurity.io Scan Test Runner` | + | **Callback URL** | `http://localhost` (not used for client credentials) | + | **This is a private consumer** | ✅ Checked | + +3. Set **Permissions**: + + | Permission | Access | + |------------------|--------| + | **Repositories** | Read | + | **Pipelines** | Read | + | **Pipelines** | Write | + +4. Click **Save** + +5. Note the **Key** (client ID) and **Secret** - the secret is only shown once! + +--- + +## 3. Configure Secrets on Scanner Registry Repository + +Navigate to the scanner registry repository (GitHub): +**Settings → Secrets and variables → Actions → New repository secret** + +| Secret Name | Value | +|---------------------------------------------|--------------------| +| `BOOST_SCAN_RUNNER_BITBUCKET_CLIENT_ID` | Key from step 2 | +| `BOOST_SCAN_RUNNER_BITBUCKET_CLIENT_SECRET` | Secret from step 2 | + + +--- + +## 4. Usage in GitHub Actions Workflow + +```yaml +- name: Generate Bitbucket OAuth Token + id: bitbucket-token + run: | + response=$(curl -s -X POST \ + "https://bitbucket.org/site/oauth2/access_token" \ + -u "${{ secrets.BITBUCKET_CLIENT_ID }}:${{ secrets.BITBUCKET_CLIENT_SECRET }}" \ + -d "grant_type=client_credentials") + + token=$(echo "$response" | jq -r '.access_token') + echo "token=$token" >> $GITHUB_OUTPUT + echo "::add-mask::$token" + +- name: Run test-action + ... +``` + +--- + +## 5. Token Details + +| Property | Value | +|--------------|--------------------------------------------| +| **Lifetime** | 2 hours | +| **Refresh** | New token generated per workflow run | +| **Scopes** | `repository`, `pipeline`, `pipeline:write` | + +--- + +## 6. API Endpoints + +| Operation | Endpoint | +|-------------------------|--------------------------------------------------------------------------------------| +| **Trigger Pipeline** | `POST https://api.bitbucket.org/2.0/repositories/{workspace}/{repo}/pipelines/` | +| **Get Pipeline Status** | `GET https://api.bitbucket.org/2.0/repositories/{workspace}/{repo}/pipelines/{uuid}` | + +--- + +## References + +- [Bitbucket OAuth2](https://support.atlassian.com/bitbucket-cloud/docs/use-oauth-on-bitbucket-cloud/) +- [OAuth2 Client Credentials Grant](https://support.atlassian.com/bitbucket-cloud/docs/use-oauth-on-bitbucket-cloud/#Client-credentials-Grant--4-LO-) +- [Bitbucket Pipelines API](https://developer.atlassian.com/cloud/bitbucket/rest/api-group-pipelines/) diff --git a/docs/setup-github.md b/docs/setup-github.md new file mode 100644 index 00000000..2d047219 --- /dev/null +++ b/docs/setup-github.md @@ -0,0 +1,94 @@ +# GitHub Test Runner Setup + +## 1. Create Test Runner Repository + +1. Create a new repository named `scan-test-runner-gitbub-actions` in your organization +2. Add the workflow file at `.github/workflows/test-scanner.yml` +3. Ensure GitHub Actions is enabled for the repository + +--- + +## 2. Create GitHub App + +1. Navigate to your organization settings: + **Organization → Settings → Developer settings → GitHub Apps → New GitHub App** + + Or use this link: `https://github.com/organizations/{ORG}/settings/apps/new` + +2. Configure the GitHub App: + + | Field | Value | + |---------------------|-------------------------------------| + | **GitHub App name** | `BoostSecurity.io Scan Test Runner` | + | **Homepage URL** | https://boostsecurity.io/ | + | **Webhook** | Uncheck "Active" (not needed) | + +3. Set **Repository permissions**: + + | Permission | Access | + |--------------|----------------| + | **Actions** | Read and write | + | **Contents** | Read-only | + +4. Set **Where can this GitHub App be installed?**: + - Select "Only on this account" + +5. Click **Create GitHub App** + +6. Note the **App ID** (displayed at the top of the app settings page) + +--- + +## 3. Generate Private Key + +1. On the GitHub App settings page, scroll to **Private keys** +2. Click **Generate a private key** +3. A `.pem` file will be downloaded - keep this secure + +--- + +## 4. Install the App + +1. On the GitHub App settings page, click **Install App** in the left sidebar +2. Select your organization +3. Choose **Only select repositories** +4. Select `test-runner-github` +5. Click **Install** + +--- + +## 5. Configure Secrets on Scanner Registry Repository + +Navigate to the scanner registry repository: +**Settings → Secrets and variables → Actions → New repository secret** + +| Secret Name | Value | +|---------------------------------------------|-----------------------------------------| +| `BOOST_SCAN_RUNNER_GITHUB_APP_ID` | The App ID from step 2 | +| `BOOST_SCAN_RUNNER_GITHUB_APP_PRIVATE_KEY` | Contents of the `.pem` file from step 3 | + +--- + +## 6. Usage in GitHub Actions Workflow + +```yaml +- name: Generate GitHub App Token + id: github-token + uses: actions/create-github-app-token@v1 + with: + app-id: ${{ secrets.GH_APP_ID }} + private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} + owner: your-org + repositories: test-runner-github + +- name: Run test-action + ... +``` + +--- + +## References + +- [Creating a GitHub App](https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/registering-a-github-app) +- [Authenticating as a GitHub App](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/about-authentication-with-a-github-app) +- [actions/create-github-app-token](https://github.com/actions/create-github-app-token) diff --git a/docs/setup-gitlab.md b/docs/setup-gitlab.md new file mode 100644 index 00000000..ab21bfee --- /dev/null +++ b/docs/setup-gitlab.md @@ -0,0 +1,82 @@ +# GitLab Test Runner Setup + +## 1. Create Test Runner Repository + +1. Create a new project named `scan-test-runner-gitlab-ci` in your GitLab group +2. Add the pipeline configuration at `.gitlab-ci.yml` +3. Ensure CI/CD pipelines are enabled for the project + +--- + +## 2. Create OAuth2 Application + +1. Navigate to your group settings: + **Group → Settings → Applications** + + Or for self-hosted: `https://{GITLAB_HOST}/groups/{GROUP}/-/settings/applications` + +2. Click **Add new application** + +3. Configure the application: + + | Field | Value | + |------------------|------------------------------------------------------| + | **Name** | `BoostSecurity.io Scan Test Runner` | + | **Redirect URI** | `http://localhost` (not used for client credentials) | + | **Confidential** | ✅ Checked | + | **Scopes** | ✅ `api` | + +4. Click **Save application** + +5. Note the **Application ID** and **Secret** - the secret is only shown once! + +--- + +## 3. Configure Secrets on Scanner Registry Repository + +Navigate to the scanner registry repository (GitHub): +**Settings → Secrets and variables → Actions → New repository secret** + +| Secret Name | Value | +|------------------------------------------|----------------------------| +| `BOOST_SCAN_RUNNER_GITLAB_CLIENT_ID` | Application ID from step 2 | +| `BOOST_SCAN_RUNNER_GITLAB_CLIENT_SECRET` | Secret from step 2 | + +--- + +## 4. Usage in GitHub Actions Workflow + +```yaml +- name: Generate GitLab OAuth Token + id: gitlab-token + run: | + response=$(curl -s -X POST "https://gitlab.com/oauth/token" \ + -d "grant_type=client_credentials" \ + -d "client_id=${{ secrets.GITLAB_CLIENT_ID }}" \ + -d "client_secret=${{ secrets.GITLAB_CLIENT_SECRET }}") + + token=$(echo "$response" | jq -r '.access_token') + echo "token=$token" >> $GITHUB_OUTPUT + echo "::add-mask::$token" + +- name: Run test-action + ... +``` + +--- + +## 5. Token Details + +| Property | Value | +|--------------|--------------------------------------------------| +| **Lifetime** | 2 hours | +| **Refresh** | New token generated per workflow run | +| **Scope** | `api` (required for pipeline trigger and status) | + +--- + +## References + +- [GitLab OAuth2 Provider](https://docs.gitlab.com/ee/integration/oauth_provider.html) +- [GitLab OAuth2 API](https://docs.gitlab.com/ee/api/oauth2.html#client-credentials-flow) +- [GitLab Pipelines API](https://docs.gitlab.com/ee/api/pipelines.html) diff --git a/docs/setup-tests.md b/docs/setup-tests.md new file mode 100644 index 00000000..1cb03db3 --- /dev/null +++ b/docs/setup-tests.md @@ -0,0 +1,128 @@ +# Setting Up Scanner Tests + +This guide explains how to write tests for scanners in the registry. + +## Overview + +Scanner tests are defined in a `tests.yaml` file within each scanner directory. When a scanner is modified in a pull request, the test system automatically detects the changes and runs the associated tests across configured CI/CD providers. + +## Test Definition Format + +Create a `tests.yaml` file in your scanner directory (e.g., `scanners/org/scanner-name/tests.yaml`): + +```yaml +version: "1.0" +tests: + - name: "Smoke test - source code" + type: "source-code" + source: + url: "https://github.com/OWASP/NodeGoat.git" + ref: "main" + + - name: "Container image scan" + type: "container-image" + source: + url: "https://github.com/example/docker-app.git" + ref: "v1.0" + scan_paths: + - "app" + - "api" +``` + +## Fields Reference + +| Field | Required | Description | +|----------------------|----------|--------------------------------------------| +| `version` | Yes | Schema version (currently "1.0") | +| `tests` | Yes | List of test specifications | +| `tests[].name` | Yes | Human-readable test name | +| `tests[].type` | Yes | Either `source-code` or `container-image` | +| `tests[].source.url` | Yes | Git repository URL (HTTPS) | +| `tests[].source.ref` | Yes | Git reference (branch, tag, or commit SHA) | +| `tests[].scan_paths` | No | Paths to scan (default: `["."]`) | +| `tests[].timeout` | No | Test timeout (default: `5m`) | + +## Test Types + +### Source Code (`source-code`) + +Tests that scan source code repositories. The scanner will clone the specified repository and run against the source files. + +```yaml +- name: "Python project scan" + type: "source-code" + source: + url: "https://github.com/example/python-app.git" + ref: "main" +``` + +### Container Image (`container-image`) + +Tests that scan container images. The repository should contain a Dockerfile or container definition. + +```yaml +- name: "Docker image scan" + type: "container-image" + source: + url: "https://github.com/example/docker-app.git" + ref: "v1.0" +``` + +## Multiple Scan Paths + +Use `scan_paths` to test multiple directories within a repository. Each path creates a separate test execution, enabling parallel testing: + +```yaml +- name: "Monorepo scan" + type: "source-code" + source: + url: "https://github.com/example/monorepo.git" + ref: "main" + scan_paths: + - "services/api" + - "services/web" + - "libs/common" +``` + +## Best Practices + +1. **Use stable references**: Prefer tags or commit SHAs over branches for reproducible tests +2. **Choose representative targets**: Select repositories that exercise your scanner's capabilities +3. **Keep tests fast**: Use smaller repositories when possible to reduce test time +4. **Test edge cases**: Include tests for both positive cases (findings expected) and negative cases (clean code) + +## How Detection Works + +The test system: + +1. Compares the PR branch against the base branch to find changed files +2. Extracts scanner identifiers from paths like `scanners/org/scanner/file.yaml` +3. Filters to only scanners that have a `tests.yaml` file +4. Runs tests for each modified scanner across all configured CI/CD providers + +## Example + +A complete `tests.yaml` for a security scanner: + +```yaml +version: "1.0" +tests: + - name: "OWASP NodeGoat - Known vulnerabilities" + type: "source-code" + source: + url: "https://github.com/OWASP/NodeGoat.git" + ref: "main" + + - name: "OWASP WebGoat - Java vulnerabilities" + type: "source-code" + source: + url: "https://github.com/WebGoat/WebGoat.git" + ref: "v2023.8" + timeout: "10m" + + - name: "Clean project - No findings expected" + type: "source-code" + source: + url: "https://github.com/example/clean-project.git" + ref: "v1.0.0" +``` diff --git a/scanners/boostsecurityio/trivy-fs/tests.yaml b/scanners/boostsecurityio/trivy-fs/tests.yaml new file mode 100644 index 00000000..d19c65d5 --- /dev/null +++ b/scanners/boostsecurityio/trivy-fs/tests.yaml @@ -0,0 +1,12 @@ +version: "1.0" +tests: + - name: "gitleaks" + type: "source-code" + source: + url: "git@github.com:gitleaks/gitleaks.git" + ref: "v8.15.2" + - name: "osv-scanner" + type: "source-code" + source: + url: "git@github.com:google/osv-scanner.git" + ref: "main" diff --git a/scanners/boostsecurityio/trivy-image/tests.yaml b/scanners/boostsecurityio/trivy-image/tests.yaml new file mode 100644 index 00000000..73e729ce --- /dev/null +++ b/scanners/boostsecurityio/trivy-image/tests.yaml @@ -0,0 +1,10 @@ +version: "1.0" +tests: + - name: "Image scanning" + type: "container-image" + source: + url: "https://github.com/martin-boost-dev/boost-poc-registry-testing-trivy" + ref: "main" + scan_paths: + - "rclone" + - "osv-scanner"