diff --git a/PR-FEED-README.md b/PR-FEED-README.md index 96ca82599..48ce731c1 100644 --- a/PR-FEED-README.md +++ b/PR-FEED-README.md @@ -8,19 +8,19 @@ This script generates RSS/Atom feeds and markdown pages from recently merged pul ## Features -- ✅ Fetches merged PRs from **both** repositories: +- Fetches merged PRs from **both** repositories: - **Private**: `rundeckpro/rundeckpro` (filtered by `release-notes/include` label) - **Public**: `rundeck/rundeck` (filtered by `release-notes/include` label) -- ✅ **Tag-based comparison**: Uses git tags instead of dates for accurate PR detection -- ✅ **SaaS cut support**: Limits PRs to those included in the most recent SaaS deployment cut -- ✅ **Submodule awareness**: Automatically finds the correct rundeck commit from rundeckpro tags -- ✅ **All merge strategies**: Handles merge commits, squash merges, and rebase merges -- ✅ Combines and sorts PRs by merge date across both repos -- ✅ Automatically removes `RUN-XXXX` prefixes from PR titles (matching notes.mjs logic) -- ✅ Generates both RSS 2.0 and Atom feeds -- ✅ Creates VuePress-compatible markdown pages -- ✅ Template-based content using Nunjucks (like notes.mjs) -- ✅ Comprehensive error handling +- **Tag-based comparison**: Uses git tags instead of dates for accurate PR detection +- **SaaS cut support**: Limits PRs to those included in the most recent SaaS deployment cut +- **Submodule awareness**: Automatically finds the correct rundeck commit from rundeckpro tags +- **All merge strategies**: Handles merge commits, squash merges, and rebase merges +- Combines and sorts PRs by merge date across both repos +- Automatically removes `RUN-XXXX` prefixes from PR titles (matching notes.mjs logic) +- Generates both RSS 2.0 and Atom feeds +- Creates VuePress-compatible markdown pages +- Template-based content using Nunjucks (like notes.mjs) +- Comprehensive error handling ## Initial Setup (One-time) @@ -295,7 +295,21 @@ node ./docs/.vuepress/pr-feed.mjs --output-dir=./docs/some-other-location ## Implementation Notes +### Shared Utilities with notes.mjs + +Both `pr-feed.mjs` and `notes.mjs` now use shared functions from `pr-utils.mjs`: + +**Shared Functions:** +- `fetchPRsBetweenTags()` - Compare two git tags to find all PRs +- `extractPRSection()` - Extract specific sections from PR bodies (e.g., "Release Notes") +- `cleanPRTitle()` - Remove RUN-XXXX prefixes from PR titles +- `parseSaasCutTag()` - Parse SaaS cut tag format to extract commit SHAs +- `getPreviousVersion()` - Auto-decrement version numbers (e.g., 5.17.0 → 5.16.0) + +This ensures consistent behavior across both scripts and reduces code duplication. + ### Pattern Matching with notes.mjs + This script follows the same patterns as `notes.mjs`: - Uses ES modules (`.mjs` extension) - Uses Nunjucks templates for content generation @@ -303,16 +317,24 @@ This script follows the same patterns as `notes.mjs`: - Loads environment variables with `dotenv` - Parses CLI arguments with `yargs` - Follows the same code style and structure +- Both now use tag-based comparison (not milestones) ### Key Differences from notes.mjs + - **SaaS vs Self-Hosted**: Focuses on SaaS deployments vs version-specific self-hosted releases -- **Tag-based comparison**: Uses git tag comparison instead of milestone-based PR queries +- **Continuous updates**: Generates standalone pages for ongoing updates vs one-time release notes - **SaaS cut awareness**: Limits PRs to those in the deployment cut (not everything in main) - **Submodule handling**: Automatically resolves rundeck submodule commits from rundeckpro tags -- **Continuous updates**: Generates standalone pages for ongoing updates - **Customer communication**: Designed for SaaS customers via RSS/Atom feeds -- **All merge strategies**: Handles merge commits, squash merges, and rebase merges -- **Shared logic**: Uses same PR title cleaning regex as notes.md.nj template +- **Feed generation**: Creates RSS 2.0 and Atom 1.0 feeds + +### Similarity to notes.mjs + +- **Both use tag-based comparison**: Compare git tags to find PRs between releases +- **Both handle all merge strategies**: Merge commits, squash merges, and rebase merges +- **Both use shared utilities**: Common functions from `pr-utils.mjs` +- **Both extract PR sections**: Pull "Release Notes" sections from PR bodies +- **Both filter by labels**: Use `release-notes/include` for enterprise PRs ## Technical Details @@ -338,6 +360,12 @@ The script parses this format to extract commit SHAs directly, eliminating the n This ensures the exact commits used in the build are compared, guaranteeing accuracy. +## Related Documentation + +- **[README.md](./README.md)** - Main documentation project setup and release notes generation +- **`notes.mjs`** - Self-hosted release notes generator (uses same shared utilities) +- **`pr-utils.mjs`** - Shared utility functions used by both scripts + ## License This script is part of the Rundeck documentation project and follows the same license as the main repository. diff --git a/README.md b/README.md index 9ebce39a5..262099395 100644 --- a/README.md +++ b/README.md @@ -84,42 +84,150 @@ The documentation is organized as follows: # How to Create Release Notes -Rundeck Core PRs are included by default. -Core PRs can excluded by labeling them with the `release-notes/exclude` label. +## Prerequisites -Rundeck Enterprise PRs are excluded by default. -Enterprise PRs can be included by labeling them with the `release-notes/include` label. +### GitHub API Token Setup -Create the file `.env` in the project root and add the line `GH_API_TOKEN=[TOKEN]` -replacing `[TOKEN]` with your GitHub API token. This token needs `repo` scope. +Create a `.env` file in the project root: +```bash +# Create .env file +touch .env + +# Add your GitHub API token +echo "GH_API_TOKEN=ghp_your_actual_token_here" > .env +``` + +Get a token at: https://github.com/settings/tokens + +**Required permissions:** +- `repo` - Full control of private repositories +- Access to `rundeckpro/rundeckpro` (private) +- Access to `rundeckpro/sidecar` (private) + +### PR Labeling + +Both Rundeck Core and Enterprise repositories use the **`release-notes/include`** label: +- PRs with this label will be included in release notes +- PRs without this label will be excluded + +**Release Notes Sections:** +The script automatically extracts content from the `## Release Notes` section in PR descriptions. Structure your PRs like this: + +```markdown ## Release Notes +Brief customer-facing description of the change. +This content will appear in the generated release notes. + +## Technical Details +Implementation specifics (not included in release notes). +``` + +## Tag-Based Release Notes (Current Approach) + +The release notes script uses **git tag comparison** to find all PRs between two releases. This ensures accuracy by capturing all changes in the actual git history. -Run the following with the milestone for the release. This will create/overwrite an existing entry for the release and automatically update related documentation and configuration files: +### Basic Usage ```bash -npm run notes -- --milestone=${1?milestone name} +# Generate release notes (auto-detects previous version) +# Example: 5.17.0 automatically compares with 5.16.0 +npm run notes -- --milestone=5.17.0 ``` -When run, the script will: -- Generate release notes for the specified milestone. - - This will require edits for proper dates, the Overview, etc. -- Add a new row for the release in `docs/history/release-calendar.md`. - - This file will require an update to the release date. -- Update `.docsearch/config.json` to set the correct version for search indexing. -- Update `docs/.vuepress/setup.js` with the new version information. -- Add the new version link to the sidebar in `docs/.vuepress/sidebar-menus/history.ts`. +### Version Auto-Detection -> Use wisely: running this command will overwrite existing release notes for the specified release and possibly duplicate config file updates. +| Target Version | Auto-detected Previous | What it compares | +|---------------|------------------------|------------------| +| `5.17.0` | `5.16.0` | All PRs between v5.16.0 and v5.17.0 tags | +| `5.17.1` | `5.17.0` | All PRs between v5.17.0 and v5.17.1 tags | +| `5.0.0` | `4.17.0` | All PRs between v4.17.0 and v5.0.0 tags | -## Draft Release Notes +### Custom Version Range -Run the following with the milestone for the release. This will create the file named draft.md to avoid overwriting any existing version: +For patch releases or special cases, specify the previous version: ```bash -npm run notes -- --milestone=${1?milestone name} --draft +# Compare 5.17.0 to 5.17.1 +npm run notes -- --milestone=5.17.1 --from-version=5.17.0 ``` +### Draft Mode (Recommended for Testing) + +Generate a draft without modifying configuration files: + +```bash +# Creates docs/history/5_x/draft.md +npm run notes:draft -- --milestone=5.17.0 + +# Review the draft +cat docs/history/5_x/draft.md + +# When satisfied, generate the final version +npm run notes -- --milestone=5.17.0 +``` + +**Tag Detection:** +- If the tag exists (e.g., `v5.17.0`), draft mode uses the exact tag comparison +- If the tag doesn't exist yet, draft mode automatically uses `HEAD` to preview what will be in the release + - You'll see: "Warning: Tag 5.17.0 not found, using HEAD for preview" + - This allows you to preview release notes before creating the tag + +## What the Script Does + +When you run `npm run notes -- --milestone=5.17.0`, it will: + +1. **Compare git tags** to find all PRs between versions (e.g., v5.16.0 → v5.17.0) +2. **Extract Release Notes sections** from PR bodies +3. **Collect contributor information** (excluding bots and internal accounts) +4. **Generate release notes markdown** at `docs/history/5_x/version-5.17.0.md` +5. **Update sidebar and navbar links** (unless `--draft` mode) +6. **Update configuration files**: + - `.docsearch/config.json` - Search indexing version + - `docs/.vuepress/setup.js` - Version information + - `docs/.vuepress/sidebar-menus/history.ts` - Version link + - `docs/.vuepress/navbar-menus/about.js` - Release notes link + - `docs/.vuepress/pr-feed-config.json` - PR feed baseline +7. **Handle all merge strategies** (merge commits, squash merges, rebase merges) + +> **Note**: The generated file requires manual edits for dates, Overview section, and final descriptions. + +## Typical Workflow + +### Self-Hosted Release Process + +```bash +# 1. Generate draft for review (before tagging) +# If tag doesn't exist yet, uses HEAD for preview +npm run notes:draft -- --milestone=5.17.0 + +# 2. Review the generated draft +code docs/history/5_x/draft.md + +# 3. Create the version tag +git tag v5.17.0 +git push origin v5.17.0 + +# 4. Generate final release notes (updates all configs) +# Now uses the exact tag for accurate PR detection +npm run notes -- --milestone=5.17.0 + +# 5. Edit the generated file for dates, overview, etc. +code docs/history/5_x/version-5.17.0.md + +# 6. Commit all changes +git add docs/history/5_x/version-5.17.0.md +git add docs/.vuepress/sidebar-menus/history.ts +git add docs/.vuepress/navbar-menus/about.js +git add docs/.vuepress/setup.js +git add .docsearch/config.json +git add docs/.vuepress/pr-feed-config.json +git commit -m "Release notes for 5.17.0" +git push +``` + +**Note:** You can preview with draft mode before creating the tag. Draft mode will use HEAD if the tag doesn't exist yet. + ## SaaS Development Updates Feed For generating RSS/Atom feeds and markdown pages showing recent PRs deployed to the SaaS platform (but not yet in self-hosted releases), see [PR-FEED-README.md](./PR-FEED-README.md). @@ -132,6 +240,61 @@ npm run pr-feed The feed date range is automatically updated when you create release notes - no manual configuration needed! +## Troubleshooting Release Notes + +### "Tag X.Y.Z not found" Warning or Error + +**In Draft Mode:** +- Shows warning: "Tag 5.18.0 not found, using HEAD for preview" +- Uses HEAD (main branch) to preview PRs +- This is normal before the tag is created + +**In Final Mode:** +- Shows error: "Tag 5.18.0 not found. Create the tag first or use --draft mode." +- Creates a placeholder file with minimal content +- Re-run after creating the tag to populate with actual PRs + +**Solution:** +1. Use draft mode to preview: `npm run notes:draft -- --milestone=5.18.0` +2. Create the tag when ready: `git tag v5.18.0 && git push origin v5.18.0` +3. Generate final notes: `npm run notes -- --milestone=5.18.0` + +**For repo-specific warnings** (`docs`, `ansible-plugin`): +- This is normal - these repos don't use version tags +- They'll be skipped gracefully + +### "GH_API_TOKEN environment variable is not set" + +**Solution**: Create `.env` file with your GitHub token (see Prerequisites above). + +### No PRs found + +**Check**: +- Version range is correct +- Tags exist in the repositories +- PRs are merged (not just closed) +- PRs have the required labels (`release-notes/include` for enterprise) + +### Get Help + +```bash +# View all available options +npm run notes -- --help +``` + +## Advanced Options + +```bash +# Specify custom previous version +npm run notes -- --milestone=5.17.1 --from-version=5.17.0 + +# Draft mode (doesn't update configs) +npm run notes:draft -- --milestone=5.17.0 + +# Direct script usage +node ./docs/.vuepress/notes.mjs --milestone=5.17.0 --from-version=5.16.0 +``` + ## Troubleshooting If you encounter errors running the site locally, follow these steps to ensure a clean environment and proper setup: diff --git a/docs/.vuepress/notes.mjs b/docs/.vuepress/notes.mjs index 8a9565efc..70689d524 100644 --- a/docs/.vuepress/notes.mjs +++ b/docs/.vuepress/notes.mjs @@ -7,12 +7,30 @@ import dotenv from 'dotenv'; import _yargs from 'yargs'; import { hideBin } from 'yargs/helpers'; import RundeckVersion from './version.mjs'; +import { fetchPRsBetweenTags, extractPRSection, getPreviousVersion } from './pr-utils.mjs'; const __dirname = path.dirname(fileURLToPath(import.meta.url)); dotenv.config(); -const argv = _yargs(hideBin(process.argv)).argv; +// Parse command line arguments +const argv = _yargs(hideBin(process.argv)) + .option('milestone', { + type: 'string', + description: 'Target version/milestone (e.g., 5.17.0)', + demandOption: true + }) + .option('from-version', { + type: 'string', + description: 'Previous version to compare from (e.g., 5.16.0). Auto-calculated if not provided.' + }) + .option('draft', { + type: 'boolean', + description: 'Generate as draft.md instead of version-X.Y.Z.md', + default: false + }) + .help() + .argv; const template = fs.readFileSync('./docs/.vuepress/notes.md.nj'); @@ -39,12 +57,66 @@ const excludeUsernames = [ ]; async function main() { + // Determine version range + const toVersion = argv.milestone; + const fromVersion = argv['from-version'] || getPreviousVersion(toVersion); + + console.log('=== Rundeck Release Notes Generator ===\n'); + console.log(`Comparing versions: ${fromVersion} → ${toVersion}\n`); + + // Check if toVersion tag exists by attempting to fetch from main repos + const gh = new Octokit({ auth: process.env.GH_API_TOKEN }); + let tagExists = false; + let useHead = false; + + // Try to find the tag in rundeck repo (main repo) + const tagFormats = [`v${toVersion}`, toVersion, `V${toVersion}`]; + for (const tag of tagFormats) { + try { + await gh.rest.repos.getReleaseByTag({ + owner: 'rundeck', + repo: 'rundeck', + tag: tag + }); + tagExists = true; + break; + } catch (error) { + if (error.status !== 404) { + // Some other error, try git refs + try { + await gh.rest.git.getRef({ + owner: 'rundeck', + repo: 'rundeck', + ref: `tags/${tag}` + }); + tagExists = true; + break; + } catch (refError) { + // Continue to next format + } + } + } + } + + // Determine behavior based on tag existence and mode + if (!tagExists) { + if (argv.draft) { + console.log(`\nWarning: Tag ${toVersion} not found, using HEAD for preview\n`); + useHead = true; + } else { + console.error(`\nERROR: Tag ${toVersion} not found.`); + console.error(` Create the tag first or use --draft mode to preview with HEAD.\n`); + console.log(`Continuing with empty results to create placeholder file...\n`); + // Continue execution but with empty results + } + } + const context = {}; - context.core = await getRepoData({ repo: 'rundeck', owner: 'rundeck' }, ['release-notes/include']); - context.enterprise = await getRepoData({ repo: 'rundeckpro', owner: 'rundeckpro' }, ['release-notes/include']); - context.docs = await getRepoData({ repo: 'docs', owner: 'rundeck' }, []); // No label filtering for docs (need all PRs for contributors) - context.ansible = await getRepoData({ repo: 'ansible-plugin', owner: 'rundeck-plugins' }, ['release-notes/include']); - context.runner = await getRepoData({ repo: 'sidecar', owner: 'rundeckpro' }, ['release-notes/include']); + context.core = await getRepoData({ repo: 'rundeck', owner: 'rundeck' }, fromVersion, toVersion, ['release-notes/include'], useHead); + context.enterprise = await getRepoData({ repo: 'rundeckpro', owner: 'rundeckpro' }, fromVersion, toVersion, ['release-notes/include'], useHead); + context.docs = await getRepoData({ repo: 'docs', owner: 'rundeck' }, fromVersion, toVersion, [], useHead); // No label filtering for docs (need all PRs for contributors) + context.ansible = await getRepoData({ repo: 'ansible-plugin', owner: 'rundeck-plugins' }, fromVersion, toVersion, ['release-notes/include'], useHead); + context.runner = await getRepoData({ repo: 'sidecar', owner: 'rundeckpro' }, fromVersion, toVersion, ['release-notes/include'], useHead); context.contributors = { ...context.core.contributors, ...context.docs.contributors, ...context.ansible.contributors }; context.version = new RundeckVersion({ versionString: argv.milestone }); @@ -163,86 +235,58 @@ function updateNavbarReleaseLink(version) { } } -// Helper: Extract a specific section from PR body -function extractPRSection(body, sectionName) { - if (!body) return null; - - // Match section headers like "## Release Notes" or "### Release Notes" - const sectionRegex = new RegExp( - `^#+\\s*${sectionName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\s*$`, - 'im' - ); - - const lines = body.split('\n'); - let inSection = false; - let sectionContent = []; - let sectionLevel = 0; +async function getRepoData(repo, fromVersion, toVersion, includeLabels, useHead = false) { + const gh = new Octokit({ auth: process.env.GH_API_TOKEN }); + + console.log(`Fetching PRs from ${repo.owner}/${repo.repo}...`); - for (let i = 0; i < lines.length; i++) { - const line = lines[i]; + try { + // Determine the head reference + const headRef = useHead ? 'main' : null; - if (sectionRegex.test(line)) { - // Found the section header - inSection = true; - sectionLevel = line.match(/^#+/)[0].length; - continue; - } + // Fetch PRs between tags using shared utility + const pulls = await fetchPRsBetweenTags( + gh, + repo.owner, + repo.repo, + fromVersion, + toVersion, + includeLabels, + [], // Exclude labels (e.g., 'wip', 'do-not-publish') - none currently needed + headRef + ); + + // Extract contributors (excluding bots and internal users) + const contributors = {}; - if (inSection) { - // Check if we've hit another section at same or higher level - const headerMatch = line.match(/^(#+)\s/); - if (headerMatch && headerMatch[1].length <= sectionLevel) { - break; // End of our section + for (const p of pulls) { + if (excludeUsernames.includes(p.user.login)) continue; + if (contributors[p.user.login]) continue; + + try { + const user = await gh.users.getByUsername({ username: p.user.login }); + contributors[user.data.login] = user.data; + } catch (error) { + console.warn(` Warning: Could not fetch user data for ${p.user.login}: ${error.message}`); } - sectionContent.push(line); } - } - - const content = sectionContent.join('\n').trim(); - return content || null; -} - -async function getRepoData(repo, includeLabels) { - const gh = new Octokit({ auth: process.env.GH_API_TOKEN }); - const milestones = await gh.issues.listMilestones({ ...repo }); + // Extract "Release Notes" section from all PRs + const pullsWithNotes = pulls.map(pr => ({ + ...pr, + releaseNotes: extractPRSection(pr.body, 'Release Notes') + })); - const milestone = milestones.data.find((m) => m.title === argv.milestone); + console.log(` Found ${pullsWithNotes.length} PRs and ${Object.keys(contributors).length} contributors\n`); - if (!milestone) { - console.error(`GitHub milestone ${argv.milestone} not found on ${repo.owner}/${repo.repo}.`); + return { + contributors, + pulls: pullsWithNotes, + }; + } catch (error) { + console.error(` Error fetching PRs from ${repo.owner}/${repo.repo}: ${error.message}`); return { contributors: {}, pulls: [] }; } - - const issuesResp = await gh.paginate(gh.issues.listForRepo, { - ...repo, - milestone: milestone.number, - state: 'closed', - labels: includeLabels.join(','), - per_page: 100, - }); - - const pulls = issuesResp.filter((i) => i.pull_request); - - const contributors = {}; - - for (const p of pulls) { - if (excludeUsernames.includes(p.user.login)) continue; - if (contributors[p.user.login]) continue; - const user = await gh.users.getByUsername({ username: p.user.login }); - contributors[user.data.login] = user.data; - } - - // Extract "Release Notes" section from all PRs - const pullsWithNotes = pulls.map(pr => ({ - ...pr, - releaseNotes: extractPRSection(pr.body, 'Release Notes') - })); - - return { - contributors, - pulls: pullsWithNotes, - }; } diff --git a/docs/.vuepress/pr-feed.mjs b/docs/.vuepress/pr-feed.mjs index 1360e5fbe..52bc24982 100644 --- a/docs/.vuepress/pr-feed.mjs +++ b/docs/.vuepress/pr-feed.mjs @@ -19,6 +19,7 @@ import dotenv from 'dotenv'; import _yargs from 'yargs'; import { hideBin } from 'yargs/helpers'; import nunjucks from 'nunjucks'; +import { fetchPRsBetweenTags, parseSaasCutTag, cleanPRTitle, extractPRSection } from './pr-utils.mjs'; const __dirname = path.dirname(fileURLToPath(import.meta.url)); @@ -84,32 +85,10 @@ function loadConfig() { } } -/** - * Parse the SaaS cut tag to extract commit SHAs - * Tag format: rba/${vNum}-RBA-${vDate}-${coreSha}-${proSha} - * Example: rba/5.18-RBA-20251030-2f39445-a6d9e14 - * - * @param {string} tag - SaaS cut tag - * @returns {Object} Object with rundeckSha and rundeckproSha, or null if parse fails - */ -function parseSaasCutTag(tag) { - // Tag format: rba/5.18-RBA-20251030-2f39445-a6d9e14 - // ^version ^date ^core ^pro - const match = tag.match(/^rba\/[\d.]+-RBA-\d{8}-([a-f0-9]+)-([a-f0-9]+)$/); - - if (!match) { - console.warn(` Warning: Could not parse SaaS cut tag format: ${tag}`); - return null; - } - - return { - rundeckSha: match[1], // coreSha - rundeck submodule commit - rundeckproSha: match[2] // proSha - rundeckpro commit - }; -} - /** * Fetch PRs merged after a specific tag using git comparison + * This is a wrapper around the shared fetchPRsBetweenTags that maintains + * backward compatibility with the "since tag" approach (tag -> head) * @param {Object} octokit - Initialized Octokit instance * @param {string} owner - Repository owner * @param {string} repo - Repository name @@ -119,117 +98,18 @@ function parseSaasCutTag(tag) { * @returns {Promise} Array of PR objects */ async function fetchPRsSinceTag(octokit, owner, repo, version, includeLabels = [], headRef = 'main') { - // Try different tag naming conventions - const tagFormats = [`v${version}`, version, `V${version}`]; - - let comparison = null; - let tagUsed = null; - - for (const tag of tagFormats) { - try { - // Compare tag to head reference - comparison = await octokit.rest.repos.compareCommits({ - owner, - repo, - base: tag, - head: headRef - }); - tagUsed = tag; - console.log(` Found tag ${tag}, ${comparison.data.ahead_by} commits ahead of ${headRef}`); - break; - } catch (error) { - if (error.status === 404) { - continue; // Try next format - } - throw error; - } - } - - if (!comparison) { - const errorMsg = `Tag for version ${version} not found in ${owner}/${repo} (tried: ${tagFormats.map(tag => `'${tag}'`).join(', ')})`; - console.error(` ERROR: ${errorMsg}`); - throw new Error(errorMsg); - } - - // Extract PR numbers from merge commits - const prNumbers = new Set(); - for (const commit of comparison.data.commits) { - // Check for standard merge commit pattern - const match = commit.commit.message.match(/Merge pull request #(\d+)/); - if (match) { - prNumbers.add(parseInt(match[1])); - } else { - // For squash merges or other commits, check if they're associated with a PR - try { - const { data: associatedPRs } = await octokit.rest.repos.listPullRequestsAssociatedWithCommit({ - owner, - repo, - commit_sha: commit.sha - }); - - // Add any merged PRs associated with this commit - associatedPRs.forEach(pr => { - if (pr.merged_at) { - prNumbers.add(pr.number); - } - }); - } catch (error) { - // Log errors for individual commits at debug level to aid troubleshooting - console.debug(` Debug: Could not fetch associated PRs for commit ${commit.sha}: ${error.message}`); - } - } - } - - console.log(` Found ${prNumbers.size} unique PRs (including squash merges)`); - - // Fetch full PR data and filter by labels - // Use batched parallel requests to improve performance and respect rate limits - // Processing 10 PRs at a time provides a good balance between speed and API courtesy - const BATCH_SIZE = 10; - const prNumbersArray = Array.from(prNumbers); - const prs = []; - - for (let i = 0; i < prNumbersArray.length; i += BATCH_SIZE) { - const batch = prNumbersArray.slice(i, i + BATCH_SIZE); - - // Fetch batch in parallel - const batchResults = await Promise.allSettled( - batch.map(prNumber => - octokit.rest.pulls.get({ - owner, - repo, - pull_number: prNumber - }) - ) - ); - - // Process results and filter by labels - batchResults.forEach((result, idx) => { - if (result.status === 'fulfilled') { - const pr = result.value.data; - const prLabels = pr.labels.map(label => label.name); - - // Check if PR has required labels - if (includeLabels.length === 0 || includeLabels.some(label => prLabels.includes(label))) { - // Check exclude labels - if (!CONFIG.excludeLabels.some(label => prLabels.includes(label))) { - prs.push({ - ...pr, - _repoOwner: owner, - _repoName: repo - }); - } - } - } else { - const prNumber = batch[idx]; - console.warn(` Warning: Could not fetch PR #${prNumber} from ${owner}/${repo}: ${result.reason?.message || 'Unknown error'}`); - } - }); - } - - console.log(` After label filtering: ${prs.length} PRs with required labels`); - - return prs; + // Use the shared utility, treating version as fromVersion and headRef as the target + // Note: fetchPRsBetweenTags expects toVersion as a tag name, but can accept a commit SHA via headRef param + return await fetchPRsBetweenTags( + octokit, + owner, + repo, + version, // fromVersion (base tag) + headRef, // toVersion (can be tag or commit SHA) + includeLabels, + CONFIG.excludeLabels, + headRef // Pass headRef again to force using it instead of trying tag formats + ); } function calculateSinceDate(config) { @@ -480,59 +360,7 @@ function generateMarkdown(prs) { return nunjucks.renderString(template.toString(), context); } -/** - * Clean PR title by removing RUN-XXXX prefixes -*/ -function cleanPRTitle(title) { - if (!title) return ''; - // Remove one or more RUN-XXXX prefixes with optional colons and spaces - return title.replace(/^(RUN-[0-9]+\s*)+:?\s*/g, '').trim(); -} - -/** - * Extract a specific section from PR body - * Looks for sections like "## Customer Summary" or "### Release Notes" - * @param {string} body - PR body text - * @param {string} sectionName - Section header to look for (case insensitive) - * @returns {string|null} Section content or null if not found - */ -function extractPRSection(body, sectionName) { - if (!body) return null; - - // Match section headers like "## Customer Summary" or "### Release Notes" - const sectionRegex = new RegExp( - `^#+\\s*${sectionName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\s*$`, - 'im' - ); - - const lines = body.split('\n'); - let inSection = false; - let sectionContent = []; - let sectionLevel = 0; - - for (let i = 0; i < lines.length; i++) { - const line = lines[i]; - - if (sectionRegex.test(line)) { - // Found the section header - inSection = true; - sectionLevel = line.match(/^#+/)[0].length; - continue; - } - - if (inSection) { - // Check if we've hit another section at same or higher level - const headerMatch = line.match(/^(#+)\s/); - if (headerMatch && headerMatch[1].length <= sectionLevel) { - break; // End of our section - } - sectionContent.push(line); - } - } - - const content = sectionContent.join('\n').trim(); - return content || null; -} +// cleanPRTitle and extractPRSection are now imported from pr-utils.mjs /** * Generate RSS 2.0 feed diff --git a/docs/.vuepress/pr-utils.mjs b/docs/.vuepress/pr-utils.mjs new file mode 100644 index 000000000..ad61b4f8a --- /dev/null +++ b/docs/.vuepress/pr-utils.mjs @@ -0,0 +1,261 @@ +/** + * pr-utils.mjs + * + * Shared utilities for fetching and processing GitHub Pull Requests + * based on tag comparisons. + */ + +/** + * Parse the SaaS cut tag to extract commit SHAs + * Tag format: rba/${vNum}-RBA-${vDate}-${coreSha}-${proSha} + * Example: rba/5.18-RBA-20251030-2f39445-a6d9e14 + * + * @param {string} tag - SaaS cut tag + * @returns {Object} Object with rundeckSha and rundeckproSha, or null if parse fails + */ +export function parseSaasCutTag(tag) { + // Tag format: rba/5.18-RBA-20251030-2f39445-a6d9e14 + // ^version ^date ^core ^pro + const match = tag.match(/^rba\/[\d.]+-RBA-\d{8}-([a-fA-F0-9]+)-([a-fA-F0-9]+)$/); + + if (!match) { + console.warn(` Warning: Could not parse SaaS cut tag format: ${tag}`); + return null; + } + + return { + rundeckSha: match[1], // coreSha - rundeck submodule commit + rundeckproSha: match[2] // proSha - rundeckpro commit + }; +} + +/** + * Fetch PRs merged between two tags using git comparison + * @param {Object} octokit - Initialized Octokit instance + * @param {string} owner - Repository owner + * @param {string} repo - Repository name + * @param {string} fromVersion - Starting version tag (e.g., "5.16.0") + * @param {string} toVersion - Ending version tag (e.g., "5.17.0") + * @param {Array} includeLabels - Labels that PRs must have (empty array = all PRs) + * @param {Array} excludeLabels - Labels to exclude from results + * @param {string} headRef - Optional head reference (defaults to trying the toVersion tag) + * @returns {Promise} Array of PR objects + */ +export async function fetchPRsBetweenTags(octokit, owner, repo, fromVersion, toVersion, includeLabels = [], excludeLabels = [], headRef = null) { + // Try different tag naming conventions for base (from) tag + const baseTagFormats = [`v${fromVersion}`, fromVersion, `V${fromVersion}`]; + + // Try different tag naming conventions for head (to) tag + const headTagFormats = headRef ? [headRef] : [`v${toVersion}`, toVersion, `V${toVersion}`]; + + let comparison = null; + let baseTagUsed = null; + let headTagUsed = null; + + // Try to find valid base tag + for (const baseTag of baseTagFormats) { + for (const headTag of headTagFormats) { + try { + // Compare base tag to head tag/ref + comparison = await octokit.rest.repos.compareCommits({ + owner, + repo, + base: baseTag, + head: headTag + }); + baseTagUsed = baseTag; + headTagUsed = headTag; + console.log(` Found tags ${baseTag}...${headTag}, ${comparison.data.ahead_by} commits ahead`); + break; + } catch (error) { + if (error.status === 404) { + continue; // Try next format + } + throw error; + } + } + if (comparison) break; + } + + if (!comparison) { + const errorMsg = `Tags for versions ${fromVersion}...${toVersion} not found in ${owner}/${repo}`; + console.warn(` Warning: ${errorMsg} - skipping this repository`); + return []; + } + + // Extract PR numbers from merge commits + const prNumbers = new Set(); + for (const commit of comparison.data.commits) { + // Check for standard merge commit pattern + const match = commit.commit.message.match(/Merge pull request #(\d+)/); + if (match) { + prNumbers.add(parseInt(match[1])); + } else { + // For squash merges or other commits, check if they're associated with a PR + try { + const { data: associatedPRs } = await octokit.rest.repos.listPullRequestsAssociatedWithCommit({ + owner, + repo, + commit_sha: commit.sha + }); + + // Add any merged PRs associated with this commit + associatedPRs.forEach(pr => { + if (pr.merged_at) { + prNumbers.add(pr.number); + } + }); + } catch (error) { + // Log errors for individual commits at debug level to aid troubleshooting + console.debug(` Debug: Could not fetch associated PRs for commit ${commit.sha}: ${error.message}`); + } + } + } + + console.log(` Found ${prNumbers.size} unique PRs (including squash merges)`); + + // Fetch full PR data and filter by labels + // Use batched parallel requests to improve performance and respect rate limits + // Processing 10 PRs at a time provides a good balance between speed and API courtesy + const BATCH_SIZE = 10; + const prNumbersArray = Array.from(prNumbers); + const prs = []; + + for (let i = 0; i < prNumbersArray.length; i += BATCH_SIZE) { + const batch = prNumbersArray.slice(i, i + BATCH_SIZE); + + // Fetch batch in parallel + const batchResults = await Promise.allSettled( + batch.map(prNumber => + octokit.rest.pulls.get({ + owner, + repo, + pull_number: prNumber + }) + ) + ); + + // Process results and filter by labels + batchResults.forEach((result, idx) => { + if (result.status === 'fulfilled') { + const pr = result.value.data; + const prLabels = pr.labels.map(label => label.name); + + // Check exclude labels first + if (excludeLabels.length > 0 && excludeLabels.some(label => prLabels.includes(label))) { + return; + } + + // Check if PR has required labels (empty array means no filtering) + if (includeLabels.length === 0 || includeLabels.some(label => prLabels.includes(label))) { + prs.push({ + ...pr, + _repoOwner: owner, + _repoName: repo + }); + } + } else { + const prNumber = batch[idx]; + console.warn(` Warning: Could not fetch PR #${prNumber} from ${owner}/${repo}: ${result.reason?.message || 'Unknown error'}`); + } + }); + } + + console.log(` After label filtering: ${prs.length} PRs with required labels`); + + return prs; +} + +/** + * Clean PR title by removing RUN-XXXX prefixes + * @param {string} title - PR title + * @returns {string} Cleaned title + */ +export function cleanPRTitle(title) { + if (!title) return ''; + // Remove one or more RUN-XXXX prefixes with optional colons and spaces + return title.replace(/^(RUN-[0-9]+\s*)+:?\s*/g, '').trim(); +} + +/** + * Extract a specific section from PR body + * Looks for sections like "## Customer Summary" or "### Release Notes" + * @param {string} body - PR body text + * @param {string} sectionName - Section header to look for (case insensitive) + * @returns {string|null} Section content or null if not found + */ +export function extractPRSection(body, sectionName) { + if (!body) return null; + + // Match section headers like "## Customer Summary" or "### Release Notes" + const sectionRegex = new RegExp( + `^#+\\s*${sectionName.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\s*$`, + 'im' + ); + + const lines = body.split('\n'); + let inSection = false; + let sectionContent = []; + let sectionLevel = 0; + + for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + + if (sectionRegex.test(line)) { + // Found the section header + inSection = true; + sectionLevel = line.match(/^#+/)[0].length; + continue; + } + + if (inSection) { + // Check if we've hit another section at same or higher level + const headerMatch = line.match(/^(#+)\s/); + if (headerMatch && headerMatch[1].length <= sectionLevel) { + break; // End of our section + } + sectionContent.push(line); + } + } + + const content = sectionContent.join('\n').trim(); + return content || null; +} + +/** + * Auto-decrement version to get previous version + * Examples: + * 5.17.0 -> 5.16.0 + * 5.17.1 -> 5.17.0 + * 5.0.0 -> 4.17.0 (assumes previous major version ended at .17.0) + * + * @param {string} version - Current version (e.g., "5.17.0") + * @returns {string} Previous version + */ +export function getPreviousVersion(version) { + const parts = version.split('.').map(Number); + + if (parts.length !== 3) { + throw new Error(`Invalid version format: ${version}. Expected format: X.Y.Z`); + } + + let [major, minor, patch] = parts; + + // If patch > 0, decrement patch + if (patch > 0) { + return `${major}.${minor}.${patch - 1}`; + } + + // If minor > 0, decrement minor + if (minor > 0) { + return `${major}.${minor - 1}.0`; + } + + // If major > 0, decrement major and assume previous ended at .17.0 + if (major > 0) { + return `${major - 1}.17.0`; + } + + throw new Error(`Cannot decrement version: ${version}`); +} + diff --git a/docs/history/5_x/version-5.17.0.md b/docs/history/5_x/version-5.17.0.md index 1e5a40d80..f42f2d307 100644 --- a/docs/history/5_x/version-5.17.0.md +++ b/docs/history/5_x/version-5.17.0.md @@ -44,6 +44,12 @@ Fixes a Runner Wizard error when creating a Linux ephemeral runner by ensuring p Adds support for passing custom arguments to scripts executed by the GitHubScriptPlugin, allowing users to specify script arguments with shell-like quoting and escaping functionality. +##### ::circle-dot:: Fix RSS Feeds plugin not recognizing Dates on Microsoft RSS Feeds + +Error that occurs when processing RSS feeds that use Z as the timezone indicator instead of standard abbreviations like GMT or UTC. + +##### ::circle-dot:: CVE-2025-58754 Axios Update + ## Rundeck Open Source Product Updates @@ -75,6 +81,9 @@ Adds a database index on the retry_execution_id column in the execution table to Adds logger configuration for the ExecutionsCleanUp job in the Remco log4j2 template for Docker images. The change enables proper logging visibility for cleanup execution history operations without requiring global log level modifications. +##### ::circle-dot:: [Remote URL Job Option is not going through a configured proxy](https://github.com/rundeck/rundeck/pull/9860) + +##### ::circle-dot:: [Fix CVE-2025-58754 Axios in ui-trellis](https://github.com/rundeck/rundeck/pull/9859) [Here is a link to the full list of public PRs](https://github.com/rundeck/rundeck/pulls?q=is%3Apr+milestone%3A5.17.0+is%3Aclosed) diff --git a/package.json b/package.json index 29835ae59..bb64dc214 100644 --- a/package.json +++ b/package.json @@ -8,6 +8,7 @@ "docs:no-cache": "vuepress dev docs --no-cache", "deploy:github": "npm run docs:build; push-dir --dir=docs/.vuepress/dist --branch=gh-pages --cleanup", "notes": "node ./docs/.vuepress/notes.mjs", + "notes:draft": "node ./docs/.vuepress/notes.mjs --draft", "pr-feed": "node ./docs/.vuepress/pr-feed.mjs --include-section=\"Release Notes\"", "docs:clean-dev": "vuepress dev docs --clean-cache", "docs:update-package": "npx vp-update"