Bundle WordPress translation files with download retry and failure handling by ivan-ottinger · Pull Request #2575 · Automattic/studio

ivan-ottinger · 2026-02-11T15:21:54Z

Related issues

Context

#2558 bundled WordPress translation files for faster site creation but was reverted in #2572 due to CI build failures — ConnectTimeoutError when downloading ~114 translation ZIP files sequentially from downloads.wordpress.org during npm install. The root cause was that language pack downloads ran on every npm install via postinstall.

Related discussion: p1770812916266539-slack-C06DRMD6VPZ.

What changed from the original PR

This PR re-applies #2558 with the following fixes:

Language pack downloads moved out of postinstall into a standalone script (scripts/download-language-packs.ts) that runs during npm run make via the forge prePackage hook.
npm install no longer downloads language packs
Retry with exponential backoff — failed HTTP requests are retried up to 3 times with exponential backoff (1s, 2s, 4s) to handle transient network issues like ConnectTimeoutError.
Download failure is fatal during make — if a download still fails after retries, the build fails, ensuring all translations are included in the release.
npm run download-language-packs available as a standalone command for manual use.
Skips translations for older bundled themes (twentytwentythree, twentytwentyfour) to reduce download count.

Proposed Changes (from original PR)

Download WordPress core, plugin, and theme translation files at build time for all 19 supported non-English locales.
At app startup, copy bundled language packs from app resources to the runtime server-files/language-packs/ directory.
During site creation for the latest WP version, copy locale-specific .l10n.php and .json files directly into the site's wp-content/languages/ directory and set WPLANG via blueprint steps (defineWpConfigConsts + setSiteOptions) — no network round-trip needed.
Fall back to the original setSiteLanguage blueprint step for non-latest WP versions or when bundled packs are unavailable.
Auto-detect bundled plugins and themes from the extracted WordPress installation for translation downloads.
Only bundle .l10n.php and .json files (WordPress 6.5+ format), skipping .po and .mo to reduce size (~29MB vs ~82MB).
Make setupWPServerFiles resilient so individual step failures don't prevent subsequent steps from running.

Testing Instructions

Testing the built app

Build the app with npm run make.
Open the content of the built app:

Ensure the translation files are bundled: Studio.app/Contents/Resources/wp-files/latest/languages:

Set the locale in the app settings to a non-English language (e.g. Swedish).
Create a new site and verify:
- Site creation completes without downloading translations from WordPress.org. → To test this, you can go offline. Also, the site creation process should take about 3 - 4 seconds less than when testing with the current trunk.
- The WP Admin is displayed in the selected language.
- Plugin and theme translations are present.
- You should also find translations in the site's wp-content.
Ensure you are online. Create a site with a non-latest WP version (e.g. 6.5) and verify it falls back to downloading translations. → The process should take 3 - 4 seconds longer compared to creating site with the latest WP version.
Again as with the previous site, everything in WP Admin should be translated, including themes and plugins.
Switch the locale to English.
Create a site and verify no translations appear in the site's wp-content.

Testing the dev environment

Run npm install.
Head over to your project root and confirm there is no languages directory present in wp-files/latest.
Run npm run download-language-packs and verify language packs are downloaded. You can find them in the project root: wp-files/latest/languages.
Run npm start and set the locale in the app settings to a non-English language (e.g. Swedish).
You can follow similar steps as with testing the built app (steps 5 - 9).
Run npm test and verify all tests pass.

Pre-merge Checklist

Have you checked for TypeScript, React or other console errors?

wojtekn · 2026-02-11T15:57:09Z

@ivan-ottinger I think we need to address the request limits part, too, so we don't have flaky builds.

ivan-ottinger · 2026-02-12T11:46:28Z

@ivan-ottinger I think we need to address the request limits part, too, so we don't have flaky builds.

I have applied logic with three retries with delay of 1, 2 and 4 seconds. We could also add a small delay between each request on top of that. What do you think?

Or would you prefer adding that caching logic as well? I wonder if it is not too much though - in case it turns out there are no issues with the current approach anymore.

nightnei

@ivan-ottinger I tested and everything works well, only one step which I didn't get how to test - The WordPress admin is displayed in the selected language.. What is WP admin?
Also left some small comments.

But I would also wait for Wojtek's feedback on his comment, so I propose postponing this PR to the next release. WDYT, is it ok?

common/lib/wp-locales.ts

scripts/download-language-packs.ts

ivan-ottinger · 2026-02-12T13:52:44Z

Thank you for the review, Vova!

@ivan-ottinger I tested and everything works well, only one step which I didn't get how to test - The WordPress admin is displayed in the selected language.. What is WP admin?

Yes, WP Admin. I have updated it in the description to make it clear. 🙂

Also left some small comments.

Thanks! I will take a look at them.

But I would also wait for Wojtek's feedback on his comment, so I propose postponing this PR to the next release. WDYT, is it ok?

I agree, let's leave this PR for the next release.

wpmobilebot · 2026-02-12T14:09:44Z

📊 Performance Test Results

Comparing dd1270d vs trunk

site-editor

Metric	trunk	`dd1270d`	Diff	Change
load	2767.00 ms	2755.00 ms	-12.00 ms	⚪ 0.0%

site-startup

Metric	trunk	`dd1270d`	Diff	Change
siteCreation	6070.00 ms	6076.00 ms	+6.00 ms	⚪ 0.0%
siteStartup	3930.00 ms	3924.00 ms	-6.00 ms	⚪ 0.0%

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff)

fredrikekelund

This works as expected and LGTM 👍 I left a couple of nit comments, but nothing blocking. We could potentially land this PR and follow up with a separate PR for addressing any small tweaks we want to do.

package.json

cli/commands/site/create.ts

fredrikekelund · 2026-02-12T15:42:42Z

src/setup-wp-server-files.ts

 export async function setupWPServerFiles() {
-	await copyBundledLatestWPVersion();
-	await copyBundledSqlite();
-	await copyBundledWPCLI();
-	await copyBundledSQLiteCommand();
-	await copyBundledTranslations();
+	const steps: Array< [ string, () => Promise< void > ] > = [
+		[ 'WordPress version', copyBundledLatestWPVersion ],
+		[ 'SQLite integration', copyBundledSqlite ],
+		[ 'WP-CLI', copyBundledWPCLI ],
+		[ 'SQLite command', copyBundledSQLiteCommand ],
+		[ 'translations', copyBundledTranslations ],
+		[ 'language packs', copyBundledLanguagePacks ],
+	];
+
+	for ( const [ name, step ] of steps ) {
+		try {
+			await step();
+		} catch ( error ) {
+			console.error( `Failed to set up bundled ${ name }:`, error );
+		}
+	}
 }


This isn't the fault of this PR, but I really want to revisit why we need to do this. We're basically copying the entire wp-files directory to a single server-files location. Why do we need to do this? Seems like potential legacy logic that we could simplify…

Good observation. I let Claude look at this and here are its observations (I haven't verified everything):

1. The app bundle is read-only — Electron's Resources/ directory (ASAR-packed) can't be modified at runtime. The app needs a writable location because it downloads newer versions of WordPress, SQLite, and WP-CLI from the internet. 2. Auto-updates — The app checks for and downloads newer WordPress versions, storing them in server-files/. The bundled version is just the initial seed. 3. Version management — server-files/wordpress-versions/ holds multiple WP versions side by side (e.g., latest/, 6.4.3/). Not possible in the static app bundle. 4. Offline resilience — The bundled files guarantee the app works on first launch without internet.

* Bundle WordPress core language packs to speed up non-English site creation Instead of downloading translation files from WordPress.org at runtime via the setSiteLanguage blueprint step, bundle core language packs for all 19 supported non-English locales at build time. During site creation for the latest WP version, files are copied locally and WPLANG is set via defineWpConfigConsts + setSiteOptions. Falls back to setSiteLanguage for non-latest versions or when bundled packs are unavailable. Also makes setupWPServerFiles resilient so individual step failures don't prevent subsequent steps from running. * Drop .po and .mo files from bundled language packs WordPress 6.5+ uses .l10n.php files, making .po (source) and .mo (legacy binary) files unnecessary at runtime. Removing them reduces the language packs from 80MB to 27MB. Also fixes ts-node module resolution by inlining the locale list in the download script instead of importing from common/lib/locale.ts. * Bundle translations for default plugins and themes Download and bundle translation files for akismet, twentytwentyfive, twentytwentyfour, and twentytwentythree alongside core translations. Plugin/theme translations are stored in languages/plugins/ and languages/themes/ subdirectories and copied to the site during creation. Adds ~2MB to the bundle. * Fix temp file path for plugin/theme language pack downloads Replace slashes in the label used for temp zip filenames to avoid creating subdirectories in the temp folder. * Extract WordPress locale list to shared module Move the list of WordPress locale codes to common/lib/wp-locales.ts so it can be imported by both the download script (ts-node) and app code (Vite) without module resolution issues. Remove the unused studioToWpLocaleMap from common/lib/locale.ts. * Add hello-dolly to bundled plugin translations * Remove as const from WP_LOCALES to fix ts-node compilation * Align language pack download messages with existing status format * Auto-detect bundled plugins and themes for translation downloads Read the wp-content/plugins and wp-content/themes directories from the extracted WordPress installation instead of hardcoding the list. Handles single-file plugins like hello.php via a slug mapping. * Remove redundant comment in site creation language setup

Prevents flaky CI builds by retrying failed requests up to 3 times with exponential backoff (1s, 2s, 4s). Failed downloads now fail the build instead of silently skipping, ensuring all translations are included.

wojtekn · 2026-02-13T09:45:51Z

I have applied logic with three retries with delay of 1, 2 and 4 seconds. We could also add a small delay between each request on top of that. What do you think?

Okay, let's try with that. It seems that the chance of getting rate-limited is lower after moving the script out from npm install, so maybe that would be enough.

ivan-ottinger · 2026-02-13T09:49:29Z

Okay, let's try with that. It seems that the chance of getting rate-limited is lower after moving the script out from npm install, so maybe that would be enough.

Thanks, Wojtek! Yes, I think it should be far less likely to hit the limit now. In case we still run into issues, I am happy to revert again and we can apply additional measures.

fredrikekelund · 2026-02-13T09:51:40Z

I realize we still package the app at least four times for every PR push (E2E tests for macOS and Windows, and performance job builds for the PR branch and trunk).

So, there's still a fair amount of downloading going on, even with this optimization. We previously discussed improving CI efficiency by reusing a single packaging artifact across multiple jobs. We should look into that.

ivan-ottinger · 2026-02-13T10:07:09Z

So, there's still a fair amount of downloading going on, even with this optimization. We previously discussed improving CI efficiency by reusing a single packaging artifact across multiple jobs. We should look into that.

That would be great. I see we have a related Linear issue here already: STU-1045.

ivan-ottinger self-assigned this Feb 11, 2026

ivan-ottinger changed the title ~~Fix language pack download timeouts in CI~~ Bundle WordPress translation files for faster site creation Feb 11, 2026

ivan-ottinger force-pushed the fix/language-pack-download-timeouts branch from e4d9c7d to dcd9d36 Compare February 11, 2026 15:33

ivan-ottinger force-pushed the fix/language-pack-download-timeouts branch from b4ee240 to 73d516c Compare February 12, 2026 10:44

ivan-ottinger marked this pull request as ready for review February 12, 2026 11:46

ivan-ottinger requested a review from a team February 12, 2026 11:46

nightnei approved these changes Feb 12, 2026

View reviewed changes

common/lib/wp-locales.ts Show resolved Hide resolved

scripts/download-language-packs.ts Outdated Show resolved Hide resolved

ivan-ottinger changed the title ~~Bundle WordPress translation files for faster site creation~~ Bundle WordPress translation files with download retry and failure handling Feb 12, 2026

fredrikekelund approved these changes Feb 12, 2026

View reviewed changes

ivan-ottinger and others added 9 commits February 12, 2026 17:21

Move language pack downloads out of postinstall into npm run make

bd691fa

Skip translation downloads for older bundled themes

1a272b6

Add retry with exponential backoff to language pack downloads

60380fe

Prevents flaky CI builds by retrying failed requests up to 3 times with exponential backoff (1s, 2s, 4s). Failed downloads now fail the build instead of silently skipping, ensuring all translations are included.

Add Hungarian locale to language pack downloads

7672a87

Refactor fetchWithRetry to use recursion instead of for loop

99abb52

Remove redundant 'Files extracted' log message

5bcca78

Rename download:language-packs script to download-language-packs

a11ea8d

Rename usedBundledPacks to isUsingBundledLanguagePacks for clarity

dd1270d

ivan-ottinger force-pushed the fix/language-pack-download-timeouts branch from 942a060 to dd1270d Compare February 12, 2026 16:21

ivan-ottinger added the Reviewer can merge label Feb 12, 2026

ivan-ottinger merged commit d675323 into trunk Feb 17, 2026
11 checks passed

ivan-ottinger deleted the fix/language-pack-download-timeouts branch February 17, 2026 10:44

Conversation

ivan-ottinger commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related issues

Context

What changed from the original PR

Proposed Changes (from original PR)

Testing Instructions

Testing the built app

Testing the dev environment

Pre-merge Checklist

Uh oh!

wojtekn commented Feb 11, 2026

Uh oh!

ivan-ottinger commented Feb 12, 2026

Uh oh!

nightnei left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ivan-ottinger commented Feb 12, 2026

Uh oh!

wpmobilebot commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Performance Test Results

site-editor

site-startup

Uh oh!

fredrikekelund left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

fredrikekelund Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

ivan-ottinger Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

wojtekn commented Feb 13, 2026

Uh oh!

ivan-ottinger commented Feb 13, 2026

Uh oh!

fredrikekelund commented Feb 13, 2026

Uh oh!

ivan-ottinger commented Feb 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ivan-ottinger commented Feb 11, 2026 •

edited

Loading

nightnei left a comment •

edited

Loading

wpmobilebot commented Feb 12, 2026 •

edited

Loading