Add JSPI support for PHP WASM extensions#2560
Conversation
247fa61 to
b1bade5
Compare
📊 Performance Test ResultsComparing cc4c26a vs trunk site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
b1bade5 to
029766e
Compare
|
|
||
| async function isMultiWorkerEnabled() { | ||
| // Enable multi-worker support for E2E tests (requires JSPI-enabled Node binary) | ||
| if ( process.env.E2E === 'true' ) { |
There was a problem hiding this comment.
I couldn't make it work without enabling multi-worker for E2E yet.
There was a problem hiding this comment.
I recommend we revert this and add more debugging to the CI E2E runs to understand why the failure occurs. Playwright has built-in support for capturing screenshots, which should be a good start.
If you write the screenshot as a PNG to {cwd}/test-results, it will show up in the Artifacts tab in Buildkite after the test finishes.
There was a problem hiding this comment.
Yes, I wouldn't like to merge the PR with this condition.
Tests were failing as the Studio site was timing out, and tests couldn't find the element, so I think saving a screenshot wouldn't help in this case. We may need to add more debugging on the Studio CLI process side (or pm2, as I suspected the problem could be there).
|
Marking as ready for review to trigger E2E tests. |
|
30% slower Site Editor load time isn't encouraging 🤔 |
|
The E2E tests previously failed due to a 502 Bad Gateway error in the headless browser that accessed wp-admin. Updating to the latest version of Playground packages fixed that. If the compare-perf job continues to report severe Site Editor load-time degradation, we should investigate. |
|
Looking at https://codevitals.run/public/Automattic/studio/metrics?metric=site-editor-load, there seems to be some inherent noise in this metric… Given that multi-worker support is around the corner and that the nominal load time isn't too bad, I suggest landing this PR and potentially returning to this if we find it to be an issue later. @wojtekn, would you mind reviewing this PR and checking my changes? Here's what I've done:
|
…port # Conflicts: # cli/package-lock.json # cli/package.json # package-lock.json # package.json
|
Thanks for the updates @fredrikekelund. Asyncify cleanup looks safer now. FYI, I merged the trunk into the branch as I merged the package updates branch there. I see the site editor load is consistently slower by 30%. @adamziel do you consider JSPI stable enough to use in production? Any thoughts on why it would result in a 30% performance drop? |
It's much more stable than Asyncify. If it wasn't for Safari and older Node.js versions, Playground wouldn't even support Asyncify anymore. The majority of Playgrounds run on JSPI. It should be much faster, too. Asyncify adds a lot of extra instrumentation code that isn't necessary with JSPI. Let's dig into this. |
This reverts commit 5dee61c.
I believe we made the test environment behave differently to appease Jest. If that’s not required now that we have Vitest, then it’s really better to follow the approach Wojtek took here. I'm temporarily undoing this to verify that it doesn't affect the performance tests.
|
I have reviewed this and I have not found any issues, but regarding performance, I have asked Claude to run the This is the summary: SummaryThe
Load Phase Breakdown (measured run)
The regression is concentrated in the "Spinners cleared" phase, which corresponds to REST API requests made by the site editor after initial render. Individual REST API Request Durations (measured run)
Key Findings
ConclusionThe regression is in PHP WASM execution speed itself, not in concurrency or browser-side behavior. JSPI appears to add overhead to every PHP request cycle. The simpler/faster endpoints (like |
|
Let's measure once again after @brandonpayton merges the worker support PR later today, it changes how some message passing works internally |
|
Multi-worker is now shipped for Playground CLI, and npm packages have been published. The --experimental-multi-worker flag is now deprecated and has no effect. |
@adamziel @brandonpayton I'm updating PHP-WASM in #2610. However, would it make sense to still investigate why requests got slower after switching from asyncify to JSPI? Even if enabling multi-worker masks the slowdown, we might be carrying that regression into the multi-worker setup as well — fixing it could make things even faster overall. |
|
@fredrikekelund thanks for fixing lock file. I think I generated it using another version of NPM. I've just pulled changes, ran |
fredrikekelund
left a comment
There was a problem hiding this comment.
The performance tests are passing again, and it seems like the switch to JSPI no longer affects performance negatively after the Playground 3.1.1 upgrade. Let's land this 👍
|
I had my tests one prompt away, so I have executed them again with the latest changes. This is the result, very promising:
Total Load Time
Individual REST API Request Durations
Key Observations
|
|
@epeicher, that's not measuring against the latest |
@fredrikekelund , that's right, it's comparing against the previous metrics I took. Let me try again comparing with latest trunk. Site Editor Load Performance: trunk vs jspi branch (latest)
REST API Request Durations (measured run)
ConclusionResults are essentially identical — the 9ms difference is within normal variance. The JSPI changes are performance-neutral. Both branches show the same concurrent request pattern with individual REST API requests completing in ~320-370ms. |
|
Thanks for checking, @epeicher! It's good to have another confirmation that JSPI no longer seems to cause any performance regression. |
|
@fredrikekelund @epeicher I merged trunk it, and I see there are no performance drops vs trunk. |
|
|
Related issues
Fixes STU-1244
Proposed Changes
I propose to:
--experimental-wasm-jspiflagTesting Instructions
Confirm:
Pre-merge Checklist