Add JPEG XL support, loaded on demand - #77584
Conversation
Add client-side JPEG XL support without growing the vips worker bundle. The 3 MB vips-jxl.wasm module is split into its own @wordpress/vips/jxl-wasm script module that is only fetched the first time a JXL image is processed. How it works: - packages/vips/src/jxl-wasm.ts is a tiny new script module whose only job is to export the base64 data URL for vips-jxl.wasm. - @wordpress/vips/jxl-wasm is registered as a new wpScriptModuleExports entry so the build emits build/modules/vips/jxl-wasm.min.js (~3 MB) as a separately loadable module. - vips-worker.ts adds vipsEnsureJxlSupport(), which on first call dynamic-imports '@wordpress/vips/jxl-wasm' to get the data URL, then RPC-passes it to the worker via setJxlWasmUrl(). - The worker (packages/vips/src/index.ts) stores the URL, adds vips-jxl.wasm to dynamicLibraries on the next getVips() call, and returns the URL from locateFile(). If vips was already initialized without JXL, the existing instance is discarded so the reinit picks up JXL support. - packages/upload-media calls vipsEnsureJxlSupport() from prepareItem whenever the input or output type is image/jxl. image/jxl is added to CLIENT_SIDE_SUPPORTED_MIME_TYPES and 'jxl' to VALID_IMAGE_FORMATS and the ImageFormat type. The vipsConvertImageFormat wrapper MIME union is widened accordingly. - JXL encoding uses effort=3 (libvips default 7 is too slow for interactive use). Size impact: - worker.min.js: unchanged (~13.1 MB, same as trunk). - jxl-wasm.min.js: new separate module (~3.0 MB), fetched only when a JXL image is encountered. Alternative to #77570, which bundles vips-jxl.wasm directly into the worker (+3 MB on every editor page load). Opened so the size bot can compare. Refs #76981.
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
Size Change: +850 kB (+10.74%) Total Size: 8.76 MB 📦 View Changed
|
|
Flaky tests detected in 4ce3bac. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/33117672777 Navigates the items list via UP/DOWN arrow keys in
|
Resolve conflict in packages/upload-media/src/store/private-actions.ts: trunk moved image format transcoding from client-side prepareItem to server-driven thumbnail generation. Adopt the new pipeline and keep the JXL WASM lazy-load, but trigger it in two places: prepareItem for JXL input files, and transcodeImageItem for server-requested JXL output.
|
A few notes from a review pass: Bundle size / lighter-weight JXL options (re #76981)
Minor code note
|
JXL is not broadly web-compatible: most browsers (including Chrome) cannot display it and the server cannot read it (GD/Imagick have no JXL decoder, and fileinfo reports it as image/x-jxl). Uploading JXL as-is was rejected outright — by the editor's allowed-MIME check and by core's wp_check_filetype_and_ext() — and even when allowed produced an undisplayable attachment with no dimensions or sub-sizes. Decode JXL to JPEG client-side with vips (the JXL WASM module is already lazy-loaded on demand) and upload the JPEG, mirroring how HEIC is handled. The original .jxl is preserved as a companion file in $metadata['original'] so no data is lost. The editor and front end now use the portable JPEG, with real dimensions and the full set of sub-sizes. - Register image/jxl as an allowed upload MIME type, and restore the type during validation via a magic-byte-checked wp_check_filetype_and_ext filter so the sideloaded original passes despite the image/x-jxl finfo mismatch. - Add original-jxl sideload handling to the REST controller, skipping the dimension read since JXL cannot be measured server-side. - Generalize the HEIC companion delete hook to clean up JXL originals too. - Add an e2e test and JXL asset covering the conversion and companion.
# Conflicts: # packages/upload-media/src/store/private-actions.ts
|
I did some testing on this feature, made some small fixes and got it working, it now properly handles uploaded JXL files. I created a JXL using Squoosh for testing: Because JPEG XL (JXL) is not supported in Chromium where client-side media is active, and generally to provide a web-safe format for users, I decided to treat JXL uploads similar to HEIC uploads. So we do the following:
The output format should also honor the See screencast: jxl.to.jpeg.on.upload.mp4 |
|
I will work on a core backport once the core client-side media feature is restored. |
# Conflicts: # packages/vips/CHANGELOG.md # packages/vips/README.md
Guard the contract that high-bit-depth (>8-bit) and gain-map JXL uploads flatten to 8-bit JPEG sub-sizes while the full-fidelity original is preserved byte-for-byte as the .jxl companion file. Add two 200x150 fixtures (a genuine 16-bit JXL and an 8-bit JXL carrying an ISO 21496-1 jhgm gain-map box) plus a self-contained generator script. Both decode in wasm-vips to a JPEG derivative; the tests assert the JPEG main + sub-size and verify the stored original is identical to the upload, proving the bit depth and gain map survive in the original.
# Conflicts: # packages/upload-media/CHANGELOG.md # packages/upload-media/src/store/types.ts
# Conflicts: # packages/upload-media/CHANGELOG.md # packages/upload-media/src/store/private-actions.ts # packages/vips/src/worker.ts # test/e2e/specs/editor/various/client-side-media-processing.spec.js
The .gen-jxl-fidelity-fixtures.mjs helper failed lint: prettier wanted the over-length magick/cjxl argument arrays wrapped one-per-line, and the file-level eslint-disable no-console had no matching eslint-enable. Wrap the arrays and add the closing directive.
# Conflicts: # lib/media/class-gutenberg-rest-attachments-controller.php # lib/media/load.php # packages/upload-media/CHANGELOG.md
|
@andrewserong & @swissspidy - this is ready for review. My main concern with this feature is the added bundle size weight from adding JXL support, especially given the current lack of browser support, see https://caniuse.com/jpegxl. Perhaps it would be best to leave this off for now, or consider making it plugin-only (that is, available in the Gutenberg plugin, but not merged to core) or even an Experiment in Gutenberg so we can easily remove it later without breaking commitments. I acknowledge that if we did choose to add JXL support this would be a signal to browsers that they should add support, but I'm hesitant to merge it into core for now. given the current state of browser support and likely very low usage of the format in the wild. |
Resolve conflict in packages/vips/src/index.ts with trunk's #79188, which switched inlined WASM from base64 data URLs to a Uint8Array wrapped in a Blob URL on demand (getWasmUrl helper). Align the JXL lazy-load path with the new scheme: the vips-jxl.wasm module now resolves to a Uint8Array, so pass those bytes across the worker RPC and wrap them in a Blob URL inside the worker, exactly like the HEIF library. Rename setJxlWasmUrl -> setJxlWasm (and vipsSetJxlWasmUrl -> vipsSetJxlWasm) since the value is bytes, not a URL, updating the worker wiring, README, and CHANGELOG to match.
@swissspidy / @andrewserong / @gregbenz - any objection to punting JXL support to a later release if/when it gets better stronger support? I'm not sure its worth the increased payload size (11% according to the size bot: #77584 (comment)) |
|
Sounds good to me 👍 |
|
@adamsilverstein I think that's fine and probably ideal at this time to manage risk and focus on landing what's already a significant change. It's definitely something to keep on the radar and test when appropriate, but I thiink it is early to deploy. https://caniuse.com/jpegxl puts support at 14% (Safari). Chromium works under chrome://flags/#enable-jxl-image-format and FireFox Nightly has it, so this can grow rapidly. But it isn't a format which will be safe to use on the web for a bit. JXL is an excellent format and would love to see it added not long after Chromium / FF start to propagate with default support. I don't see 11% size increase for a cached WASM used only for admins as a concern from my perspective, just not valuable yet given state of the ecosystem. Do we have any benchmark data on performance transcoding to JXL vs AVIF? I expect that may be a benefit (no need to test anything now if the data is not readily available for a wasm-vips JXL scenario). |
|
Punting for now sounds good to me, too 👍 |
|
Thanks for the feedback... I have converted the PR to draft to indicate we aren't planning to land it currently, I will comment on the issue as well. |
…inal The JXL source-format companion (the preserved `.jxl`) is stored under the `source_image` attachment-metadata key (`META_KEY_SOURCE_IMAGE`), the same key the HEIC original uses, not a bare `original` key. Two stale references still pointed at `original`: - A comment in `private-actions.ts` and the e2e test prose claimed the `.jxl` is stored under `$metadata['original']`. - The JXL e2e assertions read `media_details.original`, a field that is never written, so they returned `undefined` instead of exercising the preserved companion. Point all of them at `source_image`, which the controller actually writes and which `media_details` exposes (the raw attachment metadata). The bare `original` key is not used anywhere; the scaled/full passthrough uses `original_image` and the source-format companion uses `source_image`.
gutenberg_is_jxl_file() called fopen() without error suppression, so a missing or unreadable file emitted a PHP warning before the function returned false. Silence it with @fopen() and handle the false return, matching the core wp_is_jxl_file() backport.
# Conflicts: # lib/media/class-gutenberg-rest-attachments-controller.php # packages/upload-media/src/store/private-actions.ts # packages/upload-media/src/store/types.ts
# Conflicts: # lib/media/class-gutenberg-rest-attachments-controller.php # packages/upload-media/CHANGELOG.md # packages/upload-media/src/store/utils/index.ts
The file is created locally by husky install and is not tracked on trunk; a committed copy pointing at a machine-local setup breaks npm ci in CI.
Six defects found reviewing the JXL feature, each with a regression test: - prepareItem left `generate_sub_sizes` at its server-side default of true for JXL, so create_item built every sub-size itself and the client sideloaded none - silently reverting JXL uploads to server-side processing. - Detection keyed off `File.type` alone. Systems with no .jxl MIME mapping report an empty type, so the file was uploaded as a raw .jxl the server cannot read - which the new upload_mimes filter now lets through. - The decode failure used a bare 'JXL_DECODE_ERROR' string, absent from the ErrorCode enum and from getErrorMessage()'s table, so an unrecoverable failure surfaced as "Upload failed / Please try again". The message was also untranslated. - getVips() recorded "initialized with JXL" before the init resolved. A rejected init then left a permanently rejected vipsPromise that the reset guard could no longer clear, failing every later operation in the worker - including unrelated JPEG work. - getVips() discarded a live vips instance without shutting it down, stranding its WASM heap for the lifetime of the worker. - setJxlWasm() took a Uint8Array, which comctx does not recognise as transferable: it walked the ~3 MB array key by key on the calling thread and copied rather than transferred it. It now takes an ArrayBuffer. vipsEnsureJxlSupport() also no longer caches a failed download, and re-sends the bytes when it sees a replaced worker, so a recycle between prepareItem's ensure call and its conversion cannot strand a JXL on a worker that lacks the library.
|
Worth noting that Firefox and Chromium have both expressed their intent to ship: https://groups.google.com/a/mozilla.org/g/dev-platform/c/3YMV4MS34KA?pli=1 |
Chromium cannot decode JXL, so the image block's temporary blob preview fails to load and the block swaps the <img> for a spinner placeholder. The <img> only reappears once the upload finishes and the blob URL is replaced, which is beyond the default 5s expect timeout. Give the visibility assertion the same 30s window as the src assertion.
vips-worker.ts imports './worker-code.ts' with the extension, so a virtual mock registered for '../worker-code' is never consulted and jest tries to resolve the real file. That file is generated by a full build and gitignored, so the suite passed locally after a build but failed in CI.
Summary
Third experimental approach to #76981: lazy-load the 3 MB
vips-jxl.wasmmodule only when a user first processes a JXL image. No canonical plugin required, and no bundle-size hit on editor pages that never touch JXL.Sibling of:
vips-jxl.wasmdirectly (adds ~3 MB to every editor load).wp-vips-jxl) that ships the WASM on demand.This PR keeps the WASM inside Gutenberg but splits it into its own script module chunk (
@wordpress/vips/jxl-wasm) that the browser only fetches on first JXL use.Fixes #76981.
How it works
packages/vips/src/jxl-wasm.tsexports the base64 data URL forvips-jxl.wasm. Added towpScriptModuleExportsinpackages/vips/package.jsonso the build emits a standalonebuild/modules/vips/jxl-wasm.min.js(~3.0 MB).vipsEnsureJxlSupport()inpackages/vips/src/vips-worker.tsdynamic-imports@wordpress/vips/jxl-wasmand RPC-passes the URL to the worker viasetJxlWasmUrl(). The import and RPC are cached, so subsequent calls are no-ops.vips-jxl.wasmtodynamicLibrarieson the nextgetVips()call. If vips was already initialized without JXL, the existing instance is discarded and recreated with JXL support.locateFilereturns the URL when vips requestsvips-jxl.wasm.packages/upload-mediacallsvipsEnsureJxlSupport()fromprepareItemwhenever the input or output type isimage/jxl.image/jxlis added toCLIENT_SIDE_SUPPORTED_MIME_TYPES,'jxl'toVALID_IMAGE_FORMATSandImageFormat, and thevipsConvertImageFormatwrapper MIME union. JXL encoding useseffort=3(libvips default of 7 is too slow for interactive use).@wordpress/vips/workercorrectly declares a dynamicmodule_dependenciesentry on@wordpress/vips/jxl-wasm, so WordPress's import map resolves it at runtime.Screencast
jxl.to.jpeg.on.upload.mp4
Bundle size impact
Measured locally from
npm run build. Raw is the on-disk minified size; transferred (gzip) is what the browser actually downloads and what the CI size bot reports:build/modules/vips/worker.min.jsbuild/modules/vips/jxl-wasm.min.jsThe CI size bot's headline
+1.1 MB (+13.83%)is the gzipped JXL chunk, not the 3.0 MB raw figure — the actual network cost is smaller than the on-disk size. Editor sessions that never process a JXL image transfer no extra bytes (the worker grows by only 262 B). When a user uploads a JXL image, the browser fetches the separate chunk (~1.1 MB gzip) once and caches it.Comparison matrix
Test plan
npm run buildproducesbuild/modules/vips/worker.min.jsat ~13.1 MB and a separatebuild/modules/vips/jxl-wasm.min.jsat ~3.0 MB.module_dependenciesentry on@wordpress/vips/jxl-wasm.jxl-wasm.min.js..jxlfile: the browser fetchesjxl-wasm.min.jsonce, then processes the image client-side (resize, compress, thumbnails).Refs #76981.