Skip to content

Add JPEG XL support, loaded on demand - #77584

Draft
adamsilverstein wants to merge 25 commits into
trunkfrom
try/jxl-lazy-load
Draft

Add JPEG XL support, loaded on demand#77584
adamsilverstein wants to merge 25 commits into
trunkfrom
try/jxl-lazy-load

Conversation

@adamsilverstein

@adamsilverstein adamsilverstein commented Apr 23, 2026

Copy link
Copy Markdown
Member

Summary

Third experimental approach to #76981: lazy-load the 3 MB vips-jxl.wasm module only when a user first processes a JXL image. No canonical plugin required, and no bundle-size hit on editor pages that never touch JXL.

Sibling of:

This PR keeps the WASM inside Gutenberg but splits it into its own script module chunk (@wordpress/vips/jxl-wasm) that the browser only fetches on first JXL use.

Fixes #76981.

How it works

  1. New tiny script module: packages/vips/src/jxl-wasm.ts exports the base64 data URL for vips-jxl.wasm. Added to wpScriptModuleExports in packages/vips/package.json so the build emits a standalone build/modules/vips/jxl-wasm.min.js (~3.0 MB).
  2. Main-thread helper: vipsEnsureJxlSupport() in packages/vips/src/vips-worker.ts dynamic-imports @wordpress/vips/jxl-wasm and RPC-passes the URL to the worker via setJxlWasmUrl(). The import and RPC are cached, so subsequent calls are no-ops.
  3. Worker-side reinit: The worker stores the URL and adds vips-jxl.wasm to dynamicLibraries on the next getVips() call. If vips was already initialized without JXL, the existing instance is discarded and recreated with JXL support. locateFile returns the URL when vips requests vips-jxl.wasm.
  4. Upload trigger: packages/upload-media calls vipsEnsureJxlSupport() from prepareItem whenever the input or output type is image/jxl. image/jxl is added to CLIENT_SIDE_SUPPORTED_MIME_TYPES, 'jxl' to VALID_IMAGE_FORMATS and ImageFormat, and the vipsConvertImageFormat wrapper MIME union. JXL encoding uses effort=3 (libvips default of 7 is too slow for interactive use).
  5. Dynamic dependency: The module asset file for @wordpress/vips/worker correctly declares a dynamic module_dependencies entry on @wordpress/vips/jxl-wasm, so WordPress's import map resolves it at runtime.

Screencast

jxl.to.jpeg.on.upload.mp4

Bundle size impact

Measured locally from npm run build. Raw is the on-disk minified size; transferred (gzip) is what the browser actually downloads and what the CI size bot reports:

Artifact Raw (minified) Transferred (gzip) When loaded
build/modules/vips/worker.min.js 13,752,457 B (~13.1 MB) ~4.56 MB Always — unchanged vs trunk (+262 B)
build/modules/vips/jxl-wasm.min.js 3,109,421 B (~3.0 MB) ~1.1 MB On-demand — first JXL use only

The CI size bot's headline +1.1 MB (+13.83%) is the gzipped JXL chunk, not the 3.0 MB raw figure — the actual network cost is smaller than the on-disk size. Editor sessions that never process a JXL image transfer no extra bytes (the worker grows by only 262 B). When a user uploads a JXL image, the browser fetches the separate chunk (~1.1 MB gzip) once and caches it.

The chunk is the vips-jxl.wasm dynamic library (2.22 MB raw / 0.77 MB gzip) inlined as a base64 data URL inside a JS module. Base64 inlining adds ~33% on disk and ~0.33 MB to the gzip transfer (1.1 MB vs the 0.77 MB the raw .wasm would gzip to) — see the discussion below on lighter-weight alternatives.

Comparison matrix

Aspect Trunk #77570 (bundled) #76990 (plugin) This PR (lazy chunk)
Editor worker size 13.1 MB 16.1 MB 13.1 MB 13.1 MB
JXL available out of the box No Yes After install Yes
Plugin install flow None Required for non-admins None
JXL WASM hosted by Gutenberg Plugin Gutenberg
WASM downloaded only on JXL use No Yes (per plugin) Yes
Independent versioning of JXL WASM No Yes No
Graceful fallback for users who can't install N/A Yes (server-side if supported) N/A

Test plan

  • npm run build produces build/modules/vips/worker.min.js at ~13.1 MB and a separate build/modules/vips/jxl-wasm.min.js at ~3.0 MB.
  • Verify the worker asset file declares a dynamic module_dependencies entry on @wordpress/vips/jxl-wasm.
  • Load the block editor without uploading JXL: DevTools Network tab should show no request for jxl-wasm.min.js.
  • Upload a .jxl file: the browser fetches jxl-wasm.min.js once, then processes the image client-side (resize, compress, thumbnails).
  • Upload a JXL, then a JPEG in the same session: JXL-initialized vips instance handles both.
  • Configure JXL as an output format (e.g., JPEG → JXL) and verify transcoding works.
  • Verify existing image formats (JPEG, PNG, WebP, AVIF, GIF, HEIC) still work unchanged.
  • Confirm size bot report aligns with locally measured numbers.

Refs #76981.

Add client-side JPEG XL support without growing the vips worker
bundle. The 3 MB vips-jxl.wasm module is split into its own
@wordpress/vips/jxl-wasm script module that is only fetched the
first time a JXL image is processed.

How it works:
- packages/vips/src/jxl-wasm.ts is a tiny new script module whose
  only job is to export the base64 data URL for vips-jxl.wasm.
- @wordpress/vips/jxl-wasm is registered as a new wpScriptModuleExports
  entry so the build emits build/modules/vips/jxl-wasm.min.js (~3 MB)
  as a separately loadable module.
- vips-worker.ts adds vipsEnsureJxlSupport(), which on first call
  dynamic-imports '@wordpress/vips/jxl-wasm' to get the data URL,
  then RPC-passes it to the worker via setJxlWasmUrl().
- The worker (packages/vips/src/index.ts) stores the URL, adds
  vips-jxl.wasm to dynamicLibraries on the next getVips() call, and
  returns the URL from locateFile(). If vips was already initialized
  without JXL, the existing instance is discarded so the reinit picks
  up JXL support.
- packages/upload-media calls vipsEnsureJxlSupport() from prepareItem
  whenever the input or output type is image/jxl. image/jxl is added
  to CLIENT_SIDE_SUPPORTED_MIME_TYPES and 'jxl' to VALID_IMAGE_FORMATS
  and the ImageFormat type. The vipsConvertImageFormat wrapper MIME
  union is widened accordingly.
- JXL encoding uses effort=3 (libvips default 7 is too slow for
  interactive use).

Size impact:
- worker.min.js: unchanged (~13.1 MB, same as trunk).
- jxl-wasm.min.js: new separate module (~3.0 MB), fetched only when
  a JXL image is encountered.

Alternative to #77570, which bundles vips-jxl.wasm directly into the
worker (+3 MB on every editor page load). Opened so the size bot can
compare.

Refs #76981.
@github-actions

github-actions Bot commented Apr 23, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: swissspidy <swissspidy@git.wordpress.org>
Co-authored-by: gregbenz <gregbenz@git.wordpress.org>
Co-authored-by: andrewserong <andrewserong@git.wordpress.org>
Co-authored-by: BlackStar1991 <blackstar1991@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@github-actions

github-actions Bot commented Apr 23, 2026

Copy link
Copy Markdown

Size Change: +850 kB (+10.74%) ⚠️

Total Size: 8.76 MB

📦 View Changed
Filename Size Change
build/modules/vips/jxl-wasm.min.js 850 kB +850 kB (new file) 🆕
build/modules/vips/worker.min.js 3.69 MB +478 B (+0.01%)
build/scripts/upload-media/index.min.js 16.5 kB +237 B (+1.45%)

compressed-size-action

@github-actions

github-actions Bot commented Apr 23, 2026

Copy link
Copy Markdown

Flaky tests detected in 4ce3bac.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/33117672777
📝 Reported tests:

Navigates the items list via UP/DOWN arrow keys in /test/e2e/specs/site-editor/dataviews-list-layout-keyboard.spec.js, passed after 1 failed attempt.
Error: expect(locator).toBeFocused() failed

Locator:  getByLabel('Page Two')
Expected: focused
Received: inactive
Timeout:  5000ms

Call log:
  - Expect "toBeFocused" with timeout 5000ms
  - waiting for getByLabel('Page Two')
    14 × locator resolved to <button type="button" tabindex="-1" aria-pressed="false" id="view-list-0-230-item-wrapper" class="dataviews-view-list__item" aria-labelledby="view-list-0-230-label" aria-describedby="view-list-0-230-description"></button>
       - unexpected value "inactive"

    at /home/runner/work/gutenberg/gutenberg/test/e2e/specs/site-editor/dataviews-list-layout-keyboard.spec.js:112:49

Resolve conflict in packages/upload-media/src/store/private-actions.ts:
trunk moved image format transcoding from client-side prepareItem to
server-driven thumbnail generation. Adopt the new pipeline and keep
the JXL WASM lazy-load, but trigger it in two places: prepareItem for
JXL input files, and transcodeImageItem for server-requested JXL
output.
@adamsilverstein adamsilverstein self-assigned this Apr 28, 2026
@adamsilverstein
adamsilverstein requested review from andrewserong and youknowriad and removed request for youknowriad April 28, 2026 17:43
@adamsilverstein adamsilverstein added [Status] In Progress Tracking issues with work in progress [Type] Feature New feature to highlight in changelogs. [Feature] Client Side Media Media processing in the browser with WASM labels May 18, 2026
@adamsilverstein

Copy link
Copy Markdown
Member Author

A few notes from a review pass:

Bundle size / lighter-weight JXL options (re #76981)

  • The size bot reports gzip, so the headline +1.1 MB is the gzipped chunk, not the 3 MB raw figure. Updated the description's table to show both.
  • vips-jxl.wasm is 2.22 MB raw / 0.77 MB gzip. Base64-inlining it into a JS module bumps the gzip transfer to ~1.1 MB (+~0.33 MB). Cheapest same-architecture trim would be serving the raw .wasm instead of base64 — though I know we inline deliberately to avoid host MIME/path issues, so that's a tradeoff.
  • mediabunny doesn't help here — it's a video/audio toolkit, no still-image/JXL path.
  • jSquash @jsquash/jxl (standalone libjxl) is smaller — enc 0.49 MB / dec 0.30 MB gzip — but it's a second WASM runtime. We'd lose the single libvips pipeline (decode → resize → thumbnails → metadata → encode) and have to shuttle raw pixels between two heaps. Not worth it unless JXL were the only format we processed. The 3 MB is inherent to libjxl regardless of wrapper; the real win is keeping it lazy, which this PR does.
  • No browser encodes JXL (and WebCodecs has none), so a WASM encoder is unavoidable for now — which favors lazy-load over bundling.

Minor code note

  • In getVips(), the JXL reinit path (vipsPromise = undefined) orphans the previously-initialized vips instance without an explicit teardown (cleanup is the Emscripten auto-delete delay fn, not a heap teardown). It's bounded — only fires when a non-JXL image is processed before a JXL one in the same session, and only once — but given the prior OOM history (Investigate and fix crashes and console errors during client-side image upload processing #76706) a brief comment acknowledging it would help.

@adamsilverstein
adamsilverstein requested a review from ramonjd May 21, 2026 17:05
JXL is not broadly web-compatible: most browsers (including Chrome) cannot
display it and the server cannot read it (GD/Imagick have no JXL decoder, and
fileinfo reports it as image/x-jxl). Uploading JXL as-is was rejected outright
— by the editor's allowed-MIME check and by core's wp_check_filetype_and_ext()
— and even when allowed produced an undisplayable attachment with no
dimensions or sub-sizes.

Decode JXL to JPEG client-side with vips (the JXL WASM module is already
lazy-loaded on demand) and upload the JPEG, mirroring how HEIC is handled. The
original .jxl is preserved as a companion file in $metadata['original'] so no
data is lost. The editor and front end now use the portable JPEG, with real
dimensions and the full set of sub-sizes.

- Register image/jxl as an allowed upload MIME type, and restore the type
  during validation via a magic-byte-checked wp_check_filetype_and_ext filter
  so the sideloaded original passes despite the image/x-jxl finfo mismatch.
- Add original-jxl sideload handling to the REST controller, skipping the
  dimension read since JXL cannot be measured server-side.
- Generalize the HEIC companion delete hook to clean up JXL originals too.
- Add an e2e test and JXL asset covering the conversion and companion.
# Conflicts:
#	packages/upload-media/src/store/private-actions.ts
@adamsilverstein

Copy link
Copy Markdown
Member Author

I did some testing on this feature, made some small fixes and got it working, it now properly handles uploaded JXL files.

I created a JXL using Squoosh for testing:
Cliff-Palace.jxl.zip

Because JPEG XL (JXL) is not supported in Chromium where client-side media is active, and generally to provide a web-safe format for users, I decided to treat JXL uploads similar to HEIC uploads.

So we do the following:

  1. sideload the original JXL upload so it is preserved for the user
  2. output encode as JPEG for all display (sub) sizes, jpeg gets used on the front end/srcset and editor views.

The output format should also honor the image_output_formats filter, so JXL output will still be possible, just not the default.

See screencast:

jxl.to.jpeg.on.upload.mp4

@adamsilverstein

Copy link
Copy Markdown
Member Author

I will work on a core backport once the core client-side media feature is restored.

# Conflicts:
#	packages/vips/CHANGELOG.md
#	packages/vips/README.md
@adamsilverstein adamsilverstein changed the title Client-side media: Lazy-load JPEG XL (JXL) WASM on demand Add JPEG XL support, loaded on demand May 28, 2026
Guard the contract that high-bit-depth (>8-bit) and gain-map JXL uploads
flatten to 8-bit JPEG sub-sizes while the full-fidelity original is
preserved byte-for-byte as the .jxl companion file.

Add two 200x150 fixtures (a genuine 16-bit JXL and an 8-bit JXL carrying
an ISO 21496-1 jhgm gain-map box) plus a self-contained generator
script. Both decode in wasm-vips to a JPEG derivative; the tests assert
the JPEG main + sub-size and verify the stored original is identical to
the upload, proving the bit depth and gain map survive in the original.
# Conflicts:
#	packages/upload-media/CHANGELOG.md
#	packages/upload-media/src/store/types.ts
# Conflicts:
#	packages/upload-media/CHANGELOG.md
#	packages/upload-media/src/store/private-actions.ts
#	packages/vips/src/worker.ts
#	test/e2e/specs/editor/various/client-side-media-processing.spec.js
The .gen-jxl-fidelity-fixtures.mjs helper failed lint: prettier wanted
the over-length magick/cjxl argument arrays wrapped one-per-line, and the
file-level eslint-disable no-console had no matching eslint-enable. Wrap
the arrays and add the closing directive.
# Conflicts:
#	lib/media/class-gutenberg-rest-attachments-controller.php
#	lib/media/load.php
#	packages/upload-media/CHANGELOG.md
@adamsilverstein

Copy link
Copy Markdown
Member Author

@andrewserong & @swissspidy - this is ready for review.

My main concern with this feature is the added bundle size weight from adding JXL support, especially given the current lack of browser support, see https://caniuse.com/jpegxl. Perhaps it would be best to leave this off for now, or consider making it plugin-only (that is, available in the Gutenberg plugin, but not merged to core) or even an Experiment in Gutenberg so we can easily remove it later without breaking commitments.

I acknowledge that if we did choose to add JXL support this would be a signal to browsers that they should add support, but I'm hesitant to merge it into core for now. given the current state of browser support and likely very low usage of the format in the wild.

Resolve conflict in packages/vips/src/index.ts with trunk's #79188, which
switched inlined WASM from base64 data URLs to a Uint8Array wrapped in a
Blob URL on demand (getWasmUrl helper).

Align the JXL lazy-load path with the new scheme: the vips-jxl.wasm module
now resolves to a Uint8Array, so pass those bytes across the worker RPC and
wrap them in a Blob URL inside the worker, exactly like the HEIF library.
Rename setJxlWasmUrl -> setJxlWasm (and vipsSetJxlWasmUrl -> vipsSetJxlWasm)
since the value is bytes, not a URL, updating the worker wiring, README, and
CHANGELOG to match.
@adamsilverstein

Copy link
Copy Markdown
Member Author

My main concern with this feature is the added bundle size weight from adding JXL support, especially given the current lack of browser support, see https://caniuse.com/jpegxl. Perhaps it would be best to leave this off for now, or consider making it plugin-only (that is, available in the Gutenberg plugin, but not merged to core) or even an Experiment in Gutenberg so we can easily remove it later without breaking commitments.

@swissspidy / @andrewserong / @gregbenz - any objection to punting JXL support to a later release if/when it gets better stronger support? I'm not sure its worth the increased payload size (11% according to the size bot: #77584 (comment))

@swissspidy

Copy link
Copy Markdown
Member

Sounds good to me 👍

@gregbenz

Copy link
Copy Markdown

@adamsilverstein I think that's fine and probably ideal at this time to manage risk and focus on landing what's already a significant change.

It's definitely something to keep on the radar and test when appropriate, but I thiink it is early to deploy. https://caniuse.com/jpegxl puts support at 14% (Safari). Chromium works under chrome://flags/#enable-jxl-image-format and FireFox Nightly has it, so this can grow rapidly. But it isn't a format which will be safe to use on the web for a bit.

JXL is an excellent format and would love to see it added not long after Chromium / FF start to propagate with default support. I don't see 11% size increase for a cached WASM used only for admins as a concern from my perspective, just not valuable yet given state of the ecosystem.

Do we have any benchmark data on performance transcoding to JXL vs AVIF? I expect that may be a benefit (no need to test anything now if the data is not readily available for a wasm-vips JXL scenario).

@andrewserong

Copy link
Copy Markdown
Contributor

Punting for now sounds good to me, too 👍

@adamsilverstein
adamsilverstein marked this pull request as draft June 24, 2026 19:50
@adamsilverstein

Copy link
Copy Markdown
Member Author

Thanks for the feedback...

I have converted the PR to draft to indicate we aren't planning to land it currently, I will comment on the issue as well.

…inal

The JXL source-format companion (the preserved `.jxl`) is stored under the
`source_image` attachment-metadata key (`META_KEY_SOURCE_IMAGE`), the same
key the HEIC original uses, not a bare `original` key. Two stale references
still pointed at `original`:

- A comment in `private-actions.ts` and the e2e test prose claimed the
  `.jxl` is stored under `$metadata['original']`.
- The JXL e2e assertions read `media_details.original`, a field that is
  never written, so they returned `undefined` instead of exercising the
  preserved companion.

Point all of them at `source_image`, which the controller actually writes
and which `media_details` exposes (the raw attachment metadata). The bare
`original` key is not used anywhere; the scaled/full passthrough uses
`original_image` and the source-format companion uses `source_image`.
gutenberg_is_jxl_file() called fopen() without error suppression, so a
missing or unreadable file emitted a PHP warning before the function
returned false. Silence it with @fopen() and handle the false return,
matching the core wp_is_jxl_file() backport.
@adamsilverstein adamsilverstein removed the [Status] In Progress Tracking issues with work in progress label Jun 27, 2026
# Conflicts:
#	lib/media/class-gutenberg-rest-attachments-controller.php
#	packages/upload-media/src/store/private-actions.ts
#	packages/upload-media/src/store/types.ts
# Conflicts:
#	lib/media/class-gutenberg-rest-attachments-controller.php
#	packages/upload-media/CHANGELOG.md
#	packages/upload-media/src/store/utils/index.ts
The file is created locally by husky install and is not tracked on trunk;
a committed copy pointing at a machine-local setup breaks npm ci in CI.
Six defects found reviewing the JXL feature, each with a regression test:

- prepareItem left `generate_sub_sizes` at its server-side default of true
  for JXL, so create_item built every sub-size itself and the client
  sideloaded none - silently reverting JXL uploads to server-side
  processing.
- Detection keyed off `File.type` alone. Systems with no .jxl MIME mapping
  report an empty type, so the file was uploaded as a raw .jxl the server
  cannot read - which the new upload_mimes filter now lets through.
- The decode failure used a bare 'JXL_DECODE_ERROR' string, absent from
  the ErrorCode enum and from getErrorMessage()'s table, so an
  unrecoverable failure surfaced as "Upload failed / Please try again".
  The message was also untranslated.
- getVips() recorded "initialized with JXL" before the init resolved. A
  rejected init then left a permanently rejected vipsPromise that the
  reset guard could no longer clear, failing every later operation in the
  worker - including unrelated JPEG work.
- getVips() discarded a live vips instance without shutting it down,
  stranding its WASM heap for the lifetime of the worker.
- setJxlWasm() took a Uint8Array, which comctx does not recognise as
  transferable: it walked the ~3 MB array key by key on the calling
  thread and copied rather than transferred it. It now takes an
  ArrayBuffer.

vipsEnsureJxlSupport() also no longer caches a failed download, and
re-sends the bytes when it sees a replaced worker, so a recycle between
prepareItem's ensure call and its conversion cannot strand a JXL on a
worker that lacks the library.
@adamsilverstein

Copy link
Copy Markdown
Member Author

Worth noting that Firefox and Chromium have both expressed their intent to ship:

https://groups.google.com/a/mozilla.org/g/dev-platform/c/3YMV4MS34KA?pli=1
https://groups.google.com/a/chromium.org/g/blink-dev/c/-gDojQbDPRI

adamsilverstein and others added 4 commits August 25, 2026 22:24
Chromium cannot decode JXL, so the image block's temporary blob preview
fails to load and the block swaps the <img> for a spinner placeholder.
The <img> only reappears once the upload finishes and the blob URL is
replaced, which is beyond the default 5s expect timeout. Give the
visibility assertion the same 30s window as the src assertion.
vips-worker.ts imports './worker-code.ts' with the extension, so a
virtual mock registered for '../worker-code' is never consulted and jest
tries to resolve the real file. That file is generated by a full build
and gitignored, so the suite passed locally after a build but failed in
CI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Feature] Client Side Media Media processing in the browser with WASM [Type] Feature New feature to highlight in changelogs.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add JPEG XL (JXL) support

4 participants

Sponsor
SponsoredKunjungi sekarang
Promo