Skip to content

Static image rendering for Swing and JavaFX components#123

Draft
mkhawam wants to merge 2 commits into
dflib:mainfrom
mkhawam:static-image-rendering
Draft

Static image rendering for Swing and JavaFX components#123
mkhawam wants to merge 2 commits into
dflib:mainfrom
mkhawam:static-image-rendering

Conversation

@mkhawam

@mkhawam mkhawam commented Jun 23, 2026

Copy link
Copy Markdown

Summary

The portable, JupyterLab-independent first slice of #122: render GUI components as static images via
display(...), for both Swing/AWT and JavaFX. No frontend, no comms, no extension — just images
(viewable anywhere a PNG is: GitHub, nbviewer, etc.).

What's included

  • java.awt.Component → PNG. Painted off-screen into a BufferedImage and encoded by the existing
    Image pipeline. Headless-native (no native window is created).
  • javafx.scene.Node → PNG. Snapshotted on the JavaFX Application Thread
    (Node.snapshot(...)SwingFXUtils.fromFXImageBufferedImage). Done reflectively, so
    jjava-jupyter keeps no compile- or runtime dependency on JavaFX — it's optional and user-supplied
    (e.g. %maven org.openjfx:javafx-controls:...). If the toolkit can't start, it degrades to a short text
    message instead of failing the cell.
  • Name-based renderer registration (Renderer.createRegistration(String) / registerByName). A
    user-supplied JavaFX node is loaded by the JShell class loader, not the kernel's, so Class-identity
    registration can't match it; matching by fully-qualified class name (against the value's class hierarchy)
    is class-loader agnostic. This is a small, generic addition useful for any optional runtime type.

So display(component) / display(node) (and a bare last-expression value) now produce a still image.

Headless note

JavaFX snapshot() needs the JavaFX toolkit, which needs a display. On a headless server/CI, run the kernel
under Xvfb (xvfb-run). (Monocle is not a viable route on JavaFX 21 — OpenJFX 21 doesn't ship
MonoclePlatformFactory.) The Swing renderer needs none of this.

Tests

Headless unit tests: SwingTest (Component → PNG of expected size), RendererTest (name-based dispatch),
JavaFxTest (graceful fallback when JavaFX is absent). Full jjava-jupyter module green.

Relation to #122

This is the portable core split out of #122 per the suggestion to digest it one piece at a time. The
interactive parts (comm-streamed frames + mouse/keyboard, and the JupyterLab extension) stay in #122 as a
draft for a later, separate PR.

Add renderers that turn GUI components into still images via display():

- java.awt.Component (Swing/AWT): painted off-screen into a BufferedImage and
  encoded by the existing Image pipeline. Headless-native.
- javafx.scene.Node (JavaFX): snapshotted on the JavaFX Application Thread and
  converted to a BufferedImage. Done reflectively so jjava-jupyter keeps no
  compile/runtime dependency on JavaFX (it is optional and user-supplied, e.g.
  via %maven). Producing a snapshot needs a display; on a headless host run the
  kernel under Xvfb, otherwise it degrades to a short message.

To match a JavaFX node regardless of which class loader loaded it, Renderer
gains name-based registration (createRegistration(String) / registerByName),
matched against the value's class hierarchy by name.
@mkhawam

mkhawam commented Jun 24, 2026

Copy link
Copy Markdown
Author

Another question is JavaFX requires JVM Options to run in headless mode. It a bit of a pain to add the -Dglass.platform=Monocle -Dmonocle.platform=Headless at the end of the kernal.json. It would also be a pain to have another installation process just for a user that wants to use JavaFX.

Although JavaFx does give smoother interaction then swing.

@andrus

andrus commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

JJAVA_JVM_OPTS to the rescue. I never really modify kernel.json in my own install.

Demonstrates display(component) / display(node) -> static PNG for Swing/AWT and
JavaFX (the latter optional via %maven, with an FX-thread build helper).
@mkhawam mkhawam force-pushed the static-image-rendering branch from a547bd1 to 6ef98ba Compare June 24, 2026 15:30
@mkhawam

mkhawam commented Jun 24, 2026

Copy link
Copy Markdown
Author

That means it will not work out of the box. Might want to it to the docs or outputting a warning if someone wants to use javafx if those options are not enabled?

@andrus

andrus commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Disclaimer: I need more time to get a better feel of how the whole things works to comment intelligently.

But maybe we make the whole thing an optional kernel extension that is loaded explicitly (by adding it as a dependency), and this is where it can complain about missing options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants