[FIXED] Open JavaFX child dialog from JFXPanel within JFrame

Issue

I'm trying to embed a third party JavaFX application within a larger Swing application. The requirement is that it behaves as if it is a non-modal child window. I've sort of got this working helped by How to open modal dialog from JFXPanel in JavaFX?.

However window ordering is not setup correctly. It is possible to place the child Stage behind the parent JFrame. I would not expect this with a child window.

Using xprop reveals in Ubuntu 16.04 that the X11 Atom WM_TRANSIENT_FOR is only set for the child JDialog, not the child Stage

Example - app that opens two child windows. One JavaFX and one Swing. The Swing one is correctly parented. The JavaFX one is not.

public class SwingApp {

    public static void main(String[] args) throws Exception {
        JFrame parent = new JFrame();
        parent.setTitle("Parent JFrame");
        parent.setSize(200, 150);
        JFXPanel jfxPanel = new JFXPanel();
        parent.getContentPane().setLayout(new BoxLayout(parent.getContentPane(), BoxLayout.Y_AXIS));
        JButton button = new JButton("Open Swing child");
        button.addActionListener(e -> {
            JDialog child = new JDialog(parent);
            child.setModal(false);
            child.getContentPane().add(new JLabel("content"));
            child.setVisible(true);
        });
        parent.getContentPane().add(button);
        parent.getContentPane().add(jfxPanel);
        parent.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
        parent.setVisible(true);

        Platform.runLater(() -> jfxPanel.setScene(new Scene(createDummyFxApp(jfxPanel))));
    }

    private static Parent createDummyFxApp(JFXPanel openingPanel) {
        Button button = new Button("Open FX child");
        button.setOnAction(e -> {
            Stage stage = new Stage();
            stage.initModality(Modality.NONE);

            Window owner = openingPanel.getScene().getWindow();
            stage.initOwner(owner);

            stage.setTitle("Non-modal child JavaFX window");
            stage.setScene(new Scene(new HBox(new Label("content"))));
            stage.show();
        });

        return new HBox(button);
    }
}

Solution

I eventually achieved this with JNA. Firstly find the windows by title.

public List<Window> find(Pattern title) {
    Display display = x11.XOpenDisplay(null);
    Window root = x11.XDefaultRootWindow(display);
    List<Window> windows = recurse(x11, display, root, title);
    x11.XCloseDisplay(display);
    return windows;
}

private synchronized List<Window> recurse(X11 x11, Display display, Window root, Pattern pattern) {
    List<Window> windows = new ArrayList<>(1);
    X11.WindowByReference windowRef = new X11.WindowByReference();
    X11.WindowByReference parentRef = new X11.WindowByReference();
    PointerByReference childrenRef = new PointerByReference();
    IntByReference childCountRef = new IntByReference();

    x11.XQueryTree(display, root, windowRef, parentRef, childrenRef, childCountRef);
    if (childrenRef.getValue() == null) {
        return Collections.emptyList();
    }

    long[] ids;

    if (Native.LONG_SIZE == Long.BYTES) {
        ids = childrenRef.getValue().getLongArray(0, childCountRef.getValue());
    } else if (Native.LONG_SIZE == Integer.BYTES) {
        int[] intIds = childrenRef.getValue().getIntArray(0, childCountRef.getValue());
        ids = new long[intIds.length];
        for (int i = 0; i < intIds.length; i++) {
            ids[i] = intIds[i];
        }
    } else {
        throw new IllegalStateException("Unexpected size for Native.LONG_SIZE" + Native.LONG_SIZE);
    }

    for (long id : ids) {
        Window child = new Window(id);
        X11.XTextProperty name = new X11.XTextProperty();
        int result = x11.XGetWMName(display, child, name);
        String value = name.value;
        LOGGER.info(String.format("Found window %s result: %d free %s", value, result, name));

        if (value != null && pattern.matcher(value).matches()) {
            windows.add(child);
        }
        windows.addAll(recurse(x11, display, child, pattern));
    }
    return windows;
}

Then, "adopt" the window by adding the transient for atom

public void adopt(Window child, Window parent) {
    Display display = x11.XOpenDisplay(null);

    Atom wmState = x11.XInternAtom(display, "_NET_WM_STATE", false);
    Atom wmStateModal = x11.XInternAtom(display, "_NET_WM_STATE_MODAL", false);
    x11.XSetTransientForHint(display, child, parent);
    addAtom(display, child, wmState, wmStateModal);

    x11.XCloseDisplay(display);

}

Using this support method

public void addAtom(Display display, Window win, Atom key, Atom value) {

    int maskVal = X11.SubstructureRedirectMask | X11.SubstructureNotifyMask;
    NativeLong mask = new NativeLong(maskVal);

    XClientMessageEvent event = new X11.XClientMessageEvent();
    event.type = X11.ClientMessage;
    event.serial = new NativeLong(0);
    event.send_event = 1;
    event.message_type = key;
    event.window = win;
    event.format = 32;
    event.data.setType(NativeLong[].class);
    event.data.l[0] = new NativeLong(NET_WM_STATE_ADD);
    event.data.l[1] = value;
    event.data.l[2] = new NativeLong(0);
    event.data.l[3] = new NativeLong(0);
    event.data.l[4] = new NativeLong(0);

    X11.XEvent e = new X11.XEvent();
    e.setTypedValue(event);

    x11.XSendEvent(display, x11.XDefaultRootWindow(display), 0, mask, e);
    x11.XFlush(display);

}

Also had to add an error handler to X11, otherwise the default error handler crashes the VM, this happens when trying to get information on windows which have disappeared since original discovery.

   x11.XSetErrorHandler(new XErrorHandler() {
        @Override
        public int apply(Display display, XErrorEvent errorEvent) {
            LOGGER.warn(String.format("X11 error during JNA, ignore for now: %s", errorEvent.toString()));
            return 0;
        }
    });

Answered By - Adam