[FIXED] How can unit tests await the shutdown of the JavaFX Platform?

Issue

The following test suite fails on the second test, because the JavaFX platform has not properly shut down after the first test. How can I await the termination of the platform?

Ideally, I'm looking for a similar solution as the one that the JFX uses internally, using a shut down hook (example below failing test cases).

Simplified failing test code:

package sample;

import javafx.application.Platform;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import java.util.concurrent.CountDownLatch;

import static java.util.concurrent.TimeUnit.SECONDS;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;

@DisplayName( "the Java FX Platform should" )
public class JavaFXSuite
{
    @BeforeEach
    void startJFX() throws InterruptedException
    {
        final var latch = new CountDownLatch( 1 );
        Platform.startup( latch::countDown );
        assertTrue( latch.await( 1, SECONDS ) );
    }

    @AfterEach
    void stopJFX()
    {
//      final var latch = new CountDownLatch( 1 );
//      Platform.exit( latch::countDown ); //<-- won't compile: doesn't exist
//      assertTrue( latch.await( 1, SECONDS ) );

        // Platform.exit() just sets a boolean. Next test starts before platform had been exited properly
        Platform.exit();
    }

    @Test
    @DisplayName( "create its own application thread" )
    void firstTestPassesFine()
    {
        assertFalse( Platform.isFxApplicationThread() );
    }

    @Test
    @DisplayName( "not mess with tests that don't use the platform at all" )
    void supposedlyOrthogonalSecondTestFailsOnSetUp()
    {
        assertThat( 2, is( 2 ) );
    }
}

Second test fails in the set up with the output:

java.lang.IllegalStateException: Platform.exit has been called

    at javafx.graphics/com.sun.javafx.application.PlatformImpl.startup(PlatformImpl.java:179)
    at javafx.graphics/javafx.application.Platform.startup(Platform.java:111)
    at JavaFXBla/sample.JavaFXSuite.startJFX(JavaFXSuite.java:26)
    (...etc)

Digging through JavaFX internal code, I found the PlatformImpl.FinishListener. I tried to use this using a loan pattern. This fails (at runtime!), because the internal API isn't, and of course shouldn't be, exposed via its module.

Loan pattern test fixture:

package sample;

import com.sun.javafx.application.PlatformImpl;
import javafx.application.Platform;
import lombok.NonNull;
import lombok.extern.log4j.Log4j2;

import java.util.concurrent.CountDownLatch;

import static com.sun.javafx.application.PlatformImpl.addListener;

@Log4j2
public class JavaFXSuite
{
    private static final Object lock = new Object();

    protected void runningJavaFX( @NonNull final Runnable test ) throws InterruptedException
    {
        synchronized( lock )
        {
            final var latch = addShutdownHook();

            log.info( "entered" );
            test.run();

            Platform.exit();
            latch.await();
            log.info( "exited" );
        }
    }

    private CountDownLatch addShutdownHook()
    {
        final var latch = new CountDownLatch( 1 );
        addListener( new PlatformImpl.FinishListener()
        {
            @Override
            public void idle( boolean implicitExit )
            {
                latch.countDown();
            }

            @Override
            public void exitCalled()
            {
                latch.countDown();
            }
        } );

        return latch;
    }
}

sample failing runtime output:

java.lang.IllegalAccessError: superinterface check failed: class sample.JavaFXSuite$1 (in module JavaFXBla) cannot access class com.sun.javafx.application.PlatformImpl$FinishListener (in module javafx.graphics) because module javafx.graphics does not export com.sun.javafx.application to module JavaFXBla

    at JavaFXBla/sample.JavaFXSuite.addShutdownHook(JavaFXSuite.java:36)
    (...etc)

NOTE: this is a simplified example. The actual tested code (also) start the JFX runtime via the Application.launch()

Solution

So, after some more experimenting, the error persists even when using JavaFX own shutdown listening mechanism (see code below).

This makes it very likely that @James_D their interpretation is correct: One can only start JavaFX once per JVM instance.

It is therefor not possible to orthogonally run unit tests on JavaFX by starting a private JavaFX instance per test.

Counterexample:

package sample;

import javafx.application.Application;
import javafx.application.Platform;
import javafx.stage.Stage;
import lombok.extern.log4j.Log4j2;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import java.util.concurrent.CountDownLatch;

import static java.util.concurrent.TimeUnit.SECONDS;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;

@DisplayName( "the Java FX Platform should" )
@Log4j2
public class JavaFXSuite
{
    // static, because that's the only way to communicate with the created application
    private static CountDownLatch startupLatch;
    private CountDownLatch shutdownLatch;

    @BeforeEach
    @DisplayName( "while started" )
    void startJFX() throws InterruptedException
    {
        startupLatch = new CountDownLatch( 1 );

        // start the JFX the 'normal' way, by letting JFX hijack the calling thread
        // and then decrementing the latch on returning
        shutdownLatch = new CountDownLatch( 1 );
        new Thread( () ->
        {
            Application.launch( Bla.class );
            shutdownLatch.countDown();
        } ).start();
        assertTrue( startupLatch.await( 1, SECONDS ) );
    }

    public static class Bla
            extends Application
    {
        public Bla(){}

        @Override
        public void start( Stage primaryStage )
        {
            startupLatch.countDown();
        }
    }

    @AfterEach
    @DisplayName( "until halted" )
    void stopJFX() throws InterruptedException
    {
        Platform.exit();
        assertTrue( shutdownLatch.await( 1, SECONDS ) );
    }

    @Test
    @DisplayName( "create its own application thread" )
    void firstTestPassesFine()
    {
        assertFalse( Platform.isFxApplicationThread() );
    }

    @Test
    @DisplayName( "not mess with tests that don't use the platform at all" )
    void supposedlyOrthogonalSecondTestFailsOnSetUp()
    {
        assertThat( 2, is( 2 ) );
    }
}

Second test failure message:

Exception in thread "Thread-4" java.lang.IllegalStateException: Application launch must not be called more than once
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:175)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:156)
    at javafx.graphics/javafx.application.Application.launch(Application.java:233)
    at JavaFXBla/sample.JavaFXSuite.lambda$startJFX$0(JavaFXSuite.java:40)
    at java.base/java.lang.Thread.run(Thread.java:831)

Answered By - Joost Papendorp
Answer Checked By - Robin (JavaFixing Admin)