[FIXED] Testcontainers, try to load from local registry before build a Dockerfile

Issue

I'm developing some test cases using Tescontainers with Spring-Boot in order to get up a MS-SQL dockerized db. It's a huge db that takes about 40 minutes to be restored on the docker run proccess.

The steps I do to work with this image are:

1. Build Dockerfile with schema and data scripts tagging it as "db".
2. Run the container and wait about 40 minutes for database restoring.
3. Commit the container with "db-ready" tag.

The behavior I expect is than the test case try to run a cointainer from "db-ready" image and, if it fails, build then the image directly from Dockerfile. The code I tried looks like:

public static CustomMSSqlContainer getInstance() {
    if (container == null) {
        try {
            container = new CustomMSSqlContainer("myproject:db-ready");
        }catch(Exception ex) {              
            container = new CustomMSSqlContainer(new ImageFromDockerfile().withFileFromClasspath("Dockerfile", "docker/Dockerfile")
                        .withFileFromClasspath("schema.sql", "docker/schema.sql")
                        .withFileFromClasspath("entrypoint.sh", "docker/entrypoint.sh")
                        .withFileFromClasspath("data.sql", "docker/data.sql")
                        .withFileFromClasspath("data-init.sql", "docker/data-init.sql")
                        .withFileFromClasspath("start.sh", "docker/start.sh"));
        }
            
        container.waitingFor(Wait.forLogMessage("Database ready\\n", 1)
                    .withStartupTimeout(Duration.ofHours(1)))
                    .withExposedPorts(1433);
    }
        
    return (CustomMSSqlContainer)container;
}

Of course, this code doesn't works like I expect.

Any suggestions?

Solution

How we solved this

The way we do this is by building a custom image only on the Main/Dev branch. That way:

• We don't need to have the try-catch;
• We ONLY build a new container when it's actually necessary (after changes have been approved by merge request and merged into the main branch);
• Building of this container is done only on the CI pipeline (so people don't have to and can't even randomly push to the container registry)

This is an example using a JUnit test (disabled in this example, but you could use Spring Profiles to enable it):

@Test
@Disabled("Should be run only with certain profiles ;)")
public void pushNewImage() throws InterruptedException {
    // Startup the container before this point, using @Container or just container.start().
    // That should run all your scripts and wait for the waiter 

    // Get the DockerClient used by the TestContainer library (you can also use your own if they every make that private).
    final DockerClient dockerClient = customMSSqlContainer.getDockerClient();

    // Commit docker container changes into new image
    dockerClient.commitCmd(customMSSqlContainer.getContainerId())
            .withRepository("myproject")
            .withTag("db-ready")
            .exec();

    // Push new image. Logger is used for visual feedback.
    dockerClient.pushImageCmd("myproject:db-ready")
            .exec(new ResultCallback.Adapter<>() {
                @Override
                public void onNext(PushResponseItem object) {
                    log.info(object.toString()); // just log the push to the repo
                }
            }).awaitCompletion();
}

Potential pitfall with this approach

docker commit will not save anything that is saved into a volume. This is a problem, as most database images will in fact create a volume. I can't see your Dockerfile, but make sure that all data you are saving is not saved into a volume!

Read more

I've shortly talked about this at JFokus conference recently, with a person from the TestContainers core team in my room: https://youtu.be/pxxMnvu52K8?t=1922

Almost done writing a blog post on this topic, will update this answer when it's live

Answered By - Tom Cools
Answer Checked By - Terry (JavaFixing Volunteer)