[FIXED] Did Apache Flink drop the StreamOperatorTestHarness classes, or did they move to a different artifact?

Issue

I am using intellij, maven 3, and flink 1.15.1 to write a stateful streaming job. I am trying to write unit tests for my custom KeyedProcessFunction and tried following the documentation href="https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/datastream/testing/" rel="nofollow noreferrer">here along with adding the dependency mentioned here. I'm interested in using the KeyedOneInputStreamOperatorTestHarness class, but I can't find that class in any of my dependencies (redacted pom posted below). The only time I can find the class is by including

        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-java_2.12</artifactId>
            <version>1.14.5</version>
            <scope>test</scope>
            <type>test-jar</type>
        </dependency>

which is from the previous version. Flink >= 1.15.0 doesn't require the scala dependent libraries now, so there is no flink-streaming-java_2.12 for 1.15.1.

Did the class I want to use get moved somewhere else or purposefully excluded? I've tried all the google foo I know, and I can find the class in the flink repo, but not in any of the flink dependencies I've tried so far, outside of that <=1.14.5 one. Am I doing something wrong or missing some documentation? Is using the 1.14.5 library my only option or is there some new type of test utils that I don't know about?

Pom dependencies that don't have the class:

...
        <flink.version>1.15.1</flink.version>
...
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-java</artifactId>
            <version>${flink.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-clients</artifactId>
            <version>${flink.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-java</artifactId>
            <version>${flink.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-kafka</artifactId>
            <version>${flink.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-test-utils</artifactId>
            <version>${flink.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-test-utils-junit</artifactId>
            <version>${flink.version}</version>
            <scope>test</scope>
        </dependency>
...

Solution

It seems that I've figured it out, but my maven knowledge is limited so I'm not sure what the pros and cons are. Also I imagine future versions of flink will change this. The bonus is that I can use the latest flink version without including in older code.

I added these dependencies for testing (for 1.15.1 at least):

        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-test-utils</artifactId>
            <version>${flink.version}</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-java</artifactId>
            <version>${flink.version}</version>
            <scope>test</scope>
            <classifier>tests</classifier>
            <type>test-jar</type>
        </dependency>

I came across the flink readme about artifacts and the classifier tag, and I knew that the classes I needed were in flink-streaming-java package, specifically in its tests section. So including dependency for flink-streaming-java with the <classifier>tests</classifier> tag made the test classes available.

For what it's worth, there is a cryptic warning in the doc:

Useful when users should be pulling in Flink tests dependencies. This is mostly for the test harnesses and probably not what you want.

I still needed the flink-test-utils for the mock runner it provides.

Answered By - OMills
Answer Checked By - Willingham (JavaFixing Volunteer)