Issue
I want to run my pipeline using direct runner in eclipse and put a break point in my DoFn functions and debug execution. I tried to setup direct runner with following steps:
- Add direct runner maven package
- Setup maven profile for direct runner in pom.xml. My pom.xml has this profile
<profiles>
<profile>
<id>direct-runner</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-direct-java</artifactId>
<version>0.2.0-incubating</version>
</dependency>
</dependencies>
</profile>
</profiles>
- I have this maven plugin under plugin management in my pom.xml
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<executions>
<execution>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<cleanupDaemonThreads>false</cleanupDaemonThreads>
<mainClass>com.MyMainClass</mainClass>
</configuration>
</plugin>
</plugins>
</pluginManagement>
- Below is a screen shot of my eclipse debug configuration
When I run using above debug configuration job starts in GCP dataflow instead of local JVM threads and my breakpoints are never hit.
Solution
Probably is the way how you are creating your pipeline in your test methods. Try to create the pipeline using the TestPipeline util class like this
public TestPipeline p = TestPipeline.create();
Answered By - hlagos
