[FIXED] How to use java runtime 11 in EMR cluster AWS

Issue

I'm creating a cluter in EMR aws and when spark runs my application I'm getting error below:

Exception in thread "main" java.lang.UnsupportedClassVersionError: 
com/example/demodriver/MyClassFromJAR has been compiled by a more recent version of the Java Runtime (class file version 55.0), 
this version of the Java Runtime only recognizes class file versions up to 52.0

I'm using releaseLabel emr-6.5.0 on clusters and My driver jar is built in java11

How run java11 application in EMR? Or this error is about anything else?

Solution

In the most recent versions of EMR, java 11 is installed. To enable it, you can provide the following configuration.

[
    {
        "Classification": "spark-env",
        "Configurations": [
            {
                "Classification": "export",
                "Properties": {
                    "JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
                }
            }
        ]
    },
    {
        "Classification": "spark-defaults",
        "Properties": {
            "spark.executorEnv.JAVA_HOME": "/usr/lib/jvm/java-11-amazon-corretto.x86_64"
        }
    }
]

This does not seem to be documented.

The defaultJavaOptions and extraJavaOptions might contain incompatible options for java 11 which you still might need to adapt/update.

Answered By - Steven
Answer Checked By - David Goodson (JavaFixing Volunteer)