[FIXED] Allow insecure HTTPS connection for Java JDK 11 HttpClient

Issue

Sometimes it is needed to allow insecure HTTPS connections, e.g. in some web-crawling applications which should work with any site. I used one such solution with old HttpsURLConnection API which was recently superseded by the new HttpClient API in JDK 11. What is the way to allow insecure HTTPS connections (self-signed or expired certificate) with this new API?

UPD: The code I tried (in Kotlin but maps directly to Java):

    val trustAllCerts = arrayOf<TrustManager>(object: X509TrustManager {
        override fun getAcceptedIssuers(): Array<X509Certificate>? = null
        override fun checkClientTrusted(certs: Array<X509Certificate>, authType: String) {}
        override fun checkServerTrusted(certs: Array<X509Certificate>, authType: String) {}
    })

    val sslContext = SSLContext.getInstance("SSL")
    sslContext.init(null, trustAllCerts, SecureRandom())

    val sslParams = SSLParameters()
    // This should prevent host validation
    sslParams.endpointIdentificationAlgorithm = ""

    httpClient = HttpClient.newBuilder()
        .sslContext(sslContext)
        .sslParameters(sslParams)
        .build()

But on sending I have exception (trying on localhost with self-signed certificate):

java.io.IOException: No name matching localhost found

Using IP address instead of localhost gives "No subject alternative names present" exception.

After some debugging of JDK I found that sslParams are really ignored in the place the exception is thrown and some locally created instance is used. Further debugging revealed that the only way to affect the hostname verification algorithm is setting jdk.internal.httpclient.disableHostnameVerification system property to true. And that's seems to be a solution. SSLParameters in the code above have no effect so this part can be discarded. Making it configurable only globally looks like serious design flaw in new HttpClient API.

Solution

With Java 11, as well you can do a similar effort as mentioned in the selected answer in the link shared with the HttpClient built as:

HttpClient httpClient = HttpClient.newBuilder()
        .connectTimeout(Duration.ofMillis(<timeoutInSeconds> * 1000))
        .sslContext(sc) // SSL context 'sc' initialised as earlier
        .sslParameters(parameters) // ssl parameters if overriden
        .build();

with a sample request

HttpRequest requestBuilder = HttpRequest.newBuilder()
            .uri(URI.create("https://www.example.com/getSomething"))
            .GET()
            .build();

can be executed as:

httpClient.send(requestBuilder, HttpResponse.BodyHandlers.ofString()); // sends the request

---

Update from comments, to disable the hostname verification, currently one can use the system property:

-Djdk.internal.httpclient.disableHostnameVerification

which can be set programmatically as following :-

final Properties props = System.getProperties(); 
props.setProperty("jdk.internal.httpclient.disableHostnameVerification", Boolean.TRUE.toString());

Answered By - Naman
Answer Checked By - Timothy Miller (JavaFixing Admin)