Verify the connector server certificate against the host over the WebSocket connection

[vharseko]

Companion to #122, which fixes the same class of problem in the legacy connector server client (CodeQL alert #24). This one covers the Grizzly WebSocket client in connector-framework-server, which CodeQL does not model because the SSLEngine is created inside Grizzly.

Problem

ConnectionManager wraps wss:// connections in a Grizzly SSLFilter whose SSLEngine validates the certificate chain but never checks the certificate against the host. Anyone holding a certificate the client trusts could sit in the middle of the connection to the connector server; with the usual setup (server certificate imported into the JVM truststore next to the public CAs) any publicly trusted certificate was enough.

Change

• ConnectionManager.SwitchingSSLFilter:
  • createClientSSLEngine creates the engine for the host of the remote URI (not for the proxy the socket may be connected to; IPv6 brackets stripped) and enables JSSE "HTTPS" endpoint identification via SSLParameters, so the certificate is verified against that host during the handshake, before any application data is sent. Applies to custom trust managers too, plain X509TrustManager included.
  • createOptimizedTransportFilter: reads bypass Grizzly's SSLTransportFilterWrapper while the connection is still plain. The wrapper gave a connection a server-mode SSLEngine on its first read; through a proxy that read is the plain answer to CONNECT, so the client handshake waited for a ClientHello nobody sends and every wss:// connection through a proxy timed out (on master too).
  • notifyHandshakeFailed logs TLS handshake with connector server host:port failed: <cause> at WARN and records the cause on the connection.
• ClientRemoteConnectorInfoManager: the close listener reports that cause — TLS handshake failed before WebSocket is established: … with the SSLHandshakeException chained — instead of the bare Connection is closed before WebSocket is established.
• ConnectionManagerConfig: hostnameVerification getter/setter, defaulting to the same system property the legacy client honours in Verify the connector server certificate against the host over the legacy SSL connection #122, org.identityconnectors.framework.remote.hostnameVerification. As in Verify the connector server certificate against the host over the legacy SSL connection #122 the property is read on every connection unless the config sets the value explicitly, so changing it takes effect without rebuilding the framework. Only false, in any letter case, disables the check.

Tests

ClientHostnameVerificationTest (connector-server-grizzly): a server with a CN-only certificate is rejected when connecting to wss://127.0.0.1 (cause chain carries SSLHandshakeException: No subject alternative names present), a certificate with matching subjectAltName is accepted and the test connector is looked up over it, the mismatching certificate is accepted when hostnameVerification=false, and through a CONNECT proxy on 127.0.0.1 a wss://localhost connection to the CN-only certificate is accepted, i.e. checked against the host of the URI, not the proxy (an engine keyed on the proxy's 127.0.0.1 fails with No subject alternative names present).

ConnectionManagerConfigTest (connector-framework-server) pins the system-property default, its letter case, later changes of the property and the precedence of an explicit setting.

The module's test certificates had expired in January 2025 (they only kept working because a self-signed end-entity certificate that is itself a trust anchor skips validity checks). They are regenerated with 30 years of validity; serverKeystore.jks now names localhost, 127.0.0.1 and ::1, which the client verifies, serverKeystore-cn-only.jks keeps a certificate without subjectAltName for the negative test, and README.txt has the keytool commands. AsyncRemoteSecureConnectorInfoManagerTest runs unchanged against the new certificate with verification on.

Local runs: connector-framework-server 32 tests, connector-server-grizzly 38 tests, connector-server-jetty 34 tests, all green.

Compatibility

Deployments whose connector server certificate does not name the host used in the wss:// URI fail the handshake now, with the reason in the log and in the exception. The fix is a certificate that names the host; the config switch / system property is the temporary escape hatch. Worth a release-notes entry together with #122.

Noticed, not changed

• WebSocketConnectionGroup.shutdown() throws NullPointerException from remote.getPromise().cancel(true) when a client is closed while a request is registered but not yet sent (RemoteRequest.promise is null until the message is sent). Hit while writing the test; separate issue.
• With verification disabled there is no "your certificate would not pass" diagnostic like the one Verify the connector server certificate against the host over the legacy SSL connection #122 adds to the legacy client; its matcher is package-private in connector-framework-internal and can be shared once Verify the connector server certificate against the host over the legacy SSL connection #122 is merged.

[maximthomas]

praise: The host check sits exactly where the engine is built, and a failed check now says why.

• SwitchingSSLFilter.createClientSSLEngine (ConnectionManager.java:918-932) keys the engine on the remote URI host, not the socket peer, and turns on "HTTPS" endpoint identification, so a plain X509TrustManager gets the check through JSSE's wrapper too.
• notifyHandshakeFailed records the cause, and ClientRemoteConnectorInfoManager.java:187-195 chains the SSLHandshakeException instead of the bare "Connection is closed before WebSocket is established".
• The expired test certificates are regenerated with SANs, serverKeystore-cn-only.jks gives the negative row, and README.txt keeps the keytool commands.

---

suggestion (non-blocking): No test pins that the engine is created for the remote-URI host rather than the socket peer.

OpenICF-java-framework/connector-framework-server/src/main/java/org/forgerock/openicf/framework/client/ConnectionManager.java:918-926, OpenICF-java-framework/connector-server-grizzly/src/test/java/org/forgerock/openicf/framework/server/ClientHostnameVerificationTest.java:172

Every active wss test connects to wss://127.0.0.1:<port> with no proxy (AsyncConnectorInfoManagerTestBase.buildRemoteWSFrameworkConnectionInfo), so the socket peer and the URI host are the same string, and a mutant that always calls super.createClientSSLEngine(sslCtx, sslEngineConfigurator) keeps all three rows green (read, not run). The only proxy fixture, buildRemoteProxyFrameworkConnectionInfo, is called from *.java.disable files over plain ws. The IPv6 bracket strip needs no row: JSSE strips the brackets itself.

Pin: a proxied row: a CONNECT proxy on 127.0.0.1 and the URI wss://localhost:<cnOnlyPort> against serverKeystore-cn-only.jks (CN=localhost, no SAN). The code under review checks localhost, JSSE falls back to the CN and accepts. The always-super mutant checks the proxy's 127.0.0.1, which needs an IP SAN, and rejects.

---

suggestion (non-blocking): No test exercises the system-property default of hostnameVerification.

OpenICF-java-framework/connector-framework-server/src/main/java/org/forgerock/openicf/framework/client/ConnectionManagerConfig.java:85-86, OpenICF-java-framework/connector-server-grizzly/src/test/java/org/forgerock/openicf/framework/server/ClientHostnameVerificationTest.java:120

The disabled row switches verification off through setHostnameVerification(false), and nothing sets org.identityconnectors.framework.remote.hostnameVerification. So replacing the initialiser with the constant true keeps all three rows green (read, not run). Only dropping the ! goes red. The property is the escape hatch the Compatibility section gives operators.

package org.forgerock.openicf.framework.client;

import static org.testng.Assert.assertFalse;
import static org.testng.Assert.assertTrue;

import org.testng.annotations.Test;

public class ConnectionManagerConfigTest {

    @Test
    public void hostnameVerificationDefaultsFromSystemProperty() {
        final String key = ConnectionManagerConfig.HOSTNAME_VERIFICATION_PROPERTY;
        final String saved = System.getProperty(key);
        try {
            System.clearProperty(key);
            assertTrue(new ConnectionManagerConfig().isHostnameVerification());
            System.setProperty(key, "false");
            assertFalse(new ConnectionManagerConfig().isHostnameVerification());
            System.setProperty(key, "flase");
            assertTrue(new ConnectionManagerConfig().isHostnameVerification());
        } finally {
            if (saved == null) {
                System.clearProperty(key);
            } else {
                System.setProperty(key, saved);
            }
        }
    }
}

Pin: the "false" row kills the constant-true mutant; the "flase" row holds the claim that a typo can not weaken the check.

---

suggestion (non-blocking): The WebSocket client reads org.identityconnectors.framework.remote.hostnameVerification once per ConnectorFramework, while the #122 legacy client reads it on every connect.

OpenICF-java-framework/connector-framework-server/src/main/java/org/forgerock/openicf/framework/client/ConnectionManagerConfig.java:85-86, OpenICF-java-framework/connector-framework-server/src/main/java/org/forgerock/openicf/framework/client/ConnectionManager.java:315-317, OpenICF-java-framework/connector-framework-server/src/main/java/org/forgerock/openicf/framework/ConnectorFramework.java:495

A field initialiser reads the property. ConnectorFramework builds its config eagerly as a field, and createFilterChain copies the value into a final field of SwitchingSSLFilter. A property set after the framework exists turns verification off for the legacy client only. OpenIDM's servlet-filter systemProperties is one such route: it calls System.setProperty/clearProperty at component activation. Clearing the property later re-enables the legacy check, but the WebSocket client stays unverified until the framework is rebuilt. The usual -D / system.properties route is read in time, so this only bites late or runtime changes.

// ConnectionManagerConfig
private boolean hostnameVerificationSet;

public boolean isHostnameVerification() {
    return hostnameVerificationSet
            ? hostnameVerification
            : !"false".equalsIgnoreCase(System.getProperty(HOSTNAME_VERIFICATION_PROPERTY));
}

public void setHostnameVerification(boolean hostnameVerification) {
    this.hostnameVerification = hostnameVerification;
    this.hostnameVerificationSet = true;
}

Or: keep the snapshot and say in the HOSTNAME_VERIFICATION_PROPERTY Javadoc that the WebSocket client reads it when the ConnectorFramework is created. With the change above, SwitchingSSLFilter should hold the ConnectionManagerConfig and call isHostnameVerification() inside createClientSSLEngine.

---

nitpick (non-blocking): The Javadoc says only the literal false disables verification, but the code accepts false in any letter case.

OpenICF-java-framework/connector-framework-server/src/main/java/org/forgerock/openicf/framework/client/ConnectionManagerConfig.java:36-41, :85-86

"false".equalsIgnoreCase(...) also accepts FALSE and False. That has no security effect, and #122 uses the same comparison, so the text should change, in the PR description as well.

    /**
     * System property that turns off TLS hostname verification for every
     * client unless a {@link ConnectionManagerConfig} says otherwise. Shared
     * with the legacy connector server client. Only {@code false}, in any
     * letter case, disables verification, so a typo in the value can not
     * weaken it.
     */

[vharseko]

All four taken.

Remote-URI host vs. socket peer — verifiesRemoteUriHostRatherThanProxy: a CONNECT proxy on 127.0.0.1, wss://localhost:<cnOnlyPort> against serverKeystore-cn-only.jks. Ran the always-super mutant: it fails with No subject alternative names present (checked against the proxy's 127.0.0.1), the code under review connects.

Writing the row turned up a separate bug: wss through a proxy never got past the tunnel. Grizzly's SSLTransportFilterWrapper gives a connection without an SSLEngine a server-mode one on its first read; through a proxy that read is the plain 200 to CONNECT, so the client handshake waited for a ClientHello nobody sends, and the connect timed out. SwitchingSSLFilter.createOptimizedTransportFilter now lets reads bypass the wrapper until the connection is switched to TLS. Without it the proxied branch of createClientSSLEngine was unreachable.

System-property default — ConnectionManagerConfigTest, with your rows plus FALSE. The constant-true mutant fails it.

Snapshot vs. per-connect read — isHostnameVerification() reads the property on each call unless setHostnameVerification was called (Boolean field, null = follow the property), and SwitchingSSLFilter holds the config and asks it in createClientSSLEngine. Pinned by hostnameVerificationFollowsLaterPropertyChanges and explicitSettingOverridesSystemProperty.

Javadoc — now says false in any letter case, and that the property is read on every connection; PR description updated to match.

connector-framework-server 32, connector-server-grizzly 38, connector-server-jetty 34 tests, all green.