Metro WS-AT transaction coordination and https

I recently had to configure a secure and reliable webservice connection. We use WS-AtomicTransactions, WS-Security (Mutual Certificates Security, MCS) and WS-ReliableMessaging, however all this over HTTP transport (not HTTPS). Things are quite easy to set up with Netbeans (6.7.1) and Glassfish v2.1 (which uses Metro 1.1.5). But at some point it is just not working, and you have no more buttons to click in the wizard windows.

The situation was simple, I used two different projects to pilot the solution. Both had a simple EJB Session bean with a method, also published as webservice operation. From one of the EJBs I invoked the other with WS (this connection had AT, RM and Security configured). I tested the first WS with its standard Tester. The two applications got deployed on two different glassfish domains, just to make sure.

The problem came when I enabled transactions:

WSTX-COORDINATOR-3006: register sent to EPR '
https://hhcofcmds:8281/__wstx-services/wscoor/coordinator
/register
7ef29c5a-
2633-40a7-ae4a-54d94e818ec0
' failed for activityId 'uuid:WSCOOR-SUN-3871ef38-e5b2-490c-
9fbe-a39fa442daee'.  Nested exception: 'HTTP transport error: 
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX 
path building failed: sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target'

WSTX-AT-0022: Registration with durable parent failed: ' coordId=uuid:WSCOOR-
SUN-3871ef38-e5b2-490c-9fbe-a39fa442daee partId:1 '

Some forum entries suggest that I should disable transactions on the invoked WS. Well, this is not an option for me, as having transactions was my actual intent. So, WS-AT and WS-RM use some additional WS invocations to the transaction coordinator service to ensure all the stuff it has to ensure. The logs above show that someone tries to contact the coordinator over https. Which surely won't be wokring, as I use http transfer for the whole stuff.

I found a note in some forum post that the coordination stuff uses https by default. So I dived into the source to find this in class com.sun.xml.ws.tx.common.AddressManager

    private static String preferredScheme = 
System.getProperty("com.sun.xml.ws.tx.preferredScheme", "https");

The methods getPreferredAddress and getAddress use this setting. So we just have to add this line in the proper place in the domain.xml of both glassfish domains:

        <jvm-options>-Dcom.sun.xml.ws.tx.preferredScheme=http</jvm-options>

This will cause the transaction coordination to use http transfer.