GSI and OpenSSL 1.0

Introduction

Globus Toolkit 5.0.1 and later are compatible with OpenSSL 1.0.0. However, an interface change in OpenSSL 1.0.0 impacts the Globus Toolkit Grid Security Infrastructure (GSI). Specifically, OpenSSL 1.0.0 changes the hash format used for naming certificate directory files (CA certificates and CRLs). OpenSSL 1.0.0 and later will locate certificate directory files using the new hash values, while older OpenSSL versions will locate certificate directory files using the old hash values. If the Globus Toolkit components that use OpenSSL can not locate the certificate directory files, GSI authentication will fail. The following transcript shows the difference in hash values.

$ openssl version

OpenSSL 1.0.0 29 Mar 2010

$ openssl x509 -hash -noout < $X509_CERT_DIR/9b95bbf2.0

5bb7d63e

$ openssl x509 -subject_hash_old -noout < $X509_CERT_DIR/9b95bbf2.0

9b95bbf2

$ openssl crl -hash -noout < $X509_CERT_DIR/9b95bbf2.r0

5bb7d63e

For compatibility with different OpenSSL versions, it is best to create certificate directory files (or symbolic links) using both hash values. The next section documents the tools available for creating the needed links. Later sections document issues with specific Globus Toolkit components (MyProxy and Simple CA). The final section documents common error messages and their causes.

For more information on this issue, see:

• EUGridPMA Newsletter
• Globus Bug 6896
• OSG Security Blog

Credit: Thanks to Willy Weisz <weisz@vcpc.univie.ac.at> for the patch introducing the -subject_hash_old option shown in the above, in OpenSSL 1.0.0.

Tools for Updating Certificate Directory Hashes

c_rehash

OpenSSL 1.0.0 provides a tool called c_rehash that

• deletes all existing symlinks in a given directory,
• looks for *.pem files and
• creates symlinks to them using the new hash values taking into account any hash collisions and such.

However, this tool is not compatible with typical GSI configurations where the CA certificates, CRLs and signing policy files are named <hash>.0, <hash>.r0 and <hash>.signing_policy (respectively).

x509-cert-dir-rehash

The x509-cert-dir-rehash tool (available as an attachment at the bottom of this page) is a modified version of the c_rehash tool that

• won’t disturb any existing symlinks or files and
• attempts to create additional symlinks using the new hash values.

If it runs into any hash collisions, it will simply complain with appropriate non-fatal error messages.

The following example shows how to update the files in $HOME/.globus/certificates using the x509-cert-dir-rehash tool:

$ ls -l $HOME/.globus/certificates

-rw-r--r-- 1 jsmith jsmith 1456 2010-04-07 16:15 9b95bbf2.0

-rw-r--r-- 1 jsmith jsmith 4793 2010-04-07 16:15 9b95bbf2.r0

-rw-r--r-- 1 jsmith jsmith  320 2010-04-07 16:15 9b95bbf2.signing_policy

$ curl -LO http://www.cilogon.org/openssl1/x509-cert-dir-rehash

$ chmod u+x ./x509-cert-dir-rehash

$ ./x509-cert-dir-rehash $HOME/.globus/certificates

Processing files in directory /home/jsmith/.globus/certificates

$ ls -l $HOME/.globus/certificates

lrwxrwxrwx 1 jsmith jsmith   10 2010-04-07 16:23 5bb7d63e.0 -> 9b95bbf2.0

lrwxrwxrwx 1 jsmith jsmith   11 2010-04-07 16:23 5bb7d63e.r0 -> 9b95bbf2.r0

lrwxrwxrwx 1 jsmith jsmith   23 2010-04-07 16:23 5bb7d63e.signing_policy -> 9b95bbf2.signing_policy

-rw-r--r-- 1 jsmith jsmith 1456 2010-04-07 16:15 9b95bbf2.0

-rw-r--r-- 1 jsmith jsmith 4793 2010-04-07 16:15 9b95bbf2.r0

-rw-r--r-- 1 jsmith jsmith  320 2010-04-07 16:15 9b95bbf2.signing_policy

globus-update-certificate-dir

Globus Toolkit 5.0.1 also includes a globus-update-certificate-dir script that will create links with the new hash values, but it currently has the following shortcomings:

• It deletes any pre-existing links with the new hash value, which could cause trouble in situations where the pre-existing link might be pointing to a different base CA certificate file that happens to have a hash collision with the CA certificate at hand.

These issues have been reported in Globus Bug 6995 .

Note that all of these tools require OpenSSL 1.0.0 or later to compute the new hash values. They are not compatible with earlier versions of OpenSSL.

Manually Creating Symlinks

If the above tools don't do exactly what you need, you can manually create symbolic links (symlinks) for the needed hash names. For example:

$ ls -l $HOME/.globus/certificates

-rw-r--r-- 1 jsmith jsmith 1456 2010-04-07 16:15 9b95bbf2.0

-rw-r--r-- 1 jsmith jsmith 4793 2010-04-07 16:15 9b95bbf2.r0

-rw-r--r-- 1 jsmith jsmith  320 2010-04-07 16:15 9b95bbf2.signing_policy

$ cd $HOME/.globus/certificates

$ ln -s 9b95bbf2.0 5bb7d63e.0

$ ln -s 9b95bbf2.r0 5bb7d63e.r0

$ ln -s 9b95bbf2.signing_policy 5bb7d63e.signing_policy

$ ls -l $HOME/.globus/certificates

lrwxrwxrwx 1 jsmith jsmith   10 2010-04-07 16:23 5bb7d63e.0 -> 9b95bbf2.0

lrwxrwxrwx 1 jsmith jsmith   11 2010-04-07 16:23 5bb7d63e.r0 -> 9b95bbf2.r0

lrwxrwxrwx 1 jsmith jsmith   23 2010-04-07 16:23 5bb7d63e.signing_policy -> 9b95bbf2.signing_policy

-rw-r--r-- 1 jsmith jsmith 1456 2010-04-07 16:15 9b95bbf2.0

-rw-r--r-- 1 jsmith jsmith 4793 2010-04-07 16:15 9b95bbf2.r0

-rw-r--r-- 1 jsmith jsmith  320 2010-04-07 16:15 9b95bbf2.signing_policy

MyProxy Trust Roots and Bootstrapping

See Managing Trust Roots with MyProxy for an introduction to the MyProxy functionality impacted by this OpenSSL 1.0.0 interface change.

For compatibility with MyProxy clients using different OpenSSL versions, the MyProxy administrator should configure the myproxy-server to deliver trust roots named with both the new and old hash values. The administrator can use the tools described above to create the needed symbolic links in the server's X509_CERT_DIR. If the myproxy-server fails to deliver trust roots with both new and old hash values, the following problem may occur.

If the client has no trust roots installed, the 'myproxy-logon -T' command will "bootstrap" by installing an initial set of trust roots in ~/.globus/certificates. First, the client determines the CA certificate(s) and signing policy file(s) that are required to establish a secure connection with the myproxy-server and saves those files locally, using locally computed hash values (such as 5bb7d63e.0 and 5bb7d63e.signing_policy). Then, the client connects to the server, downloads a full set of trust roots, and saves the certificate files locally using the filenames specified by the server. If the myproxy-server delivers trust roots with different hash names than required by the client's OpenSSL version, later GSI commands will fail (see the Troubleshooting section below).

The client can recover from this problem by creating symbolic links with the new hash names in ~/.globus/certificates using the tools described above.

The client may also need to manually replace a <hash>.signing_policy file created by 'myproxy-logon -T' during bootstrapping. This file is initially created during the boostrapping phase to allow connecting only to the myproxy-server (i.e., to accept only the MyProxy server's certificate). For example:

$ cat $HOME/.globus/certificates/5bb7d63e.signing_policy

access_id_CA X509 '/C=US/O=National Center for Supercomputing Applications/OU=Certificate Authorities/CN=CACL'

pos_rights globus CA:sign

cond_subjects globus '"/C=US/O=National Center for Supercomputing Applications/OU=Services/CN=myproxy.ncsa.uiuc.edu"'

Normally this file is overwritten by the official version from the myproxy-server when 'myproxy-logon -T' installs the full set of trust roots. However, if myproxy-logon is using a new OpenSSL version and the myproxy-server has trust roots using only the old hashes, the <hash>.signing_policy file created locally by 'myproxy-logon -T' will have the new hash and the official version will have the old hash. For example:

$ cat $HOME/.globus/certificates/9b95bbf2.signing_policy

access_id_CA   X509    '/C=US/O=National Center for Supercomputing Applications/OU=Certificate Authorities/CN=CACL'

pos_rights     globus  CA:sign

cond_subjects  globus  '"/C=US/O=National Center for Supercomputing Applications/OU=Services/CN=*" "/C=US/O=National Center for Supercomputing Applications/OU=People/CN=*"'

This situation may result in an error indicating "The subject of the certificate... does not match the signing policies defined in $HOME/.globus/certificates/<new-hash>.signing_policy".

For the client to use the official version of the signing_policy file, $HOME/.globus/certificates/<new-hash>.signing_policy should be replaced with a symbolic link to $HOME/.globus/certificates/<old-hash>.signing_policy. For example:

$ rm $HOME/.globus/certificates/5bb7d63e.signing_policy

$ ln -s $HOME/.globus/certificates/9b95bbf2.signing_policy \

        $HOME/.globus/certificates/5bb7d63e.signing_policy

If the x509-cert-dir-update script (described above) warns about an existing signing_policy file, this is the likely cause.

Globus Simple CA

Globus Simple CA tools such as grid-cert-request and grid-ca-sign support the following CA-specific configuration files, which won’t be found after upgrading to OpenSSL 1.0.0 unless new links are created using the new hash values. Currently there is no script that automatically fixes this issue. The Globus administrator needs to manually determine the hash value, using the openssl command shown in the Introduction, and then create symlinks to the files as needed:

• globus-user-ssl.conf.<hash>
• globus-host-ssl.conf.<hash>
• grid_security.conf.<hash>
• directions.<hash>

Globus Java Components

Globus Java components such as Java WS Core and CoG jglobus locate certificate directory files using the old hash algorithm.

Troubleshooting

If OpenSSL does not find certificate directory files named with the required hash, GSI authentication will fail. The following is a list of errors that could occur and an explanation of the causes.

Cannot find trusted CA certificate with hash

globus_gsi_callback_module: Could not verify credential

globus_gsi_callback_module: Can't get the local trusted CA certificate: Cannot find trusted CA certificate with hash d492aff2 in /etc/grid-security/certificates

In this case, the GSI certificate verification algorithm could not locate the certificate for the CA that signed the certificate being verified. This will occur when using OpenSSL 1.0.0 or later when the CA certificate filenames use the old hash values, or when using older OpenSSL versions with CA certificate filenames using the new hash values. For compatibility with different OpenSSL versions, it is best to use the tools above to create certificate filenames with both old and new hash values or manually create symlinks as needed (see above).

Subject of the certificate does not match the signing policies

globus_gsi_gssapi: SSLv3 handshake problems

globus_gsi_callback_module: Could not verify credential

globus_gsi_callback_module: Error with signing policy

globus_gsi_callback_module: Error in OLD GAA code: The subject of the certificate "/C=US/O=National Center for Supercomputing Applications/OU=Services/CN=gstest.ncsa.uiuc.edu" does not match the signing policies defined in $HOME/.globus/certificates/5bb7d63e.signing_policy

This error may occur when using MyProxy trust root bootstrapping, in the case where myproxy-logon uses the new OpenSSL hash algorithm and the certificate directory filenames on the myproxy-server use the old OpenSSL hash algorithm (or vice versa). See the MyProxy Trust Roots and Bootstrapping section above for details.