Generate HTTPS certificates with ssl_setup.

Extracts common OpenSSL functionality from pki_setup and adds a new cli
command ssl_setup which re-uses this base to generate SSL certificates
for https.

Change-Id: Ia34827583bcdfbd871133250681010e642271f07
Fixes: bug 1155361

7 files changed, 210 insertions, 96 deletions

diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst
index c390a7fb..4b09f2c4 100644
--- a/doc/source/configuration.rst
+++ b/doc/source/configuration.rst
@@ -161,6 +161,7 @@ The values that specify where to read the certificates are under the
 * ``certfile`` - Location of certificate used to verify tokens.  Default is ``/etc/keystone/ssl/certs/signing_cert.pem``
 * ``keyfile`` - Location of private key used to sign tokens.  Default is ``/etc/keystone/ssl/private/signing_key.pem``
 * ``ca_certs`` - Location of certificate for the authority that issued the above certificate. Default is ``/etc/keystone/ssl/certs/ca.pem``
+* ``ca_key`` - Default is ``/etc/keystone/ssl/certs/cakey.pem``
 * ``key_size`` - Default is ``1024``
 * ``valid_days`` - Default is ``3650``
 * ``ca_password``  - Password required to read the ca_file. Default is None
@@ -176,8 +177,8 @@ the following conditions:
 * private key files must not be protected by a password
 
 When using signing certificate issued by an external CA, you do not need to
-specify ``key_size``, ``valid_days``, and ``ca_password`` as they will be
-ignored.
+specify ``key_size``, ``valid_days``, ``ca_key`` and ``ca_password`` as they
+will be ignored.
 
 The basic workflow for using a signing certificate issed by an external CA involves:
 
@@ -359,10 +360,10 @@ Reset collected data using::
 SSL
 ---
 
-Keystone may be configured to support 2-way SSL out-of-the-box.  The x509
-certificates used by Keystone must be obtained externally and configured for use
-with Keystone as described in this section.  However, a set of sample certficates
-is provided in the examples/pki/certs and examples/pki/private directories with the Keystone distribution for testing.
+Keystone may be configured to support SSL and 2-way SSL out-of-the-box.
+The X509 certificates used by keystone can be generated by keystone-manage or
+obtained externally and configured for use with Keystone as described in this
+section.
 Here is the description of each of them and their purpose:
 
 Types of certificates
@@ -390,7 +391,7 @@ provided as an example.
 Configuration
 ^^^^^^^^^^^^^
 
-To enable SSL with client authentication, modify the etc/keystone.conf file accordingly
+To enable SSL modify the etc/keystone.conf file accordingly
 under the [ssl] section.  SSL configuration example using the included sample
 certificates::
 
@@ -399,7 +400,8 @@ certificates::
     certfile = <path to keystone.pem>
     keyfile = <path to keystonekey.pem>
     ca_certs = <path to ca.pem>
-    cert_required = True
+    ca_key = <path to cakey.pem>
+    cert_required = False
 
 * ``enable``:  True enables SSL.  Defaults to False.
 * ``certfile``:  Path to Keystone public certificate file.
@@ -407,6 +409,29 @@ certificates::
 * ``ca_certs``:  Path to CA trust chain.
 * ``cert_required``:  Requires client certificate.  Defaults to False.
 
+When generating SSL certificates the following values are read
+
+* ``key_size``: Key size to create. Defaults to 1024.
+* ``valid_days``: How long the certificate is valid for. Defaults to 3650 (10 years).
+* ``ca_key``: The private key for the CA. Defaults to ``/etc/keystone/ssl/certs/cakey.pem``.
+* ``ca_password``: The password for the CA private key. Defaults to None.
+* ``cert_subject``: The subject to set in the certificate. Defaults to /C=US/ST=Unset/L=Unset/O=Unset/CN=localhost. When setting the subject it is important to set CN to be the address of the server so client validation will succeed. This generally means having the subject be at least /CN=<keystone ip>
+
+Generating SSL certificates
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Certificates for secure HTTP communication can be generated by::
+
+    $ keystone-manage ssl_setup
+
+This will create a private key, a public key and a certificate that will be
+used to encrypt communications with keystone. In the event that a Certificate
+Authority is not given a testing one will be created.
+
+It is likely in a production environment that these certificates will be
+created and provided externally.
+
+
 User CRUD
 ---------
 
@@ -620,6 +645,7 @@ through the normal REST API. At the moment, the following calls are supported:
 * ``export_legacy_catalog``: Export service catalog from a legacy (pre-Essex) database.
 * ``import_nova_auth``: Load auth data from a dump created with ``nova-manage``.
 * ``pki_setup``: Initialize the certificates for PKI based tokens.
+* ``ssl_setup``: Generate certificates for HTTPS.
 
 Invoking ``keystone-manage`` by itself will give you additional usage
 information.
@@ -1031,7 +1057,7 @@ is::
 There are some configuration options for filtering users, tenants and roles,
 if the backend is providing too much output, in such case the configuration
 will look like::
-  
+
   [ldap]
   user_filter = (memberof=CN=openstack-users,OU=workgroups,DC=openstack,DC=com)
   tenant_filter =
@@ -1054,7 +1080,7 @@ the mask then the account is disabled.
 
 It also saves the value without mask to the user identity in the attribute
 *enabled_nomask*. This is needed in order to set it back in case that we need to
-change it to enable/disable a user because it contains more information than the 
+change it to enable/disable a user because it contains more information than the
 status like password expiration. Last setting *user_enabled_mask* is needed in order
 to create a default value on the integer attribute (512 = NORMAL ACCOUNT on AD)
 
@@ -1103,4 +1129,4 @@ A few points worth mentioning regarding the above options.  If both
 tls_cacertfile and tls_cacertdir are set then tls_cacertfile will be
 used and tls_cacertdir is ignored.  Furthermore, valid options for
 tls_req_cert are demand, never, and allow.  These correspond to the
-standard options permitted by the TLS_REQCERT TLS option.
\ No newline at end of file
+standard options permitted by the TLS_REQCERT TLS option.
diff --git a/doc/source/man/keystone-manage.rst b/doc/source/man/keystone-manage.rst
index af5ad409..b7c2131c 100644
--- a/doc/source/man/keystone-manage.rst
+++ b/doc/source/man/keystone-manage.rst
@@ -48,6 +48,7 @@ Available commands:
 * ``import_legacy``: Import a legacy database.
 * ``import_nova_auth``: Import a dump of nova auth data into keystone.
 * ``pki_setup``: Initialize the certificates used to sign tokens.
+* ``ssl_setup``: Generate certificates for SSL.
 
 
 OPTIONS
diff --git a/etc/keystone.conf.sample b/etc/keystone.conf.sample
index ee2a562e..5ebddea4 100644
--- a/etc/keystone.conf.sample
+++ b/etc/keystone.conf.sample
@@ -123,7 +123,11 @@
 #certfile = /etc/keystone/ssl/certs/keystone.pem
 #keyfile = /etc/keystone/ssl/private/keystonekey.pem
 #ca_certs = /etc/keystone/ssl/certs/ca.pem
-#cert_required = True
+#key_size = 1024
+#valid_days = 3650
+#ca_password = None
+#cert_required = False
+#cert_subject = /C=US/ST=Unset/L=Unset/O=Unset/CN=localhost
 
 [signing]
 #token_format = PKI
@@ -133,6 +137,7 @@
 #key_size = 1024
 #valid_days = 3650
 #ca_password = None
+#cert_subject = /C=US/ST=Unset/L=Unset/O=Unset/CN=www.example.com
 
 [ldap]
 # url = ldap://localhost
diff --git a/keystone/cli.py b/keystone/cli.py
index fc9d3044..5dace7fa 100644
--- a/keystone/cli.py
+++ b/keystone/cli.py
@@ -54,23 +54,22 @@ class DbSync(BaseApp):
                 driver.db_sync()
 
 
-class PKISetup(BaseApp):
-    """Set up Key pairs and certificates for token signing and verification."""
-
-    name = 'pki_setup'
+class BaseCertificateSetup(BaseApp):
+    """Common user/group setup for PKI and SSL generation"""
 
     @classmethod
     def add_argument_parser(cls, subparsers):
-        parser = super(PKISetup,
+        parser = super(BaseCertificateSetup,
                        cls).add_argument_parser(subparsers)
         parser.add_argument('--keystone-user')
         parser.add_argument('--keystone-group')
         return parser
 
     @staticmethod
-    def main():
+    def get_user_group():
         keystone_user_id = None
         keystone_group_id = None
+
         try:
             a = CONF.command.keystone_user
             if a:
@@ -85,7 +84,30 @@ class PKISetup(BaseApp):
         except KeyError:
             raise ValueError("Unknown group '%s' in --keystone-group" % a)
 
-        conf_ssl = openssl.ConfigurePKI(keystone_user_id, keystone_group_id)
+        return keystone_user_id, keystone_group_id
+
+
+class PKISetup(BaseCertificateSetup):
+    """Set up Key pairs and certificates for token signing and verification."""
+
+    name = 'pki_setup'
+
+    @classmethod
+    def main(cls):
+        keystone_user_id, keystone_group_id = cls.get_user_group()
+        conf_pki = openssl.ConfigurePKI(keystone_user_id, keystone_group_id)
+        conf_pki.run()
+
+
+class SSLSetup(BaseCertificateSetup):
+    """Create key pairs and certificates for HTTPS connections"""
+
+    name = 'ssl_setup'
+
+    @classmethod
+    def main(cls):
+        keystone_user_id, keystone_group_id = cls.get_user_group()
+        conf_ssl = openssl.ConfigureSSL(keystone_user_id, keystone_group_id)
         conf_ssl.run()
 
 
@@ -150,6 +172,7 @@ CMDS = [
     ImportLegacy,
     ImportNovaAuth,
     PKISetup,
+    SSLSetup,
 ]
 
 
diff --git a/keystone/common/config.py b/keystone/common/config.py
index f749300d..a4fa4901 100644
--- a/keystone/common/config.py
+++ b/keystone/common/config.py
@@ -217,10 +217,20 @@ def configure():
 
     # ssl
     register_bool('enable', group='ssl', default=False)
-    register_str('certfile', group='ssl', default=None)
-    register_str('keyfile', group='ssl', default=None)
-    register_str('ca_certs', group='ssl', default=None)
+    register_str('certfile', group='ssl',
+                 default="/etc/keystone/ssl/certs/keystone.pem")
+    register_str('keyfile', group='ssl',
+                 default="/etc/keystone/ssl/private/keystonekey.pem")
+    register_str('ca_certs', group='ssl',
+                 default="/etc/keystone/ssl/certs/ca.pem")
+    register_str('ca_key', group='ssl',
+                 default="/etc/keystone/ssl/certs/cakey.pem")
     register_bool('cert_required', group='ssl', default=False)
+    register_int('key_size', group='ssl', default=1024)
+    register_int('valid_days', group='ssl', default=3650)
+    register_str('ca_password', group='ssl', default=None)
+    register_str('cert_subject', group='ssl',
+                 default='/C=US/ST=Unset/L=Unset/O=Unset/CN=localhost')
 
     # signing
     register_str(
@@ -237,9 +247,13 @@ def configure():
         'ca_certs',
         group='signing',
         default="/etc/keystone/ssl/certs/ca.pem")
+    register_str('ca_key', group='signing',
+                 default="/etc/keystone/ssl/certs/cakey.pem")
     register_int('key_size', group='signing', default=1024)
     register_int('valid_days', group='signing', default=3650)
     register_str('ca_password', group='signing', default=None)
+    register_str('cert_subject', group='signing',
+                 default='/C=US/ST=Unset/L=Unset/O=Unset/CN=www.example.com')
 
     # sql
     register_str('connection', group='sql', secret=True,
diff --git a/keystone/common/openssl.py b/keystone/common/openssl.py
index d4dc3af0..dd31fb71 100644
--- a/keystone/common/openssl.py
+++ b/keystone/common/openssl.py
@@ -29,41 +29,34 @@ DIR_PERMS = (stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR |
              stat.S_IROTH | stat.S_IXOTH)
 CERT_PERMS = stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH
 PRIV_PERMS = stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
-DEFAULT_SUBJECT = '/C=US/ST=Unset/L=Unset/O=Unset/CN=www.example.com'
 
 
 def file_exists(file_path):
     return os.path.exists(file_path)
 
 
-class ConfigurePKI(object):
-    """Generate files for PKI signing using OpenSSL.
-
-    Signed tokens require a private key and signing certificate which itself
-    must be signed by a CA.  This class generates them with workable defaults
-    if each of the files are not present
-
-    """
+class BaseCertificateConfigure(object):
+    """Create a certificate signing environment from a config section
+    and reasonable OpenSSL defaults"""
 
-    def __init__(self, keystone_user, keystone_group, **kw):
-        self.conf_dir = os.path.dirname(CONF.signing.ca_certs)
+    def __init__(self, conf_obj, keystone_user, keystone_group, **kwargs):
+        self.conf_dir = os.path.dirname(conf_obj.ca_certs)
         self.use_keystone_user = keystone_user
         self.use_keystone_group = keystone_group
         self.ssl_config_file_name = os.path.join(self.conf_dir, "openssl.conf")
-        self.ca_key_file = os.path.join(self.conf_dir, "cakey.pem")
         self.request_file_name = os.path.join(self.conf_dir, "req.pem")
         self.ssl_dictionary = {'conf_dir': self.conf_dir,
-                               'ca_cert': CONF.signing.ca_certs,
+                               'ca_cert': conf_obj.ca_certs,
                                'ssl_config': self.ssl_config_file_name,
-                               'ca_private_key': self.ca_key_file,
-                               'ca_cert_cn': 'hostname',
+                               'ca_private_key': conf_obj.ca_key,
                                'request_file': self.request_file_name,
-                               'signing_key': CONF.signing.keyfile,
-                               'signing_cert': CONF.signing.certfile,
-                               'default_subject': DEFAULT_SUBJECT,
-                               'key_size': int(CONF.signing.key_size),
-                               'valid_days': int(CONF.signing.valid_days),
-                               'ca_password': CONF.signing.ca_password}
+                               'signing_key': conf_obj.keyfile,
+                               'signing_cert': conf_obj.certfile,
+                               'key_size': int(conf_obj.key_size),
+                               'valid_days': int(conf_obj.valid_days),
+                               'cert_subject': conf_obj.cert_subject,
+                               'ca_password': conf_obj.ca_password}
+        self.ssl_dictionary.update(kwargs)
 
     def _make_dirs(self, file_name):
         dir = os.path.dirname(file_name)
@@ -106,20 +99,25 @@ class ConfigurePKI(object):
         self._set_permissions(self.ssl_config_file_name, PRIV_PERMS)
 
     def build_ca_cert(self):
-        if not file_exists(CONF.signing.ca_certs):
-            if not os.path.exists(self.ca_key_file):
-                self._make_dirs(self.ca_key_file)
-                self.exec_command('openssl genrsa -out %(ca_private_key)s '
-                                  '%(key_size)d -config %(ssl_config)s')
-                self._set_permissions(self.ssl_dictionary['ca_private_key'],
-                                      stat.S_IRUSR)
+        ca_key_file = self.ssl_dictionary['ca_private_key']
+        ca_cert = self.ssl_dictionary['ca_cert']
+
+        if not file_exists(ca_key_file):
+            self._make_dirs(ca_key_file)
+            self.exec_command('openssl genrsa -out %(ca_private_key)s '
+                              '%(key_size)d')
+            self._set_permissions(self.ssl_dictionary['ca_private_key'],
+                                  stat.S_IRUSR)
+
+        if not file_exists(ca_cert):
+            self._make_dirs(ca_cert)
             self.exec_command('openssl req -new -x509 -extensions v3_ca '
                               '-passin pass:%(ca_password)s '
                               '-key %(ca_private_key)s -out %(ca_cert)s '
                               '-days %(valid_days)d '
                               '-config %(ssl_config)s '
-                              '-subj %(default_subject)s')
-            self._set_permissions(self.ssl_dictionary['ca_cert'], CERT_PERMS)
+                              '-subj %(cert_subject)s')
+            self._set_permissions(ca_cert, CERT_PERMS)
 
     def build_private_key(self):
         signing_keyfile = self.ssl_dictionary['signing_key']
@@ -128,19 +126,23 @@ class ConfigurePKI(object):
             self._make_dirs(signing_keyfile)
 
             self.exec_command('openssl genrsa -out %(signing_key)s '
-                              '%(key_size)d '
-                              '-config %(ssl_config)s')
+                              '%(key_size)d ')
         self._set_permissions(os.path.dirname(signing_keyfile), PRIV_PERMS)
         self._set_permissions(signing_keyfile, stat.S_IRUSR)
 
     def build_signing_cert(self):
-        if not file_exists(CONF.signing.certfile):
-            self._make_dirs(CONF.signing.certfile)
+        signing_cert = self.ssl_dictionary['signing_cert']
+
+        if not file_exists(signing_cert):
+            self._make_dirs(signing_cert)
+
             self.exec_command('openssl req -key %(signing_key)s -new -nodes '
                               '-out %(request_file)s -config %(ssl_config)s '
-                              '-subj %(default_subject)s')
+                              '-subj %(cert_subject)s')
+
             self.exec_command('openssl ca -batch -out %(signing_cert)s '
-                              '-config %(ssl_config)s '
+                              '-config %(ssl_config)s -days %(valid_days)dd '
+                              '-cert %(ca_cert)s -keyfile %(ca_private_key)s '
                               '-infiles %(request_file)s')
 
     def run(self):
@@ -149,13 +151,41 @@ class ConfigurePKI(object):
         self.build_private_key()
         self.build_signing_cert()
 
-    sslconfig = """
+
+class ConfigurePKI(BaseCertificateConfigure):
+    """Generate files for PKI signing using OpenSSL.
+
+    Signed tokens require a private key and signing certificate which itself
+    must be signed by a CA.  This class generates them with workable defaults
+    if each of the files are not present
+
+    """
+
+    def __init__(self, keystone_user, keystone_group):
+        super(ConfigurePKI, self).__init__(CONF.signing,
+                                           keystone_user, keystone_group)
+
+
+class ConfigureSSL(BaseCertificateConfigure):
+    """Generate files for HTTPS using OpenSSL.
+
+    Creates a public/private key and certificates. If a CA is not given
+    one will be generated using provided arguments.
+    """
+
+    def __init__(self, keystone_user, keystone_group):
+        super(ConfigureSSL, self).__init__(CONF.ssl,
+                                           keystone_user, keystone_group)
+
+
+BaseCertificateConfigure.sslconfig = """
 # OpenSSL configuration file.
 #
 
 # Establish working directory.
 
 dir            = %(conf_dir)s
+
 [ ca ]
 default_ca        = CA_default
 
@@ -163,55 +193,56 @@ default_ca        = CA_default
 new_certs_dir     = $dir
 serial            = $dir/serial
 database          = $dir/index.txt
-certificate       = %(ca_cert)s
-private_key       = %(ca_private_key)s
 default_days      = 365
-default_md        = md5
+default_md        = sha1
 preserve          = no
 email_in_dn       = no
 nameopt           = default_ca
 certopt           = default_ca
-policy            = policy_match
-[ policy_match ]
-countryName             = match
-stateOrProvinceName     = match
-organizationName        = match
+policy            = policy_anything
+x509_extensions   = usr_cert
+unique_subject    = no
+
+[ policy_anything ]
+countryName             = optional
+stateOrProvinceName     = optional
+organizationName        = optional
 organizationalUnitName  = optional
 commonName              = supplied
 emailAddress            = optional
 
 [ req ]
-default_bits        = 1024 # Size of keys
-default_keyfile        = key.pem # name of generated keys
-default_md        = md5 # message digest algorithm
+default_bits       = 1024 # Size of keys
+default_keyfile    = key.pem # name of generated keys
+default_md         = default # message digest algorithm
 string_mask        = nombstr # permitted characters
-distinguished_name    = req_distinguished_name
-req_extensions        = v3_req
+distinguished_name = req_distinguished_name
+req_extensions     = v3_req
 
 [ req_distinguished_name ]
-0.organizationName    = Organization Name (company)
-organizationalUnitName    = Organizational Unit Name (department, division)
-emailAddress        = Email Address
-emailAddress_max    = 40
-localityName        = Locality Name (city, district)
-stateOrProvinceName    = State or Province Name (full name)
-countryName        = Country Name (2 letter code)
-countryName_min        = 2
-countryName_max        = 2
-commonName        = Common Name (hostname, IP, or your name)
-commonName_max        = 64
-# Default values for the above, for consistency and less typing.
-0.organizationName_default = Openstack, Inc
-localityName_default = Undefined
-stateOrProvinceName_default = Undefined
-countryName_default = US
-commonName_default = %(ca_cert_cn)s
+0.organizationName          = Organization Name (company)
+organizationalUnitName      = Organizational Unit Name (department, division)
+emailAddress                = Email Address
+emailAddress_max            = 40
+localityName                = Locality Name (city, district)
+stateOrProvinceName         = State or Province Name (full name)
+countryName                 = Country Name (2 letter code)
+countryName_min             = 2
+countryName_max             = 2
+commonName                  = Common Name (hostname, IP, or your name)
+commonName_max              = 64
 
 [ v3_ca ]
-basicConstraints = CA:TRUE
-subjectKeyIdentifier = hash
+basicConstraints       = CA:TRUE
+subjectKeyIdentifier   = hash
 authorityKeyIdentifier = keyid:always,issuer:always
 
 [ v3_req ]
-basicConstraints = CA:FALSE
-subjectKeyIdentifier = hash"""
+basicConstraints     = CA:FALSE
+subjectKeyIdentifier = hash
+
+[ usr_cert ]
+basicConstraints       = CA:FALSE
+subjectKeyIdentifier   = hash
+authorityKeyIdentifier = keyid:always,issuer:always
+"""
diff --git a/tests/test_cert_setup.py b/tests/test_cert_setup.py
index 951b06ad..fb7326b3 100644
--- a/tests/test_cert_setup.py
+++ b/tests/test_cert_setup.py
@@ -46,8 +46,15 @@ class CertSetupTestCase(test.TestCase):
         super(CertSetupTestCase, self).setUp()
         CONF.signing.certfile = os.path.join(CERTDIR, 'signing_cert.pem')
         CONF.signing.ca_certs = os.path.join(CERTDIR, "ca.pem")
+        CONF.signing.ca_key = os.path.join(CERTDIR, "cakey.pem")
         CONF.signing.keyfile = os.path.join(KEYDIR, "signing_key.pem")
 
+        CONF.ssl.ca_certs = CONF.signing.ca_certs
+        CONF.ssl.ca_key = CONF.signing.ca_key
+
+        CONF.ssl.certfile = os.path.join(CERTDIR, 'keystone.pem')
+        CONF.ssl.keyfile = os.path.join(KEYDIR, 'keystonekey.pem')
+
         self.load_backends()
         self.load_fixtures(default_fixtures)
         self.controller = token.controllers.Auth()
@@ -72,13 +79,20 @@ class CertSetupTestCase(test.TestCase):
                           self.controller.authenticate,
                           {}, body_dict)
 
-    def test_create_certs(self):
-        ssl = openssl.ConfigurePKI(None, None)
-        ssl.run()
+    def test_create_pki_certs(self):
+        pki = openssl.ConfigurePKI(None, None)
+        pki.run()
         self.assertTrue(os.path.exists(CONF.signing.certfile))
         self.assertTrue(os.path.exists(CONF.signing.ca_certs))
         self.assertTrue(os.path.exists(CONF.signing.keyfile))
 
+    def test_create_ssl_certs(self):
+        ssl = openssl.ConfigureSSL(None, None)
+        ssl.run()
+        self.assertTrue(os.path.exists(CONF.ssl.ca_certs))
+        self.assertTrue(os.path.exists(CONF.ssl.certfile))
+        self.assertTrue(os.path.exists(CONF.ssl.keyfile))
+
     def tearDown(self):
         try:
             shutil.rmtree(rootdir(SSLDIR))