Proposing merge to get feedback on orm refactoring. I am very interested in feedback to all of these changes.

This is a huge set of changes, that touches almost all of the files.  I'm sure I have broken quite a bit, but better to take the plunge now than to postpone this until later.  The idea is to allow for pluggable backends throughout the code.

Brief Overview
For compute/volume/network, there are multiple classes
service - responsible for rpc
  this currently uses the existing cast and call in rpc.py and a little bit of magic
  to call public methods on the manager class.
  each service also reports its state into the database every 10 seconds
manager - responsible for managing respective object classes
  all the business logic for the classes go here
db (db_driver) - responsible for abstracting database access
driver (domain_driver) - responsible for executing actual shell commands and implementation

Compute hasn't been fully cleaned up, but to get an idea of how it works, take a look
at volume and network

Known issues/Things to be done:

* nova-api accesses db objects directly
  It seems cleaner to have only the managers dealing with their respective objects.  This would
  mean code for 'run_instances' would move into the manager class and it would do the initial
  setup and cast out to the remote service

* db code uses flat methods to define its interface
  In my mind this is a little prettier as an abstract base class, but driver loading code
  can load a module or a class.  It works, so I'm not sure it needs to be changed but feel
  free to debate it.

* Service classes have no code in them
  Not sure if this is a problem for people, but the magic of calling the manager's methods is
  done in the base class.  We could remove the magic from the base class and explicitly
  wrap methods that we want to make available via rpc if this seems nasty.

* AuthManager Projects/Users/Roles are not integrated into this system.
  In order for everything to live happily in the backend, we need some type
  of adaptor for LDAP

* Context is not passed properly across rabbit
  Context should probably be changed to a simple dictionary so that it can be
  passed properly through the queue

* No authorization checks on access to objects
  We need to decide on which layer auth checks should happen.

* Some of the methods in ComputeManager need to be moved into other layers/managers
* Compute driver layer should be abstracted more cleanly
* Flat networking is untested and may need to be reworked
* Some of the api commands are not working yet
* Nova Swift Authentication needs to be refactored(Todd is working on this)

1 files changed, 33 insertions, 14 deletions

diff --git a/nova/auth/manager.py b/nova/auth/manager.py
index 284b29502..d5fbec7c5 100644
--- a/nova/auth/manager.py
+++ b/nova/auth/manager.py
@@ -29,11 +29,11 @@ import uuid
 import zipfile
 
 from nova import crypto
+from nova import db
 from nova import exception
 from nova import flags
 from nova import utils
 from nova.auth import signer
-from nova.network import vpn
 
 
 FLAGS = flags.FLAGS
@@ -252,6 +252,7 @@ class AuthManager(object):
         __init__ is run every time AuthManager() is called, so we only
         reset the driver if it is not set or a new driver is specified.
         """
+        self.network_manager = utils.import_object(FLAGS.network_manager)
         if driver or not getattr(self, 'driver', None):
             self.driver = utils.import_class(driver or FLAGS.auth_driver)
 
@@ -493,8 +494,8 @@ class AuthManager(object):
                 return []
             return [Project(**project_dict) for project_dict in project_list]
 
-    def create_project(self, name, manager_user,
-                       description=None, member_users=None):
+    def create_project(self, name, manager_user, description=None,
+                       member_users=None, context=None):
         """Create a project
 
         @type name: str
@@ -523,7 +524,14 @@ class AuthManager(object):
                                               description,
                                               member_users)
             if project_dict:
-                return Project(**project_dict)
+                project = Project(**project_dict)
+                try:
+                    self.network_manager.allocate_network(context,
+                                                          project.id)
+                except:
+                    drv.delete_project(project.id)
+                    raise
+                return project
 
     def add_to_project(self, user, project):
         """Add user to project"""
@@ -550,7 +558,7 @@ class AuthManager(object):
                                             Project.safe_id(project))
 
     @staticmethod
-    def get_project_vpn_data(project):
+    def get_project_vpn_data(project, context=None):
         """Gets vpn ip and port for project
 
         @type project: Project or project_id
@@ -560,15 +568,26 @@ class AuthManager(object):
         @return: A tuple containing (ip, port) or None, None if vpn has
         not been allocated for user.
         """
-        network_data = vpn.NetworkData.lookup(Project.safe_id(project))
-        if not network_data:
+
+        network_ref = db.project_get_network(context,
+                                             Project.safe_id(project))
+
+        if not network_ref['vpn_public_port']:
             raise exception.NotFound('project network data has not been set')
-        return (network_data.ip, network_data.port)
+        return (network_ref['vpn_public_address'],
+                network_ref['vpn_public_port'])
 
-    def delete_project(self, project):
+    def delete_project(self, project, context=None):
         """Deletes a project"""
+        try:
+            network_ref = db.project_get_network(context,
+                                                 Project.safe_id(project))
+            db.network_destroy(context, network_ref['id'])
+        except:
+            logging.exception('Could not destroy network for %s',
+                              project)
         with self.driver() as drv:
-            return drv.delete_project(Project.safe_id(project))
+            drv.delete_project(Project.safe_id(project))
 
     def get_user(self, uid):
         """Retrieves a user by id"""
@@ -703,15 +722,15 @@ class AuthManager(object):
         zippy.writestr(FLAGS.credential_key_file, private_key)
         zippy.writestr(FLAGS.credential_cert_file, signed_cert)
 
-        network_data = vpn.NetworkData.lookup(pid)
-        if network_data:
+        (vpn_ip, vpn_port) = self.get_project_vpn_data(project)
+        if vpn_ip:
             configfile = open(FLAGS.vpn_client_template, "r")
             s = string.Template(configfile.read())
             configfile.close()
             config = s.substitute(keyfile=FLAGS.credential_key_file,
                                   certfile=FLAGS.credential_cert_file,
-                                  ip=network_data.ip,
-                                  port=network_data.port)
+                                  ip=vpn_ip,
+                                  port=vpn_port)
             zippy.writestr(FLAGS.credential_vpn_file, config)
         else:
             logging.warn("No vpn data for project %s" %