merge trunkdoc

15 files changed, 224 insertions, 254 deletions

diff --git a/doc/source/auth.rst b/doc/source/auth.rst
index adcb759a2..0230ec5c4 100644
--- a/doc/source/auth.rst
+++ b/doc/source/auth.rst
@@ -17,19 +17,22 @@
 
 .. _auth:
 
-Auth Documentation
-==================
-
-from etherpad todo
-------------------
+Authentication and Authorization Development Guide
+==================================================
 
 ::
 
-  * Manager
-  * Driver
-    * ldap
-    * fakeldap
-    * redisectomy
+    TOOD(todd);
+        * Authentication
+            * Manager
+            * Driver
+                * ldap
+                * fakeldap
+                * redisectomy
+        * Authorizztion
+            * RBAC
+            * Quotas
+
 
 legacy docs
 -----------
diff --git a/doc/source/compute.rst b/doc/source/compute.rst
index 93070fa88..9b4d10227 100644
--- a/doc/source/compute.rst
+++ b/doc/source/compute.rst
@@ -16,71 +16,116 @@
       under the License.
 
 
-from etherpad todo
-------------------
+Virtualization Programming Guide
+================================
 
-  * FIX RST IMPORT ERRORS
-  * Compute Manager
-    * public methods
-    * responsibilities
-  * Compute Driver
-    * public methods
-    * libvirt implementation (kvm/qemu vs uml)
-    * xen implementation
-    * hyperv implementation
+This page contains the Compute Package documentation.
 
 
-Compute Documentation
-=====================
+::
 
-This page contains the Compute Package documentation.
+    TODO(todd): Document drivers
+
+
+Manager
+-------
+
+Documentation for the compute manager and related files.  For reading about
+a specific virtualization backend, read Drivers_.
 
 
 The :mod:`manager` Module
--------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. automodule:: nova.compute.manager
     :members:
     :undoc-members:
     :show-inheritance:
 
+
 The :mod:`disk` Module
-----------------------
+~~~~~~~~~~~~~~~~~~~~~~
 
 .. automodule:: nova.compute.disk
     :members:
     :undoc-members:
     :show-inheritance:
 
+
 The :mod:`instance_types` Module
---------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. automodule:: nova.compute.instance_types
     :members:
     :undoc-members:
     :show-inheritance:
 
+
 The :mod:`power_state` Module
------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. automodule:: nova.compute.power_state
     :members:
     :undoc-members:
     :show-inheritance:
 
+
+Drivers
+-------
+
+
+Libvirt Implementations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+
+Libvirt: KVM
+^^^^^^^^^^^^
+
+KVM Driver
+
+
+Libvirt: QEMU
+^^^^^^^^^^^^^
+
+QEMU Driver
+
+
+Libvirt: UML
+^^^^^^^^^^^^
+
+User Mode Linux Driver
+
+
+XEN
+~~~
+
+Xen Driver
+
+
+Hyper-V
+~~~~~~~
+
+Hyper-V Driver
+
+
+Monitoring
+----------
+
 The :mod:`monitor` Module
--------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. automodule:: nova.compute.monitor
     :members:
     :undoc-members:
     :show-inheritance:
 
-RELATED TESTS
--------------
+
+Tests
+-----
+
 
 The :mod:`compute_unittest` Module
-----------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. automodule:: nova.tests.compute_unittest
     :members:
diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst
index 7f59c0c92..29b8f064a 100644
--- a/doc/source/concepts.and.introduction.rst
+++ b/doc/source/concepts.and.introduction.rst
@@ -66,11 +66,24 @@ Concept: Instances
 
 An 'instance' is a word for a virtual machine that runs inside the cloud.
 
-Concept: Volumes
+Concept: Storage
 ----------------
 
+Volumes
+~~~~~~~
 A 'volume' is a detachable block storage device.  You can think of it as a usb hard drive.  It can only be attached to one instance at a time, and it behaves
 
+Ephemeral
+~~~~~~~~~
+::
+
+    TODO(vish): document
+
+Swift
+~~~~~
+::
+
+    TODO(vish): document
 
 Concept: Quotas
 ---------------
@@ -104,17 +117,17 @@ Nova has a concept of Fixed Ips and Floating ips.  Fixed ips are assigned to an
 There are multiple strategies available for implementing fixed ips:
 
 Flat Mode
-^^^^^^^^^
+~~~~~~~~~
 
 The simplest networking mode.  Each instance receives a fixed ip from the pool.  All instances are attached to the same bridge (br100) by default.  The bridge must be configured manually.  The networking configuration is injected into the instance before it is booted.  Note that this currently only works on linux-style systems that keep networking configuration in /etc/network/interfaces.
 
 Flat DHCP Mode
-^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~
 
 This is similar to the flat mode, in that all instances are attached to the same bridge.  In this mode nova does a bit more configuration, it will attempt to bridge into an ethernet device (eth0 by default).  It will also run dnsmasq as a dhcpserver listening on this bridge.  Instances receive their fixed ips by doing a dhcpdiscover.
 
 VLAN DHCP Mode
-^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~
 
 This is the default networking mode and supports the most features.  For multiple machine installation, it requires a switch that supports host-managed vlan tagging.  In this mode, nova will create a vlan and bridge for each project.  The project gets a range of private ips that are only accessible from inside the vlan.  In order for a user to access the instances in their project, a special vpn instance (code name cloudpipe) needs to be created.  Nova generates a certificate and key for the userto access the vpn and starts the vpn automatically.
 
@@ -136,7 +149,7 @@ Concept: nova-manage
 --------------------
 
 Introduction
-^^^^^^^^^^^^
+~~~~~~~~~~~~
 
 The nova-manage command is used to perform many essential functions for
 administration and ongoing maintenance of nova, such as user creation,
@@ -151,7 +164,7 @@ For example, to obtain a list of all projects:
 ``nova-manage project list``
 
 User Maintenance
-^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~
 
 * user admin: creates a new admin and prints exports
     * arguments: name [access] [secret]
@@ -168,7 +181,7 @@ User Maintenance
     *  leave any field blank to ignore it, admin should be 'T', 'F', or blank
 
 Project Maintenance
-^^^^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~~~~
 
 * project add: Adds user to project
     * arguments: project user
@@ -190,7 +203,7 @@ Project Maintenance
     * arguments: project_id user_id [filename='nova.zip]
 
 User Role Management
-^^^^^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~~~~~
 
 * role add: adds role to user
     * if project is specified, adds project specific role
@@ -205,7 +218,7 @@ User Role Management
 
 
 Nova Shell
-^^^^^^^^^^
+~~~~~~~~~~
 
 * shell bpython
     * start a new bpython shell
@@ -219,7 +232,7 @@ Nova Shell
     * arguments: path
 
 VPN Management
-^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~
 
 * vpn list: Print a listing of the VPNs for all projects.
     * arguments: none
@@ -230,7 +243,7 @@ VPN Management
 
 
 Floating IP Management
-^^^^^^^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~~~~~~~
 
 * floating create: Creates floating ips for host by range
     * arguments: host ip_range
@@ -240,7 +253,7 @@ Floating IP Management
     * arguments: none
 
 Network Management
-^^^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~~~
 
 * network create: Creates fixed ips for host by range
     * arguments: [fixed_range=FLAG], [num_networks=FLAG],
diff --git a/doc/source/objectstore.rst b/doc/source/database.rst
index 6b8d293f4..41b362d19 100644
--- a/doc/source/objectstore.rst
+++ b/doc/source/database.rst
@@ -15,51 +15,37 @@
       License for the specific language governing permissions and limitations
       under the License.
 
-Objectstore Documentation
-=========================
+Database Programming Guide
+==========================
 
-This page contains the Objectstore Package documentation.
+::
 
+    TODO(todd): should sqlalchemy.api be here?
+                pep-256 on db/api.py and models.py (periods)
+                document register_models (where should it be called from?)
 
-The :mod:`bucket` Module
-------------------------
+The :mod:`api` Module
+---------------------
 
-.. automodule:: nova.objectstore.bucket
+.. automodule:: nova.db.api
     :members:
     :undoc-members:
     :show-inheritance:
 
-The :mod:`handler` Module
--------------------------
 
-.. automodule:: nova.objectstore.handler
-    :members:
-    :undoc-members:
-    :show-inheritance:
+Drivers
+-------
 
-The :mod:`image` Module
------------------------
+The :mod:`sqlalchemy` Driver
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-.. automodule:: nova.objectstore.image
+.. automodule:: nova.db.sqlalchemy.api
     :members:
     :undoc-members:
     :show-inheritance:
 
-The :mod:`stored` Module
-------------------------
-
-.. automodule:: nova.objectstore.stored
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-RELATED TESTS
--------------
-
-The :mod:`objectstore_unittest` Module
---------------------------------------
 
-.. automodule:: nova.tests.objectstore_unittest
+.. automodule:: nova.db.sqlalchemy.models
     :members:
     :undoc-members:
     :show-inheritance:
diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst
index 2c8fa86f9..d9a7c46f9 100644
--- a/doc/source/development.guide.rst
+++ b/doc/source/development.guide.rst
@@ -28,7 +28,7 @@ Contents
    :maxdepth: 1
 
    database
-   storage
+   volume
    compute
    network      
    auth  
@@ -41,7 +41,6 @@ Older Contents
    :maxdepth: 1
 
    architecture
-   endpoint
    nova
    fakes
    binaries
@@ -49,6 +48,11 @@ Older Contents
    packages
 
 
+Removed
+-------
+
+* endpoint
+
 Indices and tables
 ------------------
 
diff --git a/doc/source/endpoint.rst b/doc/source/endpoint.rst
deleted file mode 100644
index 399df4161..000000000
--- a/doc/source/endpoint.rst
+++ /dev/null
@@ -1,91 +0,0 @@
-..
-      Copyright 2010 United States Government as represented by the
-      Administrator of the National Aeronautics and Space Administration. 
-      All Rights Reserved.
-
-      Licensed under the Apache License, Version 2.0 (the "License"); you may
-      not use this file except in compliance with the License. You may obtain
-      a copy of the License at
-
-          http://www.apache.org/licenses/LICENSE-2.0
-
-      Unless required by applicable law or agreed to in writing, software
-      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-      License for the specific language governing permissions and limitations
-      under the License.
-
-Endpoint Documentation
-======================
-
-This page contains the Endpoint Package documentation.
-
-The :mod:`admin` Module
------------------------
-
-.. automodule:: nova.endpoint.admin
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-The :mod:`api` Module
----------------------
-
-.. automodule:: nova.endpoint.api
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-The :mod:`cloud` Module
------------------------
-
-.. automodule:: nova.endpoint.cloud
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-The :mod:`images` Module
-------------------------
-
-.. automodule:: nova.endpoint.images
-    :members:
-    :undoc-members:
-    :show-inheritance:
-    
-    
-RELATED TESTS
---------------
-
-The :mod:`api_unittest` Module
-------------------------------
-
-.. automodule:: nova.tests.api_unittest
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-The :mod:`api_integration` Module
----------------------------------
-
-.. automodule:: nova.tests.api_integration
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-The :mod:`cloud_unittest` Module
---------------------------------
-
-.. automodule:: nova.tests.cloud_unittest
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-The :mod:`network_unittest` Module
-----------------------------------
-
-.. automodule:: nova.tests.network_unittest
-    :members:
-    :undoc-members:
-    :show-inheritance:
-
-
diff --git a/doc/source/fakes.rst b/doc/source/fakes.rst
index a993fb4c8..eccd4ab43 100644
--- a/doc/source/fakes.rst
+++ b/doc/source/fakes.rst
@@ -19,7 +19,7 @@ Nova Fakes
 ==========
 
 The :mod:`virt.fake` Module
---------------------------
+---------------------------
 
 .. automodule:: nova.virt.fake
     :members:
diff --git a/doc/source/index.rst b/doc/source/index.rst
index d4cdd1408..54cfea05a 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -30,7 +30,9 @@ Nova is written with the following design guidelines in mind:
 
 This documentation is generated by the Sphinx toolkit and lives in the source
 tree.  Additional documentation on Nova and other components of OpenStack can
-be found on the 'OpenStack wiki <http://wiki.openstack.org>'.
+be found on the `OpenStack wiki`_.
+
+.. _`OpenStack wiki`: http://wiki.openstack.org
 
 
 Contents
diff --git a/doc/source/modules.rst b/doc/source/modules.rst
index 82c61f008..9b31664b6 100644
--- a/doc/source/modules.rst
+++ b/doc/source/modules.rst
@@ -28,7 +28,11 @@ Modules:
 
    auth
    compute
-   endpoint
    fakes
    nova
    volume
+
+Removed:
+--------
+
+* endpoint
diff --git a/doc/source/network.rst b/doc/source/network.rst
index b776ae40e..f53360bd7 100644
--- a/doc/source/network.rst
+++ b/doc/source/network.rst
@@ -15,23 +15,32 @@
       License for the specific language governing permissions and limitations
       under the License.
 
-nova Networking
-================
+Networking Programming Guide
+============================
 
-
-from etherpad todo
-------------------
 ::
-  * Network Manager
-    * overview (use vish email)
-    * public methods
-    * responsibilities
-    * Vlan + DHCP implementation
-    * Flat + DHCP implementation
-    * Flat impleplementation
-  * Network Driver
-    * public methods
-    * hardware specific commands
+
+    TODO(todd): 
+        * document how managers work
+        * document backend (linux-net)
+        * document hardware specific commands
+        * pep 257
+
+The :mod:`Manager` module
+-------------------------
+
+.. automodule:: nova.network.manager
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+The :mod:`LinuxNet` driver
+--------------------------
+
+.. automodule:: nova.network.linux_net
+    :members:
+    :undoc-members:
+    :show-inheritance:
 
 
 legacy docs
diff --git a/doc/source/storage.rst b/doc/source/storage.rst
deleted file mode 100644
index 72880a4a1..000000000
--- a/doc/source/storage.rst
+++ /dev/null
@@ -1,47 +0,0 @@
-..
-      Copyright 2010 United States Government as represented by the
-      Administrator of the National Aeronautics and Space Administration. 
-      All Rights Reserved.
-
-      Licensed under the Apache License, Version 2.0 (the "License"); you may
-      not use this file except in compliance with the License. You may obtain
-      a copy of the License at
-
-          http://www.apache.org/licenses/LICENSE-2.0
-
-      Unless required by applicable law or agreed to in writing, software
-      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-      License for the specific language governing permissions and limitations
-      under the License.
-
-Storage in the Nova Cloud
-=========================
-
-
-from etherpad todo
--------------------
-::
-    * Volume Manager
-      * public methods
-      * responsibilities
-    * Volume Driver
-      * public methods
-      * AoE implementation
-      * ISCSI implementation
-
-
-legacy docs
------------
-
-There are three primary classes of storage in a nova cloud environment:
-
-* Ephemeral Storage (local disk within an instance)
-* Volume Storage (network-attached FS)
-* Object Storage (redundant KVS with locality and MR)
-
-.. toctree::
-   :maxdepth: 2
-   
-   volume
-   objectstore      
diff --git a/doc/source/volume.rst b/doc/source/volume.rst
index 619968458..f4162560c 100644
--- a/doc/source/volume.rst
+++ b/doc/source/volume.rst
@@ -15,31 +15,48 @@
       License for the specific language governing permissions and limitations
       under the License.
 
-Volume Documentation
-====================
-                           
-Nova uses ata-over-ethernet (AoE) to export storage volumes from multiple storage nodes. These AoE exports are attached (using libvirt) directly to running instances.
+Volume Programming Guide
+=========================
 
-Nova volumes are exported over the primary system VLAN (usually VLAN 1), and not over individual VLANs.
+::
 
-AoE exports are numbered according to a "shelf and blade" syntax. In order to avoid collisions, we currently perform an AoE-discover of existing exports, and then grab the next unused number. (This obviously has race condition problems, and should be replaced by allocating a shelf-id to each storage node.)
+    TODO(todd): document for iSCSI & AoE (see 'Old Docs')
+                pep-257
 
-The underlying volumes are LVM logical volumes, created on demand within a single large volume group. 
 
+The :mod:`nova.volume.manager` Module
+-------------------------------------
 
-The :mod:`storage` Module
--------------------------
+.. automodule:: nova.volume.manager
+    :members:
+    :undoc-members:
+    :show-inheritance:
 
-.. automodule:: nova.volume.storage
+The :mod:`nova.volume.driver` Module
+-------------------------------------
+
+.. automodule:: nova.volume.driver
     :members:
     :undoc-members:
     :show-inheritance:
 
-The :mod:`storage_unittest` Module
-----------------------------------
+Tests
+-----
 
-.. automodule:: nova.tests.storage_unittest
+.. automodule:: nova.tests.volume_unittest
     :members:
     :undoc-members:
     :show-inheritance:
 
+Old Docs
+--------
+
+Nova uses ata-over-ethernet (AoE) to export storage volumes from multiple storage nodes. These AoE exports are attached (using libvirt) directly to running instances.
+
+Nova volumes are exported over the primary system VLAN (usually VLAN 1), and not over individual VLANs.
+
+AoE exports are numbered according to a "shelf and blade" syntax. In order to avoid collisions, we currently perform an AoE-discover of existing exports, and then grab the next unused number. (This obviously has race condition problems, and should be replaced by allocating a shelf-id to each storage node.)
+
+The underlying volumes are LVM logical volumes, created on demand within a single large volume group. 
+
+
diff --git a/nova/auth/fakeldap.py b/nova/auth/fakeldap.py
index 1a49b73fe..176a00f06 100644
--- a/nova/auth/fakeldap.py
+++ b/nova/auth/fakeldap.py
@@ -15,12 +15,14 @@
 #    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 #    License for the specific language governing permissions and limitations
 #    under the License.
+
 """
-Fake LDAP server for test harnesses.
+Fake LDAP server for test harness, backs to ReDIS.
 
 This class does very little error checking, and knows nothing about ldap
-class definitions. It implements the minimum emulation of the python ldap
+class definitions.  It implements the minimum emulation of the python ldap
 library to work with nova.
+
 """
 
 import json
@@ -77,8 +79,8 @@ def initialize(_uri):
 def _match_query(query, attrs):
     """Match an ldap query to an attribute dictionary.
 
-    &, |, and ! are supported in the query. No syntax checking is performed,
-    so malformed querys will not work correctly.
+    The characters &, |, and ! are supported in the query. No syntax checking
+    is performed, so malformed querys will not work correctly.
     """
     # cut off the parentheses
     inner = query[1:-1]
diff --git a/nova/compute/manager.py b/nova/compute/manager.py
index 574feec7c..3346d1299 100644
--- a/nova/compute/manager.py
+++ b/nova/compute/manager.py
@@ -17,7 +17,20 @@
 #    under the License.
 
 """
-Handles all code relating to instances (guest vms)
+Handles all processes relating to instances (guest vms).
+
+The :py:class:`ComputeManager` class is a :py:class:`nova.manager.Manager` that
+handles RPC calls relating to creating instances.  It is responsible for
+building a disk image, launching it via the underlying virtualization driver,
+responding to calls to check it state, attaching persistent as well as termination.
+
+**Related Flags**
+
+:instances_path:  Where instances are kept on disk
+:compute_driver:  Name of class that is used to handle virtualization, loaded
+                  by :func:`nova.utils.import_object`
+:volume_manager:  Name of class that handles persistent storage, loaded by
+                  :func:`nova.utils.import_object`
 """
 
 import datetime
@@ -40,12 +53,12 @@ flags.DEFINE_string('compute_driver', 'nova.virt.connection.get_connection',
 
 
 class ComputeManager(manager.Manager):
-    """
-    Manages the running instances.
-    """
+    """Manages the running instances from creation to destruction."""
+
     def __init__(self, compute_driver=None, *args, **kwargs):
         """Load configuration options and connect to the hypervisor."""
         # TODO(vish): sync driver creation logic with the rest of the system
+        #             and redocument the module docstring
         if not compute_driver:
             compute_driver = FLAGS.compute_driver
         self.driver = utils.import_object(compute_driver)
@@ -54,7 +67,7 @@ class ComputeManager(manager.Manager):
         super(ComputeManager, self).__init__(*args, **kwargs)
 
     def _update_state(self, context, instance_id):
-        """Update the state of an instance from the driver info"""
+        """Update the state of an instance from the driver info."""
         # FIXME(ja): include other fields from state?
         instance_ref = self.db.instance_get(context, instance_id)
         try:
@@ -67,6 +80,7 @@ class ComputeManager(manager.Manager):
     @defer.inlineCallbacks
     @exception.wrap_exception
     def refresh_security_group(self, context, security_group_id, **_kwargs):
+        """This call passes stright through to the virtualization driver."""
         yield self.driver.refresh_security_group(security_group_id)
 
     @defer.inlineCallbacks
diff --git a/nova/virt/connection.py b/nova/virt/connection.py
index 34e37adf7..ceb7f1e4b 100644
--- a/nova/virt/connection.py
+++ b/nova/virt/connection.py
@@ -17,7 +17,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
-"""Abstraction of the underlying virtualization API"""
+"""Abstraction of the underlying virtualization API."""
 
 import logging
 import sys
@@ -39,6 +39,15 @@ def get_connection(read_only=False):
 
     Any object returned here must conform to the interface documented by
     FakeConnection.
+
+    Related flags
+    -------------
+    :connection_type:  A string literal that falls through a if/elif structure
+                       to determine what virtualization mechanism to use.
+                       Values may be:
+                            * fake
+                            * libvirt
+                            * xenapi
     """
     # TODO(termie): maybe lazy load after initial check for permissions
     # TODO(termie): check whether we can be disconnected