From 8e12d65af48e69f6ad211c6b880a3162e1d03e78 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Wed, 27 Oct 2010 00:05:42 -0400 Subject: New structure for documentation. --- doc/source/administration.guide.rst | 31 ++++++++ doc/source/concepts.and.introduction.rst | 123 +++++++++++++++++++++++++++++++ doc/source/development.guide.rst | 49 ++++++++++++ doc/source/index.rst | 52 ++++++------- 4 files changed, 226 insertions(+), 29 deletions(-) create mode 100644 doc/source/administration.guide.rst create mode 100644 doc/source/concepts.and.introduction.rst create mode 100644 doc/source/development.guide.rst (limited to 'doc/source') diff --git a/doc/source/administration.guide.rst b/doc/source/administration.guide.rst new file mode 100644 index 000000000..9bea5e2dd --- /dev/null +++ b/doc/source/administration.guide.rst @@ -0,0 +1,31 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Administration Guide +==================== + +How to deploy, monitor, and debug Nova. + + +Contents +-------- + +.. toctree:: + :maxdepth: 1 + + getting.started + diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst new file mode 100644 index 000000000..fc4ebed21 --- /dev/null +++ b/doc/source/concepts.and.introduction.rst @@ -0,0 +1,123 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Concepts and Introduction +========================= + + +Introduction +------------ + +Nova is the software that controls your Infrastructure as as Service (IaaS) +cloud computing platform. It is similar in scope to Amazon EC2 and Rackspace +CloudServers. Nova does not include any virtualization software, rather it +defines drivers that interact with underlying virtualization mechanisms that +run on your host operating system, and exposes functionality over a web API. + +This document does not attempt to explain fundamental concepts of cloud +computing, IaaS, virtualization, or other related technologies. Instead, it +focues on describing how Nova's implementation of those concepts is achieved. + + +Concept: Projects +----------------- + +Projects + + +Concept: Virtualization +----------------------- + +* KVM +* UML +* XEN +* HyperV + + +Concept: Volumes +---------------- + +Volumes + + +Concept: Quotas +--------------- + +* Defaults +* Override for project + + +Concept: API +------------ + +* EC2 +* OpenStack / Rackspace + + +Concept: Networking +------------------- + +* Simple networking +* NASA-Style Networking + + +Concept: Services +----------------- + +Services + + +Concept: Flags +-------------- + +python-gflags + + +Concept: Plugins +---------------- + +LazyPluggable + + +Concept: IPC +------------ + +Rabbit! + + +Concept: Fakes +-------------- + +Fakes + + +Concept: Scheduler +------------------ + +Scheduler + + +Concept: ORM +------------ + +ORM + + +Concept: Manager Pattern +------------------------ + +Managers diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst new file mode 100644 index 000000000..18b08d6fe --- /dev/null +++ b/doc/source/development.guide.rst @@ -0,0 +1,49 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Nova Development Guide +====================== + +Nova is written in python + + +Contents +-------- + +.. toctree:: + :maxdepth: 1 + + architecture + network + storage + auth + compute + endpoint + nova + fakes + binaries + modules + packages + + +Indices and tables +------------------ + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` + diff --git a/doc/source/index.rst b/doc/source/index.rst index 1109e9011..d4cdd1408 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -18,37 +18,31 @@ Welcome to Nova's documentation! ================================ -Nova is a cloud computing fabric controller (the main part of an IaaS system). -It is written in Python and relies on the standard AMQP messaging protocol, uses the Twisted framework, -and optionally uses the Redis distributed key value store for authorization. - -Nova is intended to be easy to extend and adapt. For example, authentication and authorization -requests by default use an RDBMS-backed datastore driver. However, there is already support -for using LDAP backing authentication (slapd) and if you wish to "fake" LDAP, there is a module -available that uses ReDIS to store authentication information in an LDAP-like backing datastore. -It has extensive test coverage, and uses the Sphinx toolkit (the same as Python itself) for code -and developer documentation. Additional documentation is available on the -'OpenStack wiki '_. -While Nova is currently in Beta use within several organizations, the codebase -is very much under active development - please test it and log bugs! - -Contents: +Nova is a cloud computing fabric controller, the main part of an IaaS system. + +Nova is written with the following design guidelines in mind: + +* Component based architecture: Quickly add new behaviors +* Highly available: Scale to very serious workloads +* Fault-Tollerant: Isloated processes avoid cascading failures +* Recoverable: Failures should be easy to diagnose, debug, and rectify +* Open Standards: Be a reference implementation for a community-driven api + +This documentation is generated by the Sphinx toolkit and lives in the source +tree. Additional documentation on Nova and other components of OpenStack can +be found on the 'OpenStack wiki '. + + +Contents +-------- .. toctree:: - :maxdepth: 2 - - getting.started - architecture - network - storage - auth - compute - endpoint - nova - fakes - binaries - modules - packages + :maxdepth: 1 + + concepts.and.introduction + administration.guide + development.guide + Indices and tables ================== -- cgit From 32a75c0181524a0d369419495f9a6e3d67d0e1e2 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Wed, 27 Oct 2010 13:49:27 -0400 Subject: Doc updates. --- doc/source/administration.guide.rst | 6 ++++-- doc/source/concepts.and.introduction.rst | 25 ++++++++++++++++++++----- doc/source/flags.rst | 23 +++++++++++++++++++++++ doc/source/multi.node.install.rst | 21 +++++++++++++++++++++ doc/source/quickstart.rst | 21 +++++++++++++++++++++ 5 files changed, 89 insertions(+), 7 deletions(-) create mode 100644 doc/source/flags.rst create mode 100644 doc/source/multi.node.install.rst create mode 100644 doc/source/quickstart.rst (limited to 'doc/source') diff --git a/doc/source/administration.guide.rst b/doc/source/administration.guide.rst index 9bea5e2dd..4ff1afc9a 100644 --- a/doc/source/administration.guide.rst +++ b/doc/source/administration.guide.rst @@ -27,5 +27,7 @@ Contents .. toctree:: :maxdepth: 1 - getting.started - + quickstart + getting.started + multi.node.install + flags diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index fc4ebed21..701e43dfb 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -15,6 +15,7 @@ License for the specific language governing permissions and limitations under the License. + Concepts and Introduction ========================= @@ -32,9 +33,17 @@ This document does not attempt to explain fundamental concepts of cloud computing, IaaS, virtualization, or other related technologies. Instead, it focues on describing how Nova's implementation of those concepts is achieved. +This page outlines concepts that you will need to understand as a user or +administrator of an OpenStack installation. Each section links to more more +detailed information in the `Administration Guide`_, but you'll probably want +to read this section straight-through before tackling the specifics presented +in the administration guide. + +.. _`Administration Guide`: administration.guide.html -Concept: Projects ------------------ + +Concept: Users and Projects +--------------------------- Projects @@ -72,7 +81,7 @@ Concept: Networking ------------------- * Simple networking -* NASA-Style Networking +* VLAN Networking Concept: Services @@ -93,8 +102,8 @@ Concept: Plugins LazyPluggable -Concept: IPC ------------- +Concept: IPC/RPC +---------------- Rabbit! @@ -121,3 +130,9 @@ Concept: Manager Pattern ------------------------ Managers + + +Concept: Security Groups +------------------------ + +Security groups diff --git a/doc/source/flags.rst b/doc/source/flags.rst new file mode 100644 index 000000000..4c950aa88 --- /dev/null +++ b/doc/source/flags.rst @@ -0,0 +1,23 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Flags and Flagfiles +=================== + +* python-gflags +* flagfiles +* list of flags by component (see concepts list) diff --git a/doc/source/multi.node.install.rst b/doc/source/multi.node.install.rst new file mode 100644 index 000000000..443e2c4c3 --- /dev/null +++ b/doc/source/multi.node.install.rst @@ -0,0 +1,21 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Multi-Node Nova +=============== + +http://etherpad.openstack.org/NovaMultinodeInstall diff --git a/doc/source/quickstart.rst b/doc/source/quickstart.rst new file mode 100644 index 000000000..7a41d4be6 --- /dev/null +++ b/doc/source/quickstart.rst @@ -0,0 +1,21 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Nova Quickstart +========================= + +http://github.com/vishvananda/novascript -- cgit From 55a0cfc96b92bd050115244c850f4ad58791c83f Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Wed, 27 Oct 2010 15:52:06 -0400 Subject: Doc skeleton from collaborative etherpad hack session. --- doc/source/administration.guide.rst | 2 + doc/source/auth.rst | 15 +++++++ doc/source/compute.rst | 21 ++++++++++ doc/source/concepts.and.introduction.rst | 70 ++++++++++++++++++++++++-------- doc/source/development.guide.rst | 15 +++++-- doc/source/monitoring.rst | 27 ++++++++++++ doc/source/multi.node.install.rst | 4 ++ doc/source/network.rst | 19 +++++++++ doc/source/services.rst | 31 ++++++++++++++ doc/source/storage.rst | 16 ++++++++ 10 files changed, 201 insertions(+), 19 deletions(-) create mode 100644 doc/source/monitoring.rst create mode 100644 doc/source/services.rst (limited to 'doc/source') diff --git a/doc/source/administration.guide.rst b/doc/source/administration.guide.rst index 4ff1afc9a..789e97381 100644 --- a/doc/source/administration.guide.rst +++ b/doc/source/administration.guide.rst @@ -30,4 +30,6 @@ Contents quickstart getting.started multi.node.install + services flags + monitoring diff --git a/doc/source/auth.rst b/doc/source/auth.rst index 3fcb309cd..2b8ad7924 100644 --- a/doc/source/auth.rst +++ b/doc/source/auth.rst @@ -18,6 +18,21 @@ Auth Documentation ================== + +from etherpad todo +------------------ + +:: + * FIX RST IMPORT ERRORS + * Manager + * Driver + * ldap + * fakeldap + * redisectomy + +legacy docs +----------- + Nova provides RBAC (Role-based access control) of the AWS-type APIs. We define the following roles: Roles-Based Access Control of AWS-style APIs using SAML Assertions diff --git a/doc/source/compute.rst b/doc/source/compute.rst index 5b08dbd5b..76957f316 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -18,6 +18,27 @@ Compute Documentation ===================== + +from etherpad todo +------------------ + +:: + * FIX RST IMPORT ERRORS + * Compute Manager + * public methods + * responsibilities + * Compute Driver + * public methods + * libvirt implementation + - kvm/qemu + - uml + * xen implementation + * hyperv implementation + + +legacy docs +----------- + This page contains the Compute Package documentation. diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 701e43dfb..5c006454d 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -45,7 +45,10 @@ in the administration guide. Concept: Users and Projects --------------------------- -Projects +* access to images is limited by project +* access/secret are per user +* keypairs are per user +* quotas are per project Concept: Virtualization @@ -55,6 +58,7 @@ Concept: Virtualization * UML * XEN * HyperV +* qemu Concept: Volumes @@ -70,6 +74,13 @@ Concept: Quotas * Override for project +Concept: RBAC +------------- + +* Intersecting Roles +* cloudadmin vs. user admin flag + + Concept: API ------------ @@ -80,14 +91,30 @@ Concept: API Concept: Networking ------------------- -* Simple networking -* VLAN Networking +* VLAN + * Cloudpipe + * Certificates (See also: CA) +* Flat Networking +* Flat with DHCP +* How to generate addresses +* Floating Addresses Concept: Services ----------------- -Services +* nova-api +* nova-scheduler +* nova-compute +* nova-volume +* nova-network +* nova-instancemonitor + + +Concept: nova-manage +-------------------- + +nova manage Concept: Flags @@ -99,7 +126,13 @@ python-gflags Concept: Plugins ---------------- -LazyPluggable +* Managers/Drivers: utils.import_object from string flag +* virt/connections: conditional loading from string flag +* db: LazyPluggable via string flag +* auth_manager: utils.import_class based on string flag +* Volumes: moving to pluggable driver instead of manager +* Network: pluggable managers +* Compute: same driver used, but pluggable at connection Concept: IPC/RPC @@ -111,28 +144,33 @@ Rabbit! Concept: Fakes -------------- -Fakes +* auth +* ldap Concept: Scheduler ------------------ -Scheduler +* simple +* random -Concept: ORM ------------- +Concept: Security Groups +------------------------ -ORM +Security groups -Concept: Manager Pattern ------------------------- +Concept: Certificate Authority +------------------------------ -Managers +Per-project CA +* Images +* VPNs -Concept: Security Groups ------------------------- +Concept: Images +--------------- -Security groups +* launching +* bundling diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 18b08d6fe..2c8fa86f9 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -27,11 +27,20 @@ Contents .. toctree:: :maxdepth: 1 - architecture - network + database storage - auth compute + network + auth + + +Older Contents +-------------- + +.. toctree:: + :maxdepth: 1 + + architecture endpoint nova fakes diff --git a/doc/source/monitoring.rst b/doc/source/monitoring.rst new file mode 100644 index 000000000..e7766a6e7 --- /dev/null +++ b/doc/source/monitoring.rst @@ -0,0 +1,27 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Monitoring +========== + +* components +* throughput +* exceptions +* hardware + +* ganglia +* syslog diff --git a/doc/source/multi.node.install.rst b/doc/source/multi.node.install.rst index 443e2c4c3..dad9dcd01 100644 --- a/doc/source/multi.node.install.rst +++ b/doc/source/multi.node.install.rst @@ -19,3 +19,7 @@ Multi-Node Nova =============== http://etherpad.openstack.org/NovaMultinodeInstall + +* Puppet +* Chef +* PXE diff --git a/doc/source/network.rst b/doc/source/network.rst index 357a0517f..ef374a726 100644 --- a/doc/source/network.rst +++ b/doc/source/network.rst @@ -18,6 +18,25 @@ nova Networking ================ + +from etherpad todo +------------------ +:: + * Network Manager + * overview (use vish email) + * public methods + * responsibilities + * Vlan + DHCP implementation + * Flat + DHCP implementation + * Flat impleplementation + * Network Driver + * public methods + * hardware specific commands + + +legacy docs +----------- + The nova networking components manage private networks, public IP addressing, VPN connectivity, and firewall rules. Components diff --git a/doc/source/services.rst b/doc/source/services.rst new file mode 100644 index 000000000..7ff8fa6c3 --- /dev/null +++ b/doc/source/services.rst @@ -0,0 +1,31 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Services +======== + +Admin guide should go beyoned concepts & introduction by talking about how +each service interacts with each other, what requirements for deployment are, +monotoring, logging, etc + + +* nova-api +* nova-scheduler +* nova-compute +* nova-volume +* nova-network +* nova-instancemonitor diff --git a/doc/source/storage.rst b/doc/source/storage.rst index f77e5f0e5..72880a4a1 100644 --- a/doc/source/storage.rst +++ b/doc/source/storage.rst @@ -18,6 +18,22 @@ Storage in the Nova Cloud ========================= + +from etherpad todo +------------------- +:: + * Volume Manager + * public methods + * responsibilities + * Volume Driver + * public methods + * AoE implementation + * ISCSI implementation + + +legacy docs +----------- + There are three primary classes of storage in a nova cloud environment: * Ephemeral Storage (local disk within an instance) -- cgit From a1287cf4e15d469163ec6465ea5f6ce200c20543 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Wed, 27 Oct 2010 17:31:46 -0400 Subject: cleanup rrd doc generation. --- doc/source/auth.rst | 26 +++++++++++++++++++----- doc/source/compute.rst | 54 ++++++++++++++++++++++---------------------------- doc/source/conf.py | 2 +- 3 files changed, 46 insertions(+), 36 deletions(-) (limited to 'doc/source') diff --git a/doc/source/auth.rst b/doc/source/auth.rst index 2b8ad7924..d739e4b8e 100644 --- a/doc/source/auth.rst +++ b/doc/source/auth.rst @@ -195,18 +195,34 @@ The :mod:`signer` Module :undoc-members: :show-inheritance: -The :mod:`users` Module ------------------------ +The :mod:`manager` Module +------------------------- -.. automodule:: nova.auth.users +.. automodule:: nova.auth.manager :members: :undoc-members: :show-inheritance: -The :mod:`users_unittest` Module +The :mod:`ldapdriver` Module +---------------------------- + +.. automodule:: nova.auth.ldapdriver + :members: + :undoc-members: + :show-inheritance: + +The :mod:`dbdriver` Module +---------------------------- + +.. automodule:: nova.auth.dbdriver + :members: + :undoc-members: + :show-inheritance: + +The :mod:`auth_unittest` Module -------------------------------- -.. automodule:: nova.tests.users_unittest +.. automodule:: nova.tests.auth_unittest :members: :undoc-members: :show-inheritance: diff --git a/doc/source/compute.rst b/doc/source/compute.rst index 76957f316..93070fa88 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -15,80 +15,74 @@ License for the specific language governing permissions and limitations under the License. -Compute Documentation -===================== - from etherpad todo ------------------ -:: * FIX RST IMPORT ERRORS * Compute Manager * public methods * responsibilities * Compute Driver * public methods - * libvirt implementation - - kvm/qemu - - uml + * libvirt implementation (kvm/qemu vs uml) * xen implementation * hyperv implementation -legacy docs ------------ +Compute Documentation +===================== This page contains the Compute Package documentation. -The :mod:`disk` Module ----------------------- +The :mod:`manager` Module +------------------------- -.. automodule:: nova.compute.disk +.. automodule:: nova.compute.manager :members: :undoc-members: :show-inheritance: -The :mod:`exception` Module ---------------------------- +The :mod:`disk` Module +---------------------- -.. automodule:: nova.compute.exception +.. automodule:: nova.compute.disk :members: :undoc-members: :show-inheritance: -The :mod:`model` Module -------------------------- +The :mod:`instance_types` Module +-------------------------------- -.. automodule:: nova.compute.model +.. automodule:: nova.compute.instance_types :members: :undoc-members: :show-inheritance: - -The :mod:`network` Module -------------------------- -.. automodule:: nova.compute.network +The :mod:`power_state` Module +----------------------------- + +.. automodule:: nova.compute.power_state :members: :undoc-members: :show-inheritance: -The :mod:`node` Module ----------------------- +The :mod:`monitor` Module +------------------------- -.. automodule:: nova.compute.node +.. automodule:: nova.compute.monitor :members: :undoc-members: :show-inheritance: - + RELATED TESTS ---------------- +------------- -The :mod:`node_unittest` Module -------------------------------- +The :mod:`compute_unittest` Module +---------------------------------- -.. automodule:: nova.tests.node_unittest +.. automodule:: nova.tests.compute_unittest :members: :undoc-members: :show-inheritance: diff --git a/doc/source/conf.py b/doc/source/conf.py index 505771ff8..3cf79503a 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -16,7 +16,7 @@ import sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. -sys.path.append([os.path.abspath('../nova'), os.path.abspath('..'), os.path.abspath('../bin')]) +sys.path.insert(0, '../..') # -- General configuration ----------------------------------------------------- -- cgit From 0211ec7d61bea7e818418b6f17fdd45770ca9337 Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Wed, 27 Oct 2010 16:33:25 -0700 Subject: updates to auth, concepts, and network, fix of docstring --- doc/source/auth.rst | 46 ++++++++++++++++----------- doc/source/concepts.and.introduction.rst | 53 +++++++++++++++++++++++-------- doc/source/conf.py | 2 +- doc/source/images/cloudpipe.png | Bin 0 -> 89812 bytes doc/source/network.rst | 34 ++++++++++---------- 5 files changed, 85 insertions(+), 50 deletions(-) create mode 100644 doc/source/images/cloudpipe.png (limited to 'doc/source') diff --git a/doc/source/auth.rst b/doc/source/auth.rst index 2b8ad7924..0e3303949 100644 --- a/doc/source/auth.rst +++ b/doc/source/auth.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -15,9 +15,10 @@ License for the specific language governing permissions and limitations under the License. -Auth Documentation -================== +.. _auth: +Auth Documentation +================== from etherpad todo ------------------ @@ -39,7 +40,7 @@ Roles-Based Access Control of AWS-style APIs using SAML Assertions “Achieving FIPS 199 Moderate certification of a hybrid cloud environment using CloudAudit and declarative C.I.A. classifications” Introduction --------------- +------------ We will investigate one method for integrating an AWS-style API with US eAuthentication-compatible federated authentication systems, to achieve access controls and limits based on traditional operational roles. Additionally, we will look at how combining this approach, with an implementation of the CloudAudit APIs, will allow us to achieve a certification under FIPS 199 Moderate classification for a hybrid cloud environment. @@ -54,24 +55,26 @@ Typical implementations of US eAuth authentication systems are structured as fol [ SUN Identity Manager or other SAML Policy Controller ] --> maps URLs to groups… [ Apache Policy Agent in front of eAuth-secured Web Application ] - + In more ideal implementations, the remainder of the application-specific account information is stored either in extended schema on the LDAP server itself, via the use of a translucent LDAP proxy, or in an independent datastore keyed off of the UID provided via SAML assertion. -Basic AWS API call structure ----------------------------- +.. _auth_roles: + +Roles +----- -AWS API calls are traditionally secured via Access and Secret Keys, which are used to sign API calls, along with traditional timestamps to prevent replay attacks. The APIs can be logically grouped into sets that align with five typical roles: +AWS API calls are traditionally secured via Access and Secret Keys, which are used to sign API calls, along with traditional timestamps to prevent replay attacks. The APIs can be logically grouped into sets that align with five typical roles: * System User * System Administrator * Network Administrator * Project Manager * Cloud Administrator -* (IT-Sec?) +* IT-Security -There is an additional, conceptual end-user that may or may not have API access: +There is an additional, conceptual end-user that may or may not have API access: -* (EXTERNAL) End-user / Third-party User +* (EXTERNAL) End-user / Third-party User Basic operations are available to any System User: @@ -106,7 +109,7 @@ Cloud Administrator: Enhancements ------------ -* SAML Token passing +* SAML Token passing * REST interfaces * SOAP interfaces @@ -151,7 +154,7 @@ Dirty Cloud – Hybrid Data Centers * CloudAudit bridge interfaces * Anything in the ARP table -A hybrid cloud environment provides dedicated, potentially co-located physical hardware with a network interconnect to the project or users’ cloud virtual network. +A hybrid cloud environment provides dedicated, potentially co-located physical hardware with a network interconnect to the project or users’ cloud virtual network. This interconnect is typically a bridged VPN connection. Any machines that can be bridged into a hybrid environment in this fashion (at Layer 2) must implement a minimum version of the CloudAudit spec, such that they can be queried to provide a complete picture of the IT-sec runtime environment. @@ -167,7 +170,7 @@ The Details System limits ------------- -The following limits need to be defined and enforced: +The following limits need to be defined and enforced: * Total number of instances allowed (user / project) * Total number of instances, per instance type (user / project) @@ -195,18 +198,18 @@ The :mod:`signer` Module :undoc-members: :show-inheritance: -The :mod:`users` Module +The :mod:`auth` Module ----------------------- -.. automodule:: nova.auth.users +.. automodule:: nova.auth.manager :members: :undoc-members: :show-inheritance: -The :mod:`users_unittest` Module +The :mod:`auth_unittest` Module -------------------------------- -.. automodule:: nova.tests.users_unittest +.. automodule:: nova.tests.auth_unittest :members: :undoc-members: :show-inheritance: @@ -219,4 +222,11 @@ The :mod:`access_unittest` Module :undoc-members: :show-inheritance: +The :mod:`ec2` Module +------------------------ + +.. automodule:: nova.api.ec2 + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 5c006454d..54a9c7ccd 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -61,25 +61,33 @@ Concept: Virtualization * qemu +Concept: Instances +------------------ + +An 'instance' is a word for a virtual machine that runs inside the cloud. + Concept: Volumes ---------------- -Volumes +A 'volume' is a detachable block storage device. You can think of it as a usb hard drive. It can only be attached to one instance at a time, and it behaves Concept: Quotas --------------- -* Defaults -* Override for project +Nova supports per-project quotas. There are currently quotas for number of instances, total number of cores, number of volumes, total number of gigabytes, and number of floating ips. Concept: RBAC ------------- -* Intersecting Roles -* cloudadmin vs. user admin flag +Nova provides roles based access control (RBAC) for access to api commands. A user can have a number of different :ref:`roles `. Roles define which api_commands a user can perform. + +It is important to know that there are user-specific (sometimes called global) roles and project-specific roles. A user's actual permissions in a particular project are the INTERSECTION of his user-specific roles and is project-specific roles. +For example: A user can access api commands allowed to the netadmin role (like allocate_address) only if he has the user-specific netadmin role AND the project-specific netadmin role. + +More information about RBAC can be found in the :ref:`auth`. Concept: API ------------ @@ -91,14 +99,31 @@ Concept: API Concept: Networking ------------------- -* VLAN - * Cloudpipe - * Certificates (See also: CA) -* Flat Networking -* Flat with DHCP -* How to generate addresses -* Floating Addresses +Nova has a concept of Fixed Ips and Floating ips. Fixed ips are assigned to an instance on creation and stay the same until the instance is explicitly terminated. Floating ips are ip addresses that can be dynamically associated with an instance. This address can be disassociated and associated with another instance at any time. + +There are multiple strategies available for implementing fixed ips: + +Flat Mode +^^^^^^^^^ + +The simplest networking mode. Each instance receives a fixed ip from the pool. All instances are attached to the same bridge (br100) by default. The bridge must be configured manually. The networking configuration is injected into the instance before it is booted. Note that this currently only works on linux-style systems that keep networking configuration in /etc/network/interfaces. +Flat DHCP Mode +^^^^^^^^^^^^^^ + +This is similar to the flat mode, in that all instances are attached to the same bridge. In this mode nova does a bit more configuration, it will attempt to bridge into an ethernet device (eth0 by default). It will also run dnsmasq as a dhcpserver listening on this bridge. Instances receive their fixed ips by doing a dhcpdiscover. + +VLAN DHCP Mode +^^^^^^^^^^^^^^ + +This is the default networking mode and supports the most features. For multiple machine installation, it requires a switch that supports host-managed vlan tagging. In this mode, nova will create a vlan and bridge for each project. The project gets a range of private ips that are only accessible from inside the vlan. In order for a user to access the instances in their project, a special vpn instance (code name cloudpipe) needs to be created. Nova generates a certificate and key for the userto access the vpn and starts the vpn automatically. + +The following diagram illustrates how the communication that occurs between the vlan (the dashed box) and the public internet (represented by the two clouds) + +.. image:: /images/cloudpipe.png + :width: 100% + +.. Concept: Services ----------------- @@ -114,7 +139,7 @@ Concept: Services Concept: nova-manage -------------------- -nova manage +nova-manage is a command line utility for performing administrative tasks and checking on the health of the system. Concept: Flags diff --git a/doc/source/conf.py b/doc/source/conf.py index 505771ff8..34f049362 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -16,7 +16,7 @@ import sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. -sys.path.append([os.path.abspath('../nova'), os.path.abspath('..'), os.path.abspath('../bin')]) +sys.path.append([os.path.abspath('../../'), os.path.abspath('../../bin')]) # -- General configuration ----------------------------------------------------- diff --git a/doc/source/images/cloudpipe.png b/doc/source/images/cloudpipe.png new file mode 100644 index 000000000..ffdd181f2 Binary files /dev/null and b/doc/source/images/cloudpipe.png differ diff --git a/doc/source/network.rst b/doc/source/network.rst index ef374a726..b776ae40e 100644 --- a/doc/source/network.rst +++ b/doc/source/network.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -43,65 +43,65 @@ Components ---------- There are several key components: -* NetworkController (Manages address and vlan allocation) +* NetworkController (Manages address and vlan allocation) * RoutingNode (NATs public IPs to private IPs, and enforces firewall rules) * AddressingNode (runs DHCP services for private networks) * BridgingNode (a subclass of the basic nova ComputeNode) * TunnelingNode (provides VPN connectivity) - + Component Diagram ----------------- Overview:: - (PUBLIC INTERNET) + (PUBLIC INTERNET) | \ / \ / \ [RoutingNode] ... [RN] [TunnelingNode] ... [TN] | \ / | | | < AMQP > | | - [AddressingNode]-- (VLAN) ... | (VLAN)... (VLAN) --- [AddressingNode] + [AddressingNode]-- (VLAN) ... | (VLAN)... (VLAN) --- [AddressingNode] \ | \ / / \ / \ / \ / \ [BridgingNode] ... [BridgingNode] - - + + [NetworkController] ... [NetworkController] \ / < AMQP > | / \ - [CloudController]...[CloudController] + [CloudController]...[CloudController] -While this diagram may not make this entirely clear, nodes and controllers communicate exclusively across the message bus (AMQP, currently). +While this diagram may not make this entirely clear, nodes and controllers communicate exclusively across the message bus (AMQP, currently). State Model ----------- Network State consists of the following facts: * VLAN assignment (to a project) -* Private Subnet assignment (to a security group) in a VLAN +* Private Subnet assignment (to a security group) in a VLAN * Private IP assignments (to running instances) * Public IP allocations (to a project) * Public IP associations (to a private IP / running instance) -While copies of this state exist in many places (expressed in IPTables rule chains, DHCP hosts files, etc), the controllers rely only on the distributed "fact engine" for state, queried over RPC (currently AMQP). The NetworkController inserts most records into this datastore (allocating addresses, etc) - however, individual nodes update state e.g. when running instances crash. +While copies of this state exist in many places (expressed in IPTables rule chains, DHCP hosts files, etc), the controllers rely only on the distributed "fact engine" for state, queried over RPC (currently AMQP). The NetworkController inserts most records into this datastore (allocating addresses, etc) - however, individual nodes update state e.g. when running instances crash. The Public Traffic Path ----------------------- - + Public Traffic:: (PUBLIC INTERNET) | - <-- [RoutingNode] + <-- [RoutingNode] | [AddressingNode] --> | - ( VLAN ) + ( VLAN ) | <-- [BridgingNode] | - + -The RoutingNode is currently implemented using IPTables rules, which implement both NATing of public IP addresses, and the appropriate firewall chains. We are also looking at using Netomata / Clusto to manage NATting within a switch or router, and/or to manage firewall rules within a hardware firewall appliance. +The RoutingNode is currently implemented using IPTables rules, which implement both NATing of public IP addresses, and the appropriate firewall chains. We are also looking at using Netomata / Clusto to manage NATting within a switch or router, and/or to manage firewall rules within a hardware firewall appliance. -Similarly, the AddressingNode currently manages running DNSMasq instances for DHCP services. However, we could run an internal DHCP server (using Scapy ala Clusto), or even switch to static addressing by inserting the private address into the disk image the same way we insert the SSH keys. (See compute for more details). +Similarly, the AddressingNode currently manages running DNSMasq instances for DHCP services. However, we could run an internal DHCP server (using Scapy ala Clusto), or even switch to static addressing by inserting the private address into the disk image the same way we insert the SSH keys. (See compute for more details). -- cgit From 938caf0eb6620599a48696e9ca14cd4013e5cffb Mon Sep 17 00:00:00 2001 From: Devin Carlen Date: Wed, 27 Oct 2010 17:14:24 -0700 Subject: added nova-manage docs --- doc/source/concepts.and.introduction.rst | 113 ++++++++++++++++++++++++++++++- 1 file changed, 111 insertions(+), 2 deletions(-) (limited to 'doc/source') diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 54a9c7ccd..76db9e9f2 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -135,11 +135,120 @@ Concept: Services * nova-network * nova-instancemonitor - Concept: nova-manage -------------------- -nova-manage is a command line utility for performing administrative tasks and checking on the health of the system. +Introduction +++++++++++++ + +The nova-manage command is used to perform many essential functions for +administration and ongoing maintenance of nova, such as user creation, +vpn management, and much more. + +The standard pattern for executing a nova-manage command is: + +``nova-manage []`` + +For example, to obtain a list of all projects: + +``nova-manage project list`` + +User Maintenance +++++++++++++++++ + +* user admin: creates a new admin and prints exports + * arguments: name [access] [secret] +* user create: creates a new user and prints exports + * arguments: name [access] [secret] +* user delete: deletes an existing user + * arguments: name +* user exports: prints access and secrets for user in export format + * arguments: name +* user list: lists all users + * arguments: none +* user modify: update a users keys & admin flag + * arguments: accesskey secretkey admin + * leave any field blank to ignore it, admin should be 'T', 'F', or blank + +Project Maintenance ++++++++++++++++++++ + +* project add: Adds user to project + * arguments: project user +* project create: Creates a new project + * arguments: name project_manager [description] +* project delete: Deletes an existing project + * arguments: project_id +* project environment: Exports environment variables to an sourcable file + * arguments: project_id user_id [filename='novarc] +* project list: lists all projects + * arguments: none +* project quota: Set or display quotas for project + * arguments: project_id [key] [value] +* project remove: Removes user from project + * arguments: project user +* project scrub: Deletes data associated with project + * arguments: project +* project zipfile: Exports credentials for project to a zip file + * arguments: project_id user_id [filename='nova.zip] + +User Role Management +++++++++++++++++++++ + +* role add: adds role to user + * if project is specified, adds project specific role + * arguments: user, role [project] +* role has: checks to see if user has role + * if project is specified, returns True if user has + the global role and the project role + * arguments: user, role [project] +* role remove: removes role from user + * if project is specified, removes project specific role + * arguments: user, role [project] + + +Nova Shell +++++++++++ + +* shell bpython + * start a new bpython shell +* shell ipython + * start a new ipython shell +* shell python + * start a new python shell +* shell run + * ??? +* shell script: Runs the script from the specifed path with flags set properly. + * arguments: path + +VPN Management +++++++++++++++ + +* vpn list: Print a listing of the VPNs for all projects. + * arguments: none +* vpn run: Start the VPN for a given project. + * arguments: project +* vpn spawn: Run all VPNs. + * arguments: none + + +Floating IP Management +++++++++++++++++++++++ + +* floating create: Creates floating ips for host by range + * arguments: host ip_range +* floating delete: Deletes floating ips by range + * arguments: range +* floating list: Prints a listing of all floating ips + * arguments: none + +Network Management +++++++++++++++++++ + +* network create: Creates fixed ips for host by range + * arguments: [fixed_range=FLAG], [num_networks=FLAG], + [network_size=FLAG], [vlan_start=FLAG], + [vpn_start=FLAG] Concept: Flags -- cgit From 196ab5b15cb87ea5bfe70050bfa0f14c724e2676 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 10:39:51 -0400 Subject: Fix wiki link. --- doc/source/index.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/index.rst b/doc/source/index.rst index d4cdd1408..54cfea05a 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -30,7 +30,9 @@ Nova is written with the following design guidelines in mind: This documentation is generated by the Sphinx toolkit and lives in the source tree. Additional documentation on Nova and other components of OpenStack can -be found on the 'OpenStack wiki '. +be found on the `OpenStack wiki`_. + +.. _`OpenStack wiki`: http://wiki.openstack.org Contents -- cgit From e85ba051c27ab7d50914c7bf91db74d7cf7faa97 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 12:00:25 -0400 Subject: clean up the compute documentation a bit. --- doc/source/compute.rst | 78 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 67 insertions(+), 11 deletions(-) (limited to 'doc/source') diff --git a/doc/source/compute.rst b/doc/source/compute.rst index 93070fa88..737b2de98 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -16,9 +16,17 @@ under the License. +Compute +======= + +This page contains the Compute Package documentation. + + from etherpad todo ------------------ +:: + * FIX RST IMPORT ERRORS * Compute Manager * public methods @@ -30,57 +38,105 @@ from etherpad todo * hyperv implementation -Compute Documentation -===================== +Manager +------- -This page contains the Compute Package documentation. +Documentation for the compute manager and related files. For reading about +a specific virtualization backend, read Drivers_. The :mod:`manager` Module -------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: nova.compute.manager :members: :undoc-members: :show-inheritance: + The :mod:`disk` Module ----------------------- +~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: nova.compute.disk :members: :undoc-members: :show-inheritance: + The :mod:`instance_types` Module --------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: nova.compute.instance_types :members: :undoc-members: :show-inheritance: + The :mod:`power_state` Module ------------------------------ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: nova.compute.power_state :members: :undoc-members: :show-inheritance: + +Drivers +------- + + +Libvirt Implementations +~~~~~~~~~~~~~~~~~~~~~~~ + + +Libvirt: KVM +^^^^^^^^^^^^ + +KVM Driver + + +Libvirt: QEMU +^^^^^^^^^^^^^ + +QEMU Driver + + +Libvirt: UML +^^^^^^^^^^^^ + +User Mode Linux Driver + + +XEN +~~~ + +Xen Driver + + +Hyper-V +~~~~~~~ + +Hyper-V Driver + + +Monitoring +---------- + The :mod:`monitor` Module -------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: nova.compute.monitor :members: :undoc-members: :show-inheritance: -RELATED TESTS -------------- + +Tests +----- + The :mod:`compute_unittest` Module ----------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: nova.tests.compute_unittest :members: -- cgit From 44885b02a6c4db9ca2cbfc4067cd37fa3c504aee Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 12:09:59 -0400 Subject: Add missing rst file. --- doc/source/database.rst | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 doc/source/database.rst (limited to 'doc/source') diff --git a/doc/source/database.rst b/doc/source/database.rst new file mode 100644 index 000000000..657e4399f --- /dev/null +++ b/doc/source/database.rst @@ -0,0 +1,24 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Nova Database Programming Guide +=============================== + +:: + + * general interface + * sqlalchemy implementation -- cgit From 8fa3db896ca73bb8f72228afc387c9db500b9ae9 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 12:10:08 -0400 Subject: Clean up indentation error by preformatting. --- doc/source/concepts.and.introduction.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) (limited to 'doc/source') diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 5c006454d..ee03bbdd5 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -91,13 +91,15 @@ Concept: API Concept: Networking ------------------- -* VLAN - * Cloudpipe - * Certificates (See also: CA) -* Flat Networking -* Flat with DHCP -* How to generate addresses -* Floating Addresses +:: + + * VLAN + * Cloudpipe + * Certificates (See also: CA) + * Flat Networking + * Flat with DHCP + * How to generate addresses + * Floating Addresses Concept: Services -- cgit From 6cbc507cd821d91a9ce21e78b50748fa59ddf356 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 12:14:50 -0400 Subject: Remove outdated endpoint documentation. --- doc/source/development.guide.rst | 6 ++- doc/source/endpoint.rst | 91 ---------------------------------------- doc/source/modules.rst | 6 ++- 3 files changed, 10 insertions(+), 93 deletions(-) delete mode 100644 doc/source/endpoint.rst (limited to 'doc/source') diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 2c8fa86f9..1581b8baf 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -41,7 +41,6 @@ Older Contents :maxdepth: 1 architecture - endpoint nova fakes binaries @@ -49,6 +48,11 @@ Older Contents packages +Removed +------- + +* endpoint + Indices and tables ------------------ diff --git a/doc/source/endpoint.rst b/doc/source/endpoint.rst deleted file mode 100644 index 399df4161..000000000 --- a/doc/source/endpoint.rst +++ /dev/null @@ -1,91 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Endpoint Documentation -====================== - -This page contains the Endpoint Package documentation. - -The :mod:`admin` Module ------------------------ - -.. automodule:: nova.endpoint.admin - :members: - :undoc-members: - :show-inheritance: - -The :mod:`api` Module ---------------------- - -.. automodule:: nova.endpoint.api - :members: - :undoc-members: - :show-inheritance: - -The :mod:`cloud` Module ------------------------ - -.. automodule:: nova.endpoint.cloud - :members: - :undoc-members: - :show-inheritance: - -The :mod:`images` Module ------------------------- - -.. automodule:: nova.endpoint.images - :members: - :undoc-members: - :show-inheritance: - - -RELATED TESTS --------------- - -The :mod:`api_unittest` Module ------------------------------- - -.. automodule:: nova.tests.api_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`api_integration` Module ---------------------------------- - -.. automodule:: nova.tests.api_integration - :members: - :undoc-members: - :show-inheritance: - -The :mod:`cloud_unittest` Module --------------------------------- - -.. automodule:: nova.tests.cloud_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`network_unittest` Module ----------------------------------- - -.. automodule:: nova.tests.network_unittest - :members: - :undoc-members: - :show-inheritance: - - diff --git a/doc/source/modules.rst b/doc/source/modules.rst index 82c61f008..9b31664b6 100644 --- a/doc/source/modules.rst +++ b/doc/source/modules.rst @@ -28,7 +28,11 @@ Modules: auth compute - endpoint fakes nova volume + +Removed: +-------- + +* endpoint -- cgit From 89c881d651efcbba56406048112c3cf2cb676a8e Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 12:26:17 -0400 Subject: Fix underlining -> heading in rst file. --- doc/source/fakes.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/fakes.rst b/doc/source/fakes.rst index a993fb4c8..eccd4ab43 100644 --- a/doc/source/fakes.rst +++ b/doc/source/fakes.rst @@ -19,7 +19,7 @@ Nova Fakes ========== The :mod:`virt.fake` Module --------------------------- +--------------------------- .. automodule:: nova.virt.fake :members: -- cgit From 7d367e04c3e0dcdf7ce2121c464884c0825a552c Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 13:23:43 -0400 Subject: Clean up volumes / storage info. --- doc/source/concepts.and.introduction.rst | 6 ++-- doc/source/database.rst | 32 ++++++++++++++++++---- doc/source/development.guide.rst | 2 +- doc/source/storage.rst | 47 -------------------------------- doc/source/volume.rst | 17 ++++++++++-- 5 files changed, 46 insertions(+), 58 deletions(-) delete mode 100644 doc/source/storage.rst (limited to 'doc/source') diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index ee03bbdd5..c3057c958 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -61,10 +61,12 @@ Concept: Virtualization * qemu -Concept: Volumes +Concept: Storage ---------------- -Volumes +* Ephemeral +* Volumes +* Swift Concept: Quotas diff --git a/doc/source/database.rst b/doc/source/database.rst index 657e4399f..f0672f608 100644 --- a/doc/source/database.rst +++ b/doc/source/database.rst @@ -15,10 +15,32 @@ License for the specific language governing permissions and limitations under the License. -Nova Database Programming Guide -=============================== +Database Programming Guide +========================== -:: +The :mod:`api` Module +--------------------- + +.. automodule:: nova.db.api + :members: + :undoc-members: + :show-inheritance: + + +Drivers +------- + +The :mod:sqlalchemy Driver +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.db.sqlalchemy.api + :members: + :undoc-members: + :show-inheritance: + + +.. automodule:: nova.db.sqlalchemy.models + :members: + :undoc-members: + :show-inheritance: - * general interface - * sqlalchemy implementation diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 1581b8baf..d9a7c46f9 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -28,7 +28,7 @@ Contents :maxdepth: 1 database - storage + volume compute network auth diff --git a/doc/source/storage.rst b/doc/source/storage.rst deleted file mode 100644 index 72880a4a1..000000000 --- a/doc/source/storage.rst +++ /dev/null @@ -1,47 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Storage in the Nova Cloud -========================= - - -from etherpad todo -------------------- -:: - * Volume Manager - * public methods - * responsibilities - * Volume Driver - * public methods - * AoE implementation - * ISCSI implementation - - -legacy docs ------------ - -There are three primary classes of storage in a nova cloud environment: - -* Ephemeral Storage (local disk within an instance) -* Volume Storage (network-attached FS) -* Object Storage (redundant KVS with locality and MR) - -.. toctree:: - :maxdepth: 2 - - volume - objectstore diff --git a/doc/source/volume.rst b/doc/source/volume.rst index 619968458..c55961a46 100644 --- a/doc/source/volume.rst +++ b/doc/source/volume.rst @@ -15,9 +15,20 @@ License for the specific language governing permissions and limitations under the License. -Volume Documentation -==================== - +Volume Programming Guide +========================= + +The :mod:`nova.volume.manager` Module +------------------------------------- + +.. automodule:: nova.volume.manager + :members: + :undoc-members: + :show-inheritance: + +OLDHAT +------ + Nova uses ata-over-ethernet (AoE) to export storage volumes from multiple storage nodes. These AoE exports are attached (using libvirt) directly to running instances. Nova volumes are exported over the primary system VLAN (usually VLAN 1), and not over individual VLANs. -- cgit From b6539d86c7217290d46682e214a9e82cfc810447 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 13:58:20 -0400 Subject: Remove objectstore, not referenced anywhere. --- doc/source/objectstore.rst | 66 ---------------------------------------------- 1 file changed, 66 deletions(-) delete mode 100644 doc/source/objectstore.rst (limited to 'doc/source') diff --git a/doc/source/objectstore.rst b/doc/source/objectstore.rst deleted file mode 100644 index 6b8d293f4..000000000 --- a/doc/source/objectstore.rst +++ /dev/null @@ -1,66 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Objectstore Documentation -========================= - -This page contains the Objectstore Package documentation. - - -The :mod:`bucket` Module ------------------------- - -.. automodule:: nova.objectstore.bucket - :members: - :undoc-members: - :show-inheritance: - -The :mod:`handler` Module -------------------------- - -.. automodule:: nova.objectstore.handler - :members: - :undoc-members: - :show-inheritance: - -The :mod:`image` Module ------------------------ - -.. automodule:: nova.objectstore.image - :members: - :undoc-members: - :show-inheritance: - -The :mod:`stored` Module ------------------------- - -.. automodule:: nova.objectstore.stored - :members: - :undoc-members: - :show-inheritance: - -RELATED TESTS -------------- - -The :mod:`objectstore_unittest` Module --------------------------------------- - -.. automodule:: nova.tests.objectstore_unittest - :members: - :undoc-members: - :show-inheritance: - -- cgit From 9321ae23a1741267bc911e70b6755ac5c3251fbc Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 14:04:11 -0400 Subject: volume cleanups --- doc/source/volume.rst | 40 ++++++++++++++++++++++------------------ 1 file changed, 22 insertions(+), 18 deletions(-) (limited to 'doc/source') diff --git a/doc/source/volume.rst b/doc/source/volume.rst index c55961a46..f83556795 100644 --- a/doc/source/volume.rst +++ b/doc/source/volume.rst @@ -26,31 +26,35 @@ The :mod:`nova.volume.manager` Module :undoc-members: :show-inheritance: -OLDHAT ------- - -Nova uses ata-over-ethernet (AoE) to export storage volumes from multiple storage nodes. These AoE exports are attached (using libvirt) directly to running instances. - -Nova volumes are exported over the primary system VLAN (usually VLAN 1), and not over individual VLANs. - -AoE exports are numbered according to a "shelf and blade" syntax. In order to avoid collisions, we currently perform an AoE-discover of existing exports, and then grab the next unused number. (This obviously has race condition problems, and should be replaced by allocating a shelf-id to each storage node.) - -The underlying volumes are LVM logical volumes, created on demand within a single large volume group. - - -The :mod:`storage` Module -------------------------- +The :mod:`nova.volume.driver` Module +------------------------------------- -.. automodule:: nova.volume.storage +.. automodule:: nova.volume.driver :members: :undoc-members: :show-inheritance: -The :mod:`storage_unittest` Module ----------------------------------- +Tests +----- -.. automodule:: nova.tests.storage_unittest +.. automodule:: nova.tests.volume_unittest :members: :undoc-members: :show-inheritance: +Old Docs +-------- + +:: + + TODO(todd): change for iSCSI & AoE and move to top of document + +Nova uses ata-over-ethernet (AoE) to export storage volumes from multiple storage nodes. These AoE exports are attached (using libvirt) directly to running instances. + +Nova volumes are exported over the primary system VLAN (usually VLAN 1), and not over individual VLANs. + +AoE exports are numbered according to a "shelf and blade" syntax. In order to avoid collisions, we currently perform an AoE-discover of existing exports, and then grab the next unused number. (This obviously has race condition problems, and should be replaced by allocating a shelf-id to each storage node.) + +The underlying volumes are LVM logical volumes, created on demand within a single large volume group. + + -- cgit From 39fd6f43873e6d314504f53dc4cb1c8a3a3cfbc3 Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Thu, 28 Oct 2010 11:33:29 -0700 Subject: wip architecture, a few auth formatting fixes, binaries, and overview --- doc/source/architecture.rst | 20 ++++++------ doc/source/auth.rst | 45 ++++++++++++++------------- doc/source/binaries.rst | 53 ++++++++++++++++++++++++-------- doc/source/concepts.and.introduction.rst | 3 ++ doc/source/services.rst | 25 +++++++-------- 5 files changed, 91 insertions(+), 55 deletions(-) (limited to 'doc/source') diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index 11813d2c8..eba6cbfb8 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -30,14 +30,16 @@ Below you will find a helpful explanation. :: - [ User Manager ] ---- ( LDAP ) - | - | / [ Storage ] - ( ATAoE ) - [ API server ] -> [ Cloud ] < AMQP > - | \ [ Nodes ] - ( libvirt/kvm ) - < HTTP > - | - [ S3 ] + [ Auth Manager ] ---- ( LDAP ) + | + | + | + | / [ Storage ] - ( ATAoE/iSCSI ) + [ Web Dashboard ] -> [ nova-api ] < AMQP > - + | \ [ Nodes ] - ( libvirt/kvm ) + < HTTP > + | + [ nova-objectstore ] * API: receives http requests from boto, converts commands to/from API format, and sending requests to cloud controller diff --git a/doc/source/auth.rst b/doc/source/auth.rst index 4d3037253..adcb759a2 100644 --- a/doc/source/auth.rst +++ b/doc/source/auth.rst @@ -24,7 +24,7 @@ from etherpad todo ------------------ :: - * FIX RST IMPORT ERRORS + * Manager * Driver * ldap @@ -65,46 +65,47 @@ Roles AWS API calls are traditionally secured via Access and Secret Keys, which are used to sign API calls, along with traditional timestamps to prevent replay attacks. The APIs can be logically grouped into sets that align with five typical roles: -* System User -* System Administrator +* Base User +* System Administrator/Developer (currently have the same permissions) * Network Administrator * Project Manager -* Cloud Administrator -* IT-Security +* Cloud Administrator/IT-Security (currently have the same permissions) There is an additional, conceptual end-user that may or may not have API access: * (EXTERNAL) End-user / Third-party User -Basic operations are available to any System User: +Basic operations are available to any : -* Launch Instance -* Terminate Instance (their own) -* Create keypair -* Delete keypair -* Create, Upload, Delete: Buckets and Keys (Object Store) – their own -* Create, Attach, Delete Volume (Block Store) – their own +* Describe Instances +* Describe Images +* Describe Volumes +* Describe Keypairs +* Create Keypair +* Delete Keypair +* Create, Upload, Delete: Buckets and Keys (Object Store) -System Administrators: +System Administrators/Developers/Project Manager: +* Create, Attach, Delete Volume (Block Store) +* Launch, Reboot, Terminate Instance * Register/Unregister Machine Image (project-wide) -* Change Machine Image properties (public / private) * Request / Review CloudAudit Scans +Project Manager: + +* Add and remove other users (currently no api) +* Set roles (currently no api) + Network Administrator: +* Change Machine Image properties (public / private) * Change Firewall Rules, define Security Groups * Allocate, Associate, Deassociate Public IP addresses -Project Manager: - -* Launch and Terminate Instances (project-wide) -* CRUD of Object and Block store (project-wide) - -Cloud Administrator: +Cloud Administrator/IT-Security: -* Register / Unregister Kernel and Ramdisk Images -* Register / Unregister Machine Image (any) +* All permissions Enhancements ------------ diff --git a/doc/source/binaries.rst b/doc/source/binaries.rst index 90a9581f7..6771f345e 100644 --- a/doc/source/binaries.rst +++ b/doc/source/binaries.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -18,14 +18,43 @@ Nova Binaries =============== -* nova-api -* nova-compute -* nova-manage -* nova-objectstore -* nova-volume - -The configuration of these binaries relies on "flagfiles" using the google -gflags package. If present, the nova.conf file will be used as the flagfile -- otherwise, it must be specified on the command line:: - - $ python node_worker.py --flagfile flagfile +The configuration of these binaries relies on "flagfiles" using the google +gflags package:: + + $ nova-xxxxx --flagfile flagfile + +The binaries can all run on the same machine or be spread out amongst multiple boxes in a large deployment. + +nova-manage +----------- + +Nova manage is a command line utility to administer the system. It will autmatically try to load a flagfile from /etc/nova/nova-manage.conf to save you having to type it. Info on the commands can be found :ref:`here `. + +nova-api +-------- + +Nova api receives xml requests and sends them to the rest of the system. It is a wsgi app that routes and authenticate requests. It supports the ec2 and openstack apis. + +nova-objectstore +---------------- + +Nova objectstore is an ultra simple file-based storage system for images that replicates most of the S3 Api. It will soon be replaced with glance and a simple image manager. + +nova-compute +------------ + +Nova compute is responsible for managing virtual machines. It loads a Service object which exposes the public methods on ComputeManager via rpc. + +nova-volume +----------- + +Nova volume is responsible for managing attachable block storage devices. It loads a Service object which exposes the public methods on VolumeManager via rpc. + +nova-network +------------ + +Nova network is responsible for managing floating and fixed ips, dhcp, bridging and vlans. It loads a Service object which exposes the public methods on one of the subclasses of NetworkManager. Different networking strategies are as simple as changing the network_manager flag:: + + $ nova-network --network_manager=nova.network.manager.FlatManager + +IMPORTANT: Make sure that you also set the network_manager on nova-api and nova_compute, since make some calls to network manager in process instead of through rpc. More information on the interactions between services, managers, and drivers can be found :ref:`here ` diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 76db9e9f2..7f4aa153c 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -135,6 +135,9 @@ Concept: Services * nova-network * nova-instancemonitor + +.. _manage_usage: + Concept: nova-manage -------------------- diff --git a/doc/source/services.rst b/doc/source/services.rst index 7ff8fa6c3..607f5974b 100644 --- a/doc/source/services.rst +++ b/doc/source/services.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -15,17 +15,18 @@ License for the specific language governing permissions and limitations under the License. -Services -======== +.. _service_manager_driver: -Admin guide should go beyoned concepts & introduction by talking about how -each service interacts with each other, what requirements for deployment are, -monotoring, logging, etc +Services Managers and Drivers +============================= +Describe division of responsibilities for each component. -* nova-api -* nova-scheduler -* nova-compute -* nova-volume -* nova-network -* nova-instancemonitor +Service +------- + +Manager +------- + +Driver +------ -- cgit From 73773c2897894a4b7c40268f04fa100425411824 Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Thu, 28 Oct 2010 11:38:43 -0700 Subject: fix title levels --- doc/source/concepts.and.introduction.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) (limited to 'doc/source') diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 7f4aa153c..f36949835 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -142,7 +142,7 @@ Concept: nova-manage -------------------- Introduction -++++++++++++ +^^^^^^^^^^^^ The nova-manage command is used to perform many essential functions for administration and ongoing maintenance of nova, such as user creation, @@ -157,7 +157,7 @@ For example, to obtain a list of all projects: ``nova-manage project list`` User Maintenance -++++++++++++++++ +^^^^^^^^^^^^^^^^ * user admin: creates a new admin and prints exports * arguments: name [access] [secret] @@ -174,7 +174,7 @@ User Maintenance * leave any field blank to ignore it, admin should be 'T', 'F', or blank Project Maintenance -+++++++++++++++++++ +^^^^^^^^^^^^^^^^^^^ * project add: Adds user to project * arguments: project user @@ -196,7 +196,7 @@ Project Maintenance * arguments: project_id user_id [filename='nova.zip] User Role Management -++++++++++++++++++++ +^^^^^^^^^^^^^^^^^^^^ * role add: adds role to user * if project is specified, adds project specific role @@ -211,7 +211,7 @@ User Role Management Nova Shell -++++++++++ +^^^^^^^^^^ * shell bpython * start a new bpython shell @@ -225,7 +225,7 @@ Nova Shell * arguments: path VPN Management -++++++++++++++ +^^^^^^^^^^^^^^ * vpn list: Print a listing of the VPNs for all projects. * arguments: none @@ -236,7 +236,7 @@ VPN Management Floating IP Management -++++++++++++++++++++++ +^^^^^^^^^^^^^^^^^^^^^^ * floating create: Creates floating ips for host by range * arguments: host ip_range @@ -246,7 +246,7 @@ Floating IP Management * arguments: none Network Management -++++++++++++++++++ +^^^^^^^^^^^^^^^^^^ * network create: Creates fixed ips for host by range * arguments: [fixed_range=FLAG], [num_networks=FLAG], -- cgit From 828b299a5660537062b11e6e58cfdf4c840053e1 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 14:39:25 -0400 Subject: cleanup todos --- doc/source/compute.rst | 13 +------------ doc/source/database.rst | 10 ++++++++-- doc/source/volume.rst | 10 ++++++---- 3 files changed, 15 insertions(+), 18 deletions(-) (limited to 'doc/source') diff --git a/doc/source/compute.rst b/doc/source/compute.rst index 737b2de98..f3173122a 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -22,20 +22,9 @@ Compute This page contains the Compute Package documentation. -from etherpad todo ------------------- - :: - * FIX RST IMPORT ERRORS - * Compute Manager - * public methods - * responsibilities - * Compute Driver - * public methods - * libvirt implementation (kvm/qemu vs uml) - * xen implementation - * hyperv implementation + TODO(todd): Document drivers Manager diff --git a/doc/source/database.rst b/doc/source/database.rst index f0672f608..41b362d19 100644 --- a/doc/source/database.rst +++ b/doc/source/database.rst @@ -18,6 +18,12 @@ Database Programming Guide ========================== +:: + + TODO(todd): should sqlalchemy.api be here? + pep-256 on db/api.py and models.py (periods) + document register_models (where should it be called from?) + The :mod:`api` Module --------------------- @@ -30,8 +36,8 @@ The :mod:`api` Module Drivers ------- -The :mod:sqlalchemy Driver -~~~~~~~~~~~~~~~~~~~~~~~~~~ +The :mod:`sqlalchemy` Driver +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: nova.db.sqlalchemy.api :members: diff --git a/doc/source/volume.rst b/doc/source/volume.rst index f83556795..f4162560c 100644 --- a/doc/source/volume.rst +++ b/doc/source/volume.rst @@ -18,6 +18,12 @@ Volume Programming Guide ========================= +:: + + TODO(todd): document for iSCSI & AoE (see 'Old Docs') + pep-257 + + The :mod:`nova.volume.manager` Module ------------------------------------- @@ -45,10 +51,6 @@ Tests Old Docs -------- -:: - - TODO(todd): change for iSCSI & AoE and move to top of document - Nova uses ata-over-ethernet (AoE) to export storage volumes from multiple storage nodes. These AoE exports are attached (using libvirt) directly to running instances. Nova volumes are exported over the primary system VLAN (usually VLAN 1), and not over individual VLANs. -- cgit From 1915bb227d1eb847a681173772f4b9a769d64331 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 15:10:15 -0400 Subject: Todo cleanups in docs. --- doc/source/auth.rst | 24 +++++++++++++----------- doc/source/compute.rst | 4 ++-- doc/source/network.rst | 39 ++++++++++++++++++++++++--------------- 3 files changed, 39 insertions(+), 28 deletions(-) (limited to 'doc/source') diff --git a/doc/source/auth.rst b/doc/source/auth.rst index d739e4b8e..f4a7e0147 100644 --- a/doc/source/auth.rst +++ b/doc/source/auth.rst @@ -15,20 +15,22 @@ License for the specific language governing permissions and limitations under the License. -Auth Documentation -================== +Authentication and Authorization Development Guide +================================================== +:: -from etherpad todo ------------------- + TOOD(todd); + * Authentication + * Manager + * Driver + * ldap + * fakeldap + * redisectomy + * Authorizztion + * RBAC + * Quotas -:: - * FIX RST IMPORT ERRORS - * Manager - * Driver - * ldap - * fakeldap - * redisectomy legacy docs ----------- diff --git a/doc/source/compute.rst b/doc/source/compute.rst index f3173122a..9b4d10227 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -16,8 +16,8 @@ under the License. -Compute -======= +Virtualization Programming Guide +================================ This page contains the Compute Package documentation. diff --git a/doc/source/network.rst b/doc/source/network.rst index ef374a726..f9434b453 100644 --- a/doc/source/network.rst +++ b/doc/source/network.rst @@ -15,23 +15,32 @@ License for the specific language governing permissions and limitations under the License. -nova Networking -================ +Networking Programming Guide +============================ - -from etherpad todo ------------------- :: - * Network Manager - * overview (use vish email) - * public methods - * responsibilities - * Vlan + DHCP implementation - * Flat + DHCP implementation - * Flat impleplementation - * Network Driver - * public methods - * hardware specific commands + + TODO(todd): + * document how managers work + * document backend (linux-net) + * document hardware specific commands + * pep 257 + +The :mod:`Manager` module +------------------------- + +.. automodule:: nova.network.manager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`LinuxNet` driver +-------------------------- + +.. automodule:: nova.network.linux_net + :members: + :undoc-members: + :show-inheritance: legacy docs -- cgit From 7dedbca140c7e923c7b2fcaa8679a19249fb8350 Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Thu, 28 Oct 2010 14:27:24 -0700 Subject: link binaries section to concepts --- doc/source/binaries.rst | 2 ++ doc/source/concepts.and.introduction.rst | 10 ++-------- 2 files changed, 4 insertions(+), 8 deletions(-) (limited to 'doc/source') diff --git a/doc/source/binaries.rst b/doc/source/binaries.rst index 6771f345e..3ea07d9c2 100644 --- a/doc/source/binaries.rst +++ b/doc/source/binaries.rst @@ -15,6 +15,8 @@ License for the specific language governing permissions and limitations under the License. +.. _binaries: + Nova Binaries =============== diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index f36949835..7f59c0c92 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -125,16 +125,10 @@ The following diagram illustrates how the communication that occurs between the .. -Concept: Services +Concept: Binaries ----------------- -* nova-api -* nova-scheduler -* nova-compute -* nova-volume -* nova-network -* nova-instancemonitor - +Nova is implemented by a number of related binaries. These binaries can run on the same machine or many machines. A detailed description of each binary is given in the :ref:`binaries section ` of the developer guide. .. _manage_usage: -- cgit From 654a61c26d94a719e0ea665699b8075084ad79e3 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 18:59:25 -0400 Subject: Clean up todos and the like for docs. --- doc/source/auth.rst | 131 +++++++++++++++++++-------------------- doc/source/compute.rst | 5 +- doc/source/database.rst | 2 + doc/source/development.guide.rst | 9 ++- doc/source/network.rst | 1 + doc/source/volume.rst | 1 + 6 files changed, 81 insertions(+), 68 deletions(-) (limited to 'doc/source') diff --git a/doc/source/auth.rst b/doc/source/auth.rst index 0230ec5c4..203efe1a5 100644 --- a/doc/source/auth.rst +++ b/doc/source/auth.rst @@ -22,16 +22,71 @@ Authentication and Authorization Development Guide :: - TOOD(todd); - * Authentication - * Manager - * Driver - * ldap - * fakeldap - * redisectomy - * Authorizztion - * RBAC - * Quotas + TOOD(todd): * Authorizztion + * RBAC + * Quotas + + +Auth Manager +------------ + +The :mod:`manager` Module +~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.auth.manager + :members: + :undoc-members: + :show-inheritance: + +Drivers +------- + +The :mod:`ldapdriver` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.auth.ldapdriver + :members: + :undoc-members: + :show-inheritance: + +The :mod:`dbdriver` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.auth.dbdriver + :members: + :undoc-members: + :show-inheritance: + +API Integration +--------------- + +The :mod:`signer` Module +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.auth.signer + :members: + :undoc-members: + :show-inheritance: + + +Related Tests +------------- + +The :mod:`auth_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.auth_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`access_unittest` Module +--------------------------------- + +.. automodule:: nova.tests.access_unittest + :members: + :undoc-members: + :show-inheritance: legacy docs @@ -194,59 +249,3 @@ Further Challenges -The :mod:`signer` Module ------------------------- - -.. automodule:: nova.auth.signer - :members: - :undoc-members: - :show-inheritance: - -The :mod:`manager` Module -------------------------- - -.. automodule:: nova.auth.manager - :members: - :undoc-members: - :show-inheritance: - -The :mod:`ldapdriver` Module ----------------------------- - -.. automodule:: nova.auth.ldapdriver - :members: - :undoc-members: - :show-inheritance: - -The :mod:`dbdriver` Module ----------------------------- - -.. automodule:: nova.auth.dbdriver - :members: - :undoc-members: - :show-inheritance: - -The :mod:`auth_unittest` Module --------------------------------- - -.. automodule:: nova.tests.auth_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`access_unittest` Module ---------------------------------- - -.. automodule:: nova.tests.access_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`ec2` Module ---------------------- - -.. automodule:: nova.api.ec2 - :members: - :undoc-members: - :show-inheritance: - diff --git a/doc/source/compute.rst b/doc/source/compute.rst index 9b4d10227..d29b96781 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -24,7 +24,10 @@ This page contains the Compute Package documentation. :: - TODO(todd): Document drivers + TODO(todd): * Document drivers + * get docstrings to come in for ComputeManager + * fix formatting of ascii art in disk module + * document instance_types and power_states Manager diff --git a/doc/source/database.rst b/doc/source/database.rst index 41b362d19..bab6b472d 100644 --- a/doc/source/database.rst +++ b/doc/source/database.rst @@ -23,6 +23,8 @@ Database Programming Guide TODO(todd): should sqlalchemy.api be here? pep-256 on db/api.py and models.py (periods) document register_models (where should it be called from?) + document any relevant test cases + document flags The :mod:`api` Module --------------------- diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index d9a7c46f9..04fee329a 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -18,7 +18,14 @@ Nova Development Guide ====================== -Nova is written in python +Nova is written in python. + +:: + + TODO(todd): * API + * Exceptions + * Nova libraries (utils, etc) + * Building packages Contents diff --git a/doc/source/network.rst b/doc/source/network.rst index f53360bd7..b044c6413 100644 --- a/doc/source/network.rst +++ b/doc/source/network.rst @@ -25,6 +25,7 @@ Networking Programming Guide * document backend (linux-net) * document hardware specific commands * pep 257 + * document flags The :mod:`Manager` module ------------------------- diff --git a/doc/source/volume.rst b/doc/source/volume.rst index f4162560c..27977761a 100644 --- a/doc/source/volume.rst +++ b/doc/source/volume.rst @@ -22,6 +22,7 @@ Volume Programming Guide TODO(todd): document for iSCSI & AoE (see 'Old Docs') pep-257 + document flags The :mod:`nova.volume.manager` Module -- cgit From 05ada3f47a4250fb278ecc84c16f51922106b83d Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 28 Oct 2010 20:32:32 -0400 Subject: Finished TODO item --- doc/source/database.rst | 1 - 1 file changed, 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/database.rst b/doc/source/database.rst index bab6b472d..926f9be26 100644 --- a/doc/source/database.rst +++ b/doc/source/database.rst @@ -21,7 +21,6 @@ Database Programming Guide :: TODO(todd): should sqlalchemy.api be here? - pep-256 on db/api.py and models.py (periods) document register_models (where should it be called from?) document any relevant test cases document flags -- cgit From ee4ce8a55194be3fcb1e861e4206451cc7812d46 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 29 Oct 2010 11:23:49 -0400 Subject: Remove "nova Packages and Dependencies" --- doc/source/development.guide.rst | 6 ------ doc/source/packages.rst | 29 ----------------------------- 2 files changed, 35 deletions(-) delete mode 100644 doc/source/packages.rst (limited to 'doc/source') diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 04fee329a..61a664171 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -52,14 +52,8 @@ Older Contents fakes binaries modules - packages -Removed -------- - -* endpoint - Indices and tables ------------------ diff --git a/doc/source/packages.rst b/doc/source/packages.rst deleted file mode 100644 index 6029ad7d7..000000000 --- a/doc/source/packages.rst +++ /dev/null @@ -1,29 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -nova Packages & Dependencies -============================ - -Nova is being built on Ubuntu Lucid. - -The following packages are required: - - apt-get install python-ipy, python-libvirt, python-boto, python-pycurl, python-twisted, python-daemon, python-redis, python-carrot, python-lockfile - -In addition you need to install python: - - * python-gflags - http://code.google.com/p/python-gflags/ -- cgit From 133cd9973e17458bea3594490e70ccd3c524cf12 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 29 Oct 2010 11:58:57 -0400 Subject: Document Fakes --- doc/source/development.guide.rst | 1 + doc/source/fakes.rst | 34 ++++++++++++++++++++++++++++++++-- 2 files changed, 33 insertions(+), 2 deletions(-) (limited to 'doc/source') diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 61a664171..0d852a098 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -39,6 +39,7 @@ Contents compute network auth + fakes Older Contents diff --git a/doc/source/fakes.rst b/doc/source/fakes.rst index eccd4ab43..72e5c287b 100644 --- a/doc/source/fakes.rst +++ b/doc/source/fakes.rst @@ -15,8 +15,13 @@ License for the specific language governing permissions and limitations under the License. -Nova Fakes -========== +Fakes Programming Guide +======================= + +:: + + TODO(todd): * document general info about fakes + The :mod:`virt.fake` Module --------------------------- @@ -41,3 +46,28 @@ The :mod:`fakerabbit` Module :members: :undoc-members: :show-inheritance: + +The :class:`FakeAOEDriver` Class +-------------------------------- + +.. autoclass:: nova.volume.driver.FakeAOEDriver + :members: + :undoc-members: + :show-inheritance: + +The :class:`service_unittest.FakeManager` Class +----------------------------------------------- + +.. autoclass:: nova.tests.service_unittest.FakeManager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`api.openstack.fakes` Module +------------------------------------------------ + +.. automodule:: nova.tests.api.openstack.fakes + :members: + :undoc-members: + :show-inheritance: + -- cgit From 5feb2edcc322a8d44d3d698e2d3c27d81d16fe3f Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 29 Oct 2010 12:16:49 -0400 Subject: Fakes cleanup (stop duplicate autodoc of FakeAOEDriver). --- doc/source/volume.rst | 1 + 1 file changed, 1 insertion(+) (limited to 'doc/source') diff --git a/doc/source/volume.rst b/doc/source/volume.rst index 27977761a..91678d22a 100644 --- a/doc/source/volume.rst +++ b/doc/source/volume.rst @@ -40,6 +40,7 @@ The :mod:`nova.volume.driver` Module :members: :undoc-members: :show-inheritance: + :exclude-members: FakeAOEDriver Tests ----- -- cgit From 3ec095bed60490c844067c8d58ed43dbedee5f0a Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 29 Oct 2010 12:35:46 -0400 Subject: Update database page a bit. --- doc/source/database.rst | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) (limited to 'doc/source') diff --git a/doc/source/database.rst b/doc/source/database.rst index 926f9be26..67f940fe1 100644 --- a/doc/source/database.rst +++ b/doc/source/database.rst @@ -20,8 +20,7 @@ Database Programming Guide :: - TODO(todd): should sqlalchemy.api be here? - document register_models (where should it be called from?) + TODO(todd): document register_models (where should it be called from?) document any relevant test cases document flags @@ -37,17 +36,26 @@ The :mod:`api` Module Drivers ------- -The :mod:`sqlalchemy` Driver -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Driver: Sqlalchemy +~~~~~~~~~~~~~~~~~~ + +The :mod:`sqlalchemy.api` Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. automodule:: nova.db.sqlalchemy.api + +The :mod:`sqlalchemy.models` Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. automodule:: nova.db.sqlalchemy.models :members: :undoc-members: :show-inheritance: +The :mod:`sqlalchemy.session` Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. automodule:: nova.db.sqlalchemy.models +.. automodule:: nova.db.sqlalchemy.session :members: :undoc-members: :show-inheritance: - -- cgit From 489ddea1668c742f62acd6fd3e9af78f2f782912 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 29 Oct 2010 15:30:39 -0400 Subject: Update database docs. --- doc/source/database.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) (limited to 'doc/source') diff --git a/doc/source/database.rst b/doc/source/database.rst index 67f940fe1..b58ea147d 100644 --- a/doc/source/database.rst +++ b/doc/source/database.rst @@ -18,12 +18,6 @@ Database Programming Guide ========================== -:: - - TODO(todd): document register_models (where should it be called from?) - document any relevant test cases - document flags - The :mod:`api` Module --------------------- @@ -59,3 +53,9 @@ The :mod:`sqlalchemy.session` Module :members: :undoc-members: :show-inheritance: + +Tests +----- + +Tests are lacking for the db api layer and for the sqlalchemy driver. +Failures in the drivers would be dectected in other test cases, though. -- cgit From 5ffbfdd32bda3de071f994760ab9539bed40172a Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 29 Oct 2010 15:40:58 -0400 Subject: Remove fakes duplication. --- doc/source/development.guide.rst | 1 - 1 file changed, 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 0d852a098..df8eb341a 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -50,7 +50,6 @@ Older Contents architecture nova - fakes binaries modules -- cgit From 3bc84d66d35976794b559ad305dd10eec450216f Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 29 Oct 2010 16:19:57 -0400 Subject: Change volume TODO list. --- doc/source/volume.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) (limited to 'doc/source') diff --git a/doc/source/volume.rst b/doc/source/volume.rst index 91678d22a..39b33a500 100644 --- a/doc/source/volume.rst +++ b/doc/source/volume.rst @@ -20,9 +20,7 @@ Volume Programming Guide :: - TODO(todd): document for iSCSI & AoE (see 'Old Docs') - pep-257 - document flags + TODO(todd): rework after iSCSI merge (see 'Old Docs') The :mod:`nova.volume.manager` Module -- cgit From 878eb4d25075f8d78f24ad9f78eb5d43702192ca Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Mon, 1 Nov 2010 16:13:18 -0400 Subject: Virt documentation. --- doc/source/compute.rst | 53 +++++++++++++++++++++++++------------------------- 1 file changed, 27 insertions(+), 26 deletions(-) (limited to 'doc/source') diff --git a/doc/source/compute.rst b/doc/source/compute.rst index d29b96781..ba5f2917c 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -24,10 +24,7 @@ This page contains the Compute Package documentation. :: - TODO(todd): * Document drivers - * get docstrings to come in for ComputeManager - * fix formatting of ascii art in disk module - * document instance_types and power_states + TODO(todd): * document instance_types and power_states Manager @@ -45,6 +42,13 @@ The :mod:`manager` Module :undoc-members: :show-inheritance: +The :mod:`connection` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.virt.connection + :members: + :undoc-members: + :show-inheritance: The :mod:`disk` Module ~~~~~~~~~~~~~~~~~~~~~~ @@ -76,40 +80,37 @@ The :mod:`power_state` Module Drivers ------- - Libvirt Implementations ~~~~~~~~~~~~~~~~~~~~~~~ +The libvirt driver is capable of supporting KVM, QEMU, and UML. -Libvirt: KVM -^^^^^^^^^^^^ - -KVM Driver - - -Libvirt: QEMU -^^^^^^^^^^^^^ - -QEMU Driver - - -Libvirt: UML -^^^^^^^^^^^^ - -User Mode Linux Driver +The :mod:`libvirt_conn` Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. automodule:: nova.virt.libvirt_conn + :members: + :undoc-members: + :show-inheritance: XEN ~~~ -Xen Driver - +The :mod:`xenapi` Module +^^^^^^^^^^^^^^^^^^^^^^^^ -Hyper-V -~~~~~~~ +.. automodule:: nova.virt.xenapi + :members: + :undoc-members: + :show-inheritance: -Hyper-V Driver +FAKE +~~~~ +.. automodule:: nova.virt.fake + :members: + :undoc-members: + :show-inheritance: Monitoring ---------- -- cgit From 501850f3d470da646378c8e7de7657024411d2e0 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Mon, 1 Nov 2010 16:26:35 -0400 Subject: :noindex: on the fakes page for virt.fakes which is included in compute.rst --- doc/source/fakes.rst | 1 + 1 file changed, 1 insertion(+) (limited to 'doc/source') diff --git a/doc/source/fakes.rst b/doc/source/fakes.rst index 72e5c287b..5988e2ab8 100644 --- a/doc/source/fakes.rst +++ b/doc/source/fakes.rst @@ -30,6 +30,7 @@ The :mod:`virt.fake` Module :members: :undoc-members: :show-inheritance: + :noindex: The :mod:`fakeldap` Module -------------------------- -- cgit From c8e2341c98ffacfafffbadb7d204f10ff87cf89c Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Mon, 1 Nov 2010 20:33:03 -0400 Subject: API endpoint documentation. --- doc/source/api.rst | 112 +++++++++++++++++++++++++++++++++++++++ doc/source/development.guide.rst | 13 +---- doc/source/index.rst | 10 ++-- 3 files changed, 118 insertions(+), 17 deletions(-) create mode 100644 doc/source/api.rst (limited to 'doc/source') diff --git a/doc/source/api.rst b/doc/source/api.rst new file mode 100644 index 000000000..03ec6195d --- /dev/null +++ b/doc/source/api.rst @@ -0,0 +1,112 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +API Endpoints +============= + +:: + + TODO(todd): get actual docstrings from ec2/osapi_verions instead of @wsgify + +Nova has a system for managing multiple APIs on different subdomains. +Currently there is support for the OpenStack API, as well as the Amazon EC2 +API. + +Common Components +----------------- + +The :mod:`nova.api` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api + :members: + :undoc-members: + :show-inheritance: + +The :mod:`cloud` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.cloud + :members: + :undoc-members: + :show-inheritance: + +OpenStack API +------------- + +The :mod:`openstack` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack + :members: + :undoc-members: + :show-inheritance: + +The :mod:`auth` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.auth + :members: + :undoc-members: + :show-inheritance: + +The :mod:`backup_schedules` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.backup_schedules + :members: + :undoc-members: + :show-inheritance: + +The :mod:`faults` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.faults + :members: + :undoc-members: + :show-inheritance: + +The :mod:`flavors` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.flavors + :members: + :undoc-members: + :show-inheritance: + +The :mod:`images` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.images + :members: + :undoc-members: + :show-inheritance: + +The :mod:`ratelimiting` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.ratelimiting + :members: + :undoc-members: + :show-inheritance: + +The :mod:`servers` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.servers + :members: + :undoc-members: + :show-inheritance: + +The :mod:`sharedipgroups` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.sharedipgroups + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index df8eb341a..8addc7813 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -39,21 +39,10 @@ Contents compute network auth + api fakes -Older Contents --------------- - -.. toctree:: - :maxdepth: 1 - - architecture - nova - binaries - modules - - Indices and tables ------------------ diff --git a/doc/source/index.rst b/doc/source/index.rst index 54cfea05a..261919629 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -22,11 +22,11 @@ Nova is a cloud computing fabric controller, the main part of an IaaS system. Nova is written with the following design guidelines in mind: -* Component based architecture: Quickly add new behaviors -* Highly available: Scale to very serious workloads -* Fault-Tollerant: Isloated processes avoid cascading failures -* Recoverable: Failures should be easy to diagnose, debug, and rectify -* Open Standards: Be a reference implementation for a community-driven api +* **Component based architecture**: Quickly add new behaviors +* **Highly available**: Scale to very serious workloads +* **Fault-Tollerant**: Isloated processes avoid cascading failures +* **Recoverable**: Failures should be easy to diagnose, debug, and rectify +* **Open Standards**: Be a reference implementation for a community-driven api This documentation is generated by the Sphinx toolkit and lives in the source tree. Additional documentation on Nova and other components of OpenStack can -- cgit From ff588ab5baf400c243daeff82e7ca2fd27d87143 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Mon, 1 Nov 2010 21:13:51 -0400 Subject: Exceptions docs. --- doc/source/development.guide.rst | 4 ++-- doc/source/exceptions.rst | 27 +++++++++++++++++++++++++++ doc/source/nova.rst | 9 +-------- 3 files changed, 30 insertions(+), 10 deletions(-) create mode 100644 doc/source/exceptions.rst (limited to 'doc/source') diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 8addc7813..7322545de 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -22,8 +22,7 @@ Nova is written in python. :: - TODO(todd): * API - * Exceptions + TODO(todd): * Nova libraries (utils, etc) * Building packages @@ -40,6 +39,7 @@ Contents network auth api + exceptions fakes diff --git a/doc/source/exceptions.rst b/doc/source/exceptions.rst new file mode 100644 index 000000000..aaf5b2c1a --- /dev/null +++ b/doc/source/exceptions.rst @@ -0,0 +1,27 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Exceptions Programming Guide +============================ + +The :mod:`nova.excepton` Module +------------------------------- + +.. automodule:: nova.exception + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/nova.rst b/doc/source/nova.rst index 4b9c44a5f..a5573cb41 100644 --- a/doc/source/nova.rst +++ b/doc/source/nova.rst @@ -34,14 +34,6 @@ The :mod:`adminclient` Module :undoc-members: :show-inheritance: -The :mod:`datastore` Module ---------------------------- - -.. automodule:: nova.datastore - :members: - :undoc-members: - :show-inheritance: - The :mod:`exception` Module --------------------------- @@ -49,6 +41,7 @@ The :mod:`exception` Module :members: :undoc-members: :show-inheritance: + :noindex: The :mod:`flags` Module --------------------------- -- cgit From e0f889443f5c0732db28871f350c45e7c8e8d031 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Mon, 1 Nov 2010 21:47:16 -0400 Subject: Add ec2 api docs. --- doc/source/api.rst | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) (limited to 'doc/source') diff --git a/doc/source/api.rst b/doc/source/api.rst index 03ec6195d..75905d8b3 100644 --- a/doc/source/api.rst +++ b/doc/source/api.rst @@ -110,3 +110,54 @@ The :mod:`sharedipgroups` Module :members: :undoc-members: :show-inheritance: + +EC2 API +------- + +The :mod:`nova.api.ec2` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2 + :members: + :undoc-members: + :show-inheritance: + +The :mod:`admin` Module +~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.admin + :members: + :undoc-members: + :show-inheritance: + +The :mod:`apirequest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.apirequest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`cloud` Module +~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.cloud + :members: + :undoc-members: + :show-inheritance: + +The :mod:`images` Module +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.images + :members: + :undoc-members: + :show-inheritance: + +The :mod:`metadatarequesthandler` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.metadatarequesthandler + :members: + :undoc-members: + :show-inheritance: -- cgit From 500f101c64a8e5db91111a7afd7c95ac360b67fb Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Mon, 1 Nov 2010 22:32:41 -0400 Subject: Language change for conformity. --- doc/source/api.rst | 4 ++-- doc/source/auth.rst | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) (limited to 'doc/source') diff --git a/doc/source/api.rst b/doc/source/api.rst index 75905d8b3..3f1c69a15 100644 --- a/doc/source/api.rst +++ b/doc/source/api.rst @@ -15,8 +15,8 @@ License for the specific language governing permissions and limitations under the License. -API Endpoints -============= +API Endpoint Programming Guide +============================== :: diff --git a/doc/source/auth.rst b/doc/source/auth.rst index 203efe1a5..61db39ec3 100644 --- a/doc/source/auth.rst +++ b/doc/source/auth.rst @@ -17,7 +17,7 @@ .. _auth: -Authentication and Authorization Development Guide +Authentication and Authorization Programming Guide ================================================== :: -- cgit From 56c22eab57bc1096c0cd7e6756b42d163649fae1 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Mon, 1 Nov 2010 23:32:56 -0400 Subject: More docs. --- doc/source/compute.rst | 8 ++++ doc/source/development.guide.rst | 7 ++- doc/source/exceptions.rst | 27 ----------- doc/source/nova.rst | 101 +++++++++++++++++++++++++++++++-------- doc/source/scheduler.rst | 51 ++++++++++++++++++++ 5 files changed, 144 insertions(+), 50 deletions(-) delete mode 100644 doc/source/exceptions.rst create mode 100644 doc/source/scheduler.rst (limited to 'doc/source') diff --git a/doc/source/compute.rst b/doc/source/compute.rst index ba5f2917c..e9e37ebf8 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -58,6 +58,14 @@ The :mod:`disk` Module :undoc-members: :show-inheritance: +The :mod:`images` Module +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.virt.images + :members: + :undoc-members: + :show-inheritance: + The :mod:`instance_types` Module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 7322545de..261059d9b 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -22,9 +22,7 @@ Nova is written in python. :: - TODO(todd): - * Nova libraries (utils, etc) - * Building packages + TODO(todd): * Building packages Contents @@ -39,8 +37,9 @@ Contents network auth api - exceptions + scheduler fakes + nova Indices and tables diff --git a/doc/source/exceptions.rst b/doc/source/exceptions.rst deleted file mode 100644 index aaf5b2c1a..000000000 --- a/doc/source/exceptions.rst +++ /dev/null @@ -1,27 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Exceptions Programming Guide -============================ - -The :mod:`nova.excepton` Module -------------------------------- - -.. automodule:: nova.exception - :members: - :undoc-members: - :show-inheritance: diff --git a/doc/source/nova.rst b/doc/source/nova.rst index a5573cb41..59fd56c85 100644 --- a/doc/source/nova.rst +++ b/doc/source/nova.rst @@ -15,21 +15,29 @@ License for the specific language governing permissions and limitations under the License. -NOVA Libraries -=============== +Nova Libraries Programming Guide +================================ -The :mod:`crypto` Module ------------------------- +The :mod:`adminclient` Module +----------------------------- -.. automodule:: nova.crypto +.. automodule:: nova.adminclient :members: :undoc-members: :show-inheritance: - -The :mod:`adminclient` Module ------------------------------ -.. automodule:: nova.adminclient +The :mod:`context` Module +------------------------- + +.. automodule:: nova.context + :members: + :undoc-members: + :show-inheritance: + +The :mod:`crypto` Module +------------------------ + +.. automodule:: nova.crypto :members: :undoc-members: :show-inheritance: @@ -41,44 +49,99 @@ The :mod:`exception` Module :members: :undoc-members: :show-inheritance: - :noindex: - + The :mod:`flags` Module ---------------------------- +----------------------- .. automodule:: nova.flags :members: :undoc-members: :show-inheritance: +The :mod:`manager` Module +------------------------- + +.. automodule:: nova.manager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`process` Module +------------------------- + +.. automodule:: nova.process + :members: + :undoc-members: + :show-inheritance: + +The :mod:`quota` Module +----------------------- + +.. automodule:: nova.quota + :members: + :undoc-members: + :show-inheritance: + The :mod:`rpc` Module ---------------------------- +--------------------- .. automodule:: nova.rpc :members: :undoc-members: :show-inheritance: - + The :mod:`server` Module ---------------------------- +------------------------ .. automodule:: nova.server :members: :undoc-members: :show-inheritance: - + +The :mod:`service` Module +------------------------- + +.. automodule:: nova.service + :members: + :undoc-members: + :show-inheritance: + The :mod:`test` Module ---------------------------- +---------------------- .. automodule:: nova.test :members: :undoc-members: :show-inheritance: - + +The :mod:`twistd` Module +------------------------ + +.. automodule:: nova.twistd + :members: + :undoc-members: + :show-inheritance: + The :mod:`utils` Module ---------------------------- +----------------------- .. automodule:: nova.utils :members: :undoc-members: :show-inheritance: + +The :mod:`validate` Module +-------------------------- + +.. automodule:: nova.validate + :members: + :undoc-members: + :show-inheritance: + +The :mod:`wsgi` Module +---------------------- + +.. automodule:: nova.wsgi + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/scheduler.rst b/doc/source/scheduler.rst new file mode 100644 index 000000000..e809b0c54 --- /dev/null +++ b/doc/source/scheduler.rst @@ -0,0 +1,51 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Scheduler Programming Guide +=========================== + +The :mod:`manager` Module +------------------------- + +.. automodule:: nova.scheduler.manager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`driver` Module +------------------------ + +.. automodule:: nova.scheduler.driver + :members: + :undoc-members: + :show-inheritance: + +The :mod:`chance` Module +------------------------ + +.. automodule:: nova.scheduler.chance + :members: + :undoc-members: + :show-inheritance: + +The :mod:`simple` Module +------------------------ + +.. automodule:: nova.scheduler.simple + :members: + :undoc-members: + :show-inheritance: -- cgit From e50b8627ccac7f051d4ebea8879bd8d5c083837c Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Tue, 2 Nov 2010 13:57:50 -0700 Subject: cloudpipe docs --- doc/source/cloudpipe.rst | 77 ++++++++++++++++++++++++++++++++ doc/source/concepts.and.introduction.rst | 2 +- 2 files changed, 78 insertions(+), 1 deletion(-) create mode 100644 doc/source/cloudpipe.rst (limited to 'doc/source') diff --git a/doc/source/cloudpipe.rst b/doc/source/cloudpipe.rst new file mode 100644 index 000000000..952a75351 --- /dev/null +++ b/doc/source/cloudpipe.rst @@ -0,0 +1,77 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + +.. _cloudpipe: + +Cloudpipe +========= + +Cloudpipe is a method for connecting end users to their project insnances in vlan mode. + +Overview +-------- + +The support code for cloudpipe implements admin commands (via nova-manage) to automatically create a vm for a project that allows users to vpn into the private network of their project. Access to this vpn is provided through a public port on the network host for the project. This allows users to have free access to the virtual machines in their project without exposing those machines to the public internet. + +Cloudpipe Image +--------------- + +The cloudpipe image is basically just a linux instance with openvpn installed. It needs a simple script to grab user data from the metadata server, b64 decode it into a zip file, and run the autorun.sh script from inside the zip. The autorun script will configure and run openvpn to run using the data from nova. + +It is also useful to have a cron script that will periodically redownload the metadata and copy the new crl. This will keep revoked users from connecting and will disconnect any users that are connected with revoked certificates when their connection is renegotiated (every hour). + +Cloudpipe Launch +---------------- + +When you use nova-manage to launch a cloudpipe for a user, it goes through the following process: + +#. creates a keypair called -vpn and saves it in the keys directory +#. creates a security group -vpn and opens up 1194 and icmp +#. creates a cert and private key for the vpn instance and saves it in the CA/projects// directory +#. zips up the info and puts it b64 encoded as user data +#. launches an m1.tiny instance with the above settings using the flag-specified vpn image + +Vpn Access +---------- + +In vlan networking mode, the second ip in each private network is reserved for the cloudpipe instance. This gives a consistent ip to the instance so that nova-network can create forwarding rules for access from the outside world. The network for each project is given a specific high-numbered port on the public ip of the network host. This port is automatically forwarded to 1194 on the vpn instance. + +If specific high numbered ports do not work for your users, you can always allocate and associate a public ip to the instance, and then change the vpn_public_ip and vpn_public_port in the database. This will be turned into a nova-manage command or a flag soon. + + +Certificates and Revocation +--------------------------- + +If the use_project_ca flag is set (required to for cloudpipes to work securely), then each project has its own ca. This ca is used to sign the certificate for the vpn, and is also passed to the user for bundling images. When a certificate is revoked using nova-manage, a new Certificate Revocation List (crl) is generated. As long as cloudpipe has an updated crl, it will block revoked users from connecting to the vpn. + +The :mod:`cloudpipe` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.cloudpipe.pipelib + :members: + :undoc-members: + :show-inheritance: + +The :mod:`crypto` Module +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.crypto + :members: + :undoc-members: + :show-inheritance: + diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 29b8f064a..d2ab6f14b 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -129,7 +129,7 @@ This is similar to the flat mode, in that all instances are attached to the same VLAN DHCP Mode ~~~~~~~~~~~~~~ -This is the default networking mode and supports the most features. For multiple machine installation, it requires a switch that supports host-managed vlan tagging. In this mode, nova will create a vlan and bridge for each project. The project gets a range of private ips that are only accessible from inside the vlan. In order for a user to access the instances in their project, a special vpn instance (code name cloudpipe) needs to be created. Nova generates a certificate and key for the userto access the vpn and starts the vpn automatically. +This is the default networking mode and supports the most features. For multiple machine installation, it requires a switch that supports host-managed vlan tagging. In this mode, nova will create a vlan and bridge for each project. The project gets a range of private ips that are only accessible from inside the vlan. In order for a user to access the instances in their project, a special vpn instance (code named :ref:`cloudpipe `) needs to be created. Nova generates a certificate and key for the userto access the vpn and starts the vpn automatically. More information on cloudpipe can be found :ref:`here `. The following diagram illustrates how the communication that occurs between the vlan (the dashed box) and the public internet (represented by the two clouds) -- cgit From e493e324eb9a9fe31e72551b34bab768b507bc1d Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Tue, 2 Nov 2010 18:05:47 -0400 Subject: Document final undocumented python modules. --- doc/source/api.rst | 108 +++++++++++++++++++++++++++++++++++++++ doc/source/cloudpipe.rst | 27 ++++++++++ doc/source/compute.rst | 8 ++- doc/source/development.guide.rst | 3 ++ doc/source/glance.rst | 27 ++++++++++ doc/source/network.rst | 11 ++++ doc/source/nova.rst | 83 ++++++++++++++++++++++++++++++ doc/source/objectstore.rst | 62 ++++++++++++++++++++++ doc/source/scheduler.rst | 11 ++++ 9 files changed, 339 insertions(+), 1 deletion(-) create mode 100644 doc/source/cloudpipe.rst create mode 100644 doc/source/glance.rst create mode 100644 doc/source/objectstore.rst (limited to 'doc/source') diff --git a/doc/source/api.rst b/doc/source/api.rst index 3f1c69a15..0908d9e57 100644 --- a/doc/source/api.rst +++ b/doc/source/api.rst @@ -161,3 +161,111 @@ The :mod:`metadatarequesthandler` Module :members: :undoc-members: :show-inheritance: + +Tests +----- + +The :mod:`api_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`api_integration` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api_integration + :members: + :undoc-members: + :show-inheritance: + +The :mod:`cloud_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.cloud_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`api.fakes` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.fakes + :members: + :undoc-members: + :show-inheritance: + +The :mod:`api.test_wsgi` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.test_wsgi + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_api` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_api + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_auth` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_auth + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_faults` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_faults + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_flavors` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_flavors + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_images` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_images + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_ratelimiting` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_ratelimiting + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_servers` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_servers + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_sharedipgroups` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_sharedipgroups + :members: + :undoc-members: + :show-inheritance: + diff --git a/doc/source/cloudpipe.rst b/doc/source/cloudpipe.rst new file mode 100644 index 000000000..f2e7cb85a --- /dev/null +++ b/doc/source/cloudpipe.rst @@ -0,0 +1,27 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Cloudpipe Programmer Guide +========================== + +The :mod:`pipelib` Module +------------------------- + +.. automodule:: nova.cloudpipe.pipelib + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/compute.rst b/doc/source/compute.rst index e9e37ebf8..ff6aeab92 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -135,7 +135,6 @@ The :mod:`monitor` Module Tests ----- - The :mod:`compute_unittest` Module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -144,3 +143,10 @@ The :mod:`compute_unittest` Module :undoc-members: :show-inheritance: +The :mod:`virt_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.virt_unittest + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 261059d9b..306decccb 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -40,6 +40,9 @@ Contents scheduler fakes nova + cloudpipe + objectstore + glance Indices and tables diff --git a/doc/source/glance.rst b/doc/source/glance.rst new file mode 100644 index 000000000..3d0f4ebec --- /dev/null +++ b/doc/source/glance.rst @@ -0,0 +1,27 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Glance Integration Programming Guide +==================================== + +The :mod:`image.service` Module +------------------------------- + +.. automodule:: nova.image.service + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/network.rst b/doc/source/network.rst index b044c6413..cbff22d3f 100644 --- a/doc/source/network.rst +++ b/doc/source/network.rst @@ -43,6 +43,17 @@ The :mod:`LinuxNet` driver :undoc-members: :show-inheritance: +Tests +----- + +The :mod:`network_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.network_unittest + :members: + :undoc-members: + :show-inheritance: + legacy docs ----------- diff --git a/doc/source/nova.rst b/doc/source/nova.rst index 59fd56c85..17d32c010 100644 --- a/doc/source/nova.rst +++ b/doc/source/nova.rst @@ -145,3 +145,86 @@ The :mod:`wsgi` Module :members: :undoc-members: :show-inheritance: + +Tests +----- + +The :mod:`declare_flags` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.declare_flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`fake_flags` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.fake_flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`flags_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.flags_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`process_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.process_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`quota_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.quota_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`real_flags` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.real_flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`rpc_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.rpc_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`runtime_flags` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.runtime_flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`twistd_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.twistd_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`validator_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.validator_unittest + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/objectstore.rst b/doc/source/objectstore.rst new file mode 100644 index 000000000..4087b5dd0 --- /dev/null +++ b/doc/source/objectstore.rst @@ -0,0 +1,62 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Objectstore Programming Guide +============================= + +The :mod:`handler` Module +------------------------- + +.. automodule:: nova.objectstore.handler + :members: + :undoc-members: + :show-inheritance: + +The :mod:`bucket` Module +------------------------ + +.. automodule:: nova.objectstore.bucket + :members: + :undoc-members: + :show-inheritance: + +The :mod:`stored` Module +------------------------ + +.. automodule:: nova.objectstore.stored + :members: + :undoc-members: + :show-inheritance: + +The :mod:`image` Module +----------------------- + +.. automodule:: nova.objectstore.image + :members: + :undoc-members: + :show-inheritance: + +Tests +----- + +The :mod:`objectstore_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.objectstore_unittest + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/scheduler.rst b/doc/source/scheduler.rst index e809b0c54..df820c2f2 100644 --- a/doc/source/scheduler.rst +++ b/doc/source/scheduler.rst @@ -49,3 +49,14 @@ The :mod:`simple` Module :members: :undoc-members: :show-inheritance: + +Tests +----- + +The :mod:`scheduler_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.scheduler_unittest + :members: + :undoc-members: + :show-inheritance: -- cgit From a6f867eb72599bb0bb1fe43f99d8c4b07972e5dd Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Tue, 2 Nov 2010 15:16:41 -0700 Subject: Documentation on Services, Managers, and Drivers --- doc/source/services.rst | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/services.rst b/doc/source/services.rst index 607f5974b..517a51901 100644 --- a/doc/source/services.rst +++ b/doc/source/services.rst @@ -20,13 +20,53 @@ Services Managers and Drivers ============================= -Describe division of responsibilities for each component. +The responsibilities of Services, Managers, and Drivers, can be a bit confusing to people that are new to nova. This document attempts to outline the division of responsibilities to make understanding the system a little bit easier. + +Currently, Managers and Drivers are specified by flags and loaded using utils.load_object(). This method allows for them to be implemented as singletons, classes, modules or objects. As long as the path specified by the flag leads to an object (or a callable that returns an object) that responds to getattr, it should work as a manager or driver. Service ------- +A service is a very thin wrapper around a Manager object. It exposes the manager's public methods to other components of the system via rpc. It will report state periodically to the database and is responsible for initiating any periodic tasts that need to be executed on a given host. + +The :mod:`service` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.service + :members: + :undoc-members: + :show-inheritance: + Manager ------- +Managers are responsible for a certain aspect of the sytem. It is a logical grouping of code relating to a portion of the system. In general other components should be using the manager to make changes to the components that it is responsible for. + +For example, other components that need to deal with volumes in some way, should do so by calling methods on the VolumeManager instead of directly changing fields in the database. This allows us to keep all of the code relating to volumes in the same place. + +We have adopted a basic strategy of Smart managers and dumb data, which means rather than attaching methods to data objects, components should call manager methods that act on the data. + +Methods on managers that can be executed locally should be called directly. If a particular method must execute on a remote host, this should be done via rpc to the service that wraps the manager + +Managers should be responsible for most of the db access, and non-implementation specific data. Anything implementation specific that can't be generalized should be done by the Driver. + +In general, we prefer to have one manager with multiple drivers for different implementations, but sometimes it makes sense to have multiple managers. You can think of it this way: Abstract different overall strategies at the manager level(FlatNetwork vs VlanNetwork), and different implementations at the driver level(LinuxNetDriver vs CiscoNetDriver). + +Managers will often provide methods for initial setup of a host or periodic tasksto a wrapping service. + +The :mod:`manager` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.manager + :members: + :undoc-members: + :show-inheritance: + Driver ------ + +A manager will generally load a driver for some of its tasks. The driver is responsible for specific implementation details. Anything running shell commands on a host, or dealing with other non-python code should probably be happening in a driver. + +Drivers should minimize touching the database, although it is currently acceptable for implementation specific data. This may be reconsidered at some point. + +It usually makes sense to define an Abstract Base Class for the specific driver (i.e. VolumeDriver), to define the methods that a different driver would need to implement. -- cgit From 7898451cd91821e01451ff8f65de1169eeef8ae2 Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Tue, 2 Nov 2010 16:49:34 -0700 Subject: removed some old instructions and updated concepts --- doc/source/concepts.and.introduction.rst | 18 ++++--------- doc/source/getting.started.rst | 44 +++++--------------------------- 2 files changed, 11 insertions(+), 51 deletions(-) (limited to 'doc/source') diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index d2ab6f14b..d73447ba9 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -71,19 +71,13 @@ Concept: Storage Volumes ~~~~~~~ -A 'volume' is a detachable block storage device. You can think of it as a usb hard drive. It can only be attached to one instance at a time, and it behaves -Ephemeral -~~~~~~~~~ -:: - - TODO(vish): document +A 'volume' is a detachable block storage device. You can think of it as a usb hard drive. It can only be attached to one instance at a time, so it does not work like a SAN. If you wish to expose the same volume to multiple instances, you will have to use an NFS or SAMBA share from an existing instance. -Swift -~~~~~ -:: +Local Storage +~~~~~~~~~~~~~ - TODO(vish): document +Every instance larger than m1.tiny starts with some local storage (up to 160GB for m1.xlarge). This storage is currently the second partition on the root drive. Concept: Quotas --------------- @@ -308,9 +302,7 @@ Security groups Concept: Certificate Authority ------------------------------ -Per-project CA -* Images -* VPNs +Nova does a small amount of certificate management. These certificates are used for :ref:`project vpns ` and decrypting bundled images. Concept: Images diff --git a/doc/source/getting.started.rst b/doc/source/getting.started.rst index 2df4a45ea..70f26038f 100644 --- a/doc/source/getting.started.rst +++ b/doc/source/getting.started.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -18,7 +18,7 @@ Getting Started with Nova ========================= -This code base is continually changing so dependencies also change. +This code base is continually changing so dependencies also change. Dependencies ------------ @@ -30,7 +30,7 @@ Related servers we rely on Optional servers * OpenLDAP: By default, the auth server uses the RDBMS-backed datastore by setting FLAGS.auth_driver to 'nova.auth.dbdriver.DbDriver'. But OpenLDAP (or LDAP) could be configured. -* ReDIS: By default, this is not enabled as the auth driver. +* ReDIS: By default, this is not enabled as the auth driver. Python libraries we don't vendor @@ -62,49 +62,17 @@ Configuration These instructions are incomplete, but we are actively updating the `OpenStack wiki `_ with more configuration information. -On the cloud controller - -* Add yourself to the libvirtd group, log out, and log back in -* Fix hardcoded ec2 metadata/userdata uri ($IP is the IP of the cloud), and masqurade all traffic from launched instances - -:: - - iptables -t nat -A PREROUTING -s 0.0.0.0/0 -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination $IP:8773 - iptables --table nat --append POSTROUTING --out-interface $PUBLICIFACE -j MASQUERADE - - -* Configure NginX proxy (/etc/nginx/sites-enabled/default) - -:: - - server { - listen 3333 default; - server-name localhost; - client_max_body_size 10m; - - access_log /var/log/nginx/localhost.access.log; - - location ~ /_images/.+ { - root NOVA_PATH/images; - rewrite ^/_images/(.*)$ /$1 break; - } - - location / { - proxy_pass http://localhost:3334/; - } - } - On the volume node -* Create a filesystem (you can use an actual disk if you have one spare, default is /dev/sdb) +* Create a volume group (you can use an actual disk for the volume group as well) :: # This creates a 1GB file to create volumes out of dd if=/dev/zero of=MY_FILE_PATH bs=100M count=10 losetup --show -f MY_FILE_PATH - # replace loop0 below with whatever losetup returns - echo "--storage_dev=/dev/loop0" >> NOVA_PATH/bin/nova.conf + # replace /dev/loop0 below with whatever losetup returns + vgcreate nova-volumes /dev/loop0 Running --------- -- cgit From 583d1b1c4d039f1f9751c8a2cc0cf59bb77551e0 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Tue, 2 Nov 2010 20:31:17 -0400 Subject: Fixes after trunk merge. --- doc/source/nova.rst | 24 ------------------------ 1 file changed, 24 deletions(-) (limited to 'doc/source') diff --git a/doc/source/nova.rst b/doc/source/nova.rst index 17d32c010..3963992ae 100644 --- a/doc/source/nova.rst +++ b/doc/source/nova.rst @@ -34,14 +34,6 @@ The :mod:`context` Module :undoc-members: :show-inheritance: -The :mod:`crypto` Module ------------------------- - -.. automodule:: nova.crypto - :members: - :undoc-members: - :show-inheritance: - The :mod:`exception` Module --------------------------- @@ -58,14 +50,6 @@ The :mod:`flags` Module :undoc-members: :show-inheritance: -The :mod:`manager` Module -------------------------- - -.. automodule:: nova.manager - :members: - :undoc-members: - :show-inheritance: - The :mod:`process` Module ------------------------- @@ -98,14 +82,6 @@ The :mod:`server` Module :undoc-members: :show-inheritance: -The :mod:`service` Module -------------------------- - -.. automodule:: nova.service - :members: - :undoc-members: - :show-inheritance: - The :mod:`test` Module ---------------------- -- cgit From ea79ca7e8855cf130cadb63fc42780c90a51fb15 Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Tue, 2 Nov 2010 17:36:03 -0700 Subject: update of the architecture and fix some links --- doc/source/administration.guide.rst | 3 +-- doc/source/architecture.rst | 46 +++++++++++++++++++------------------ doc/source/development.guide.rst | 8 ++++--- doc/source/getting.started.rst | 38 ++++++++++++++++++++---------- 4 files changed, 56 insertions(+), 39 deletions(-) (limited to 'doc/source') diff --git a/doc/source/administration.guide.rst b/doc/source/administration.guide.rst index 789e97381..8c36d3d44 100644 --- a/doc/source/administration.guide.rst +++ b/doc/source/administration.guide.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -30,6 +30,5 @@ Contents quickstart getting.started multi.node.install - services flags monitoring diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index eba6cbfb8..1e23e1361 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -15,36 +15,38 @@ License for the specific language governing permissions and limitations under the License. -nova System Architecture +Nova System Architecture ======================== Nova is built on a shared-nothing, messaging-based architecture. All of the major nova components can be run on multiple servers. This means that most component to component communication must go via message queue. In order to avoid blocking each component while waiting for a response, we use deferred objects, with a callback that gets triggered when a response is received. -In order to achieve shared-nothing with multiple copies of the same component (especially when the component is an API server that needs to reply with state information in a timely fashion), we need to keep all of our system state in a distributed data system. Updates to system state are written into this system, using atomic transactions when necessary. Requests for state are read out of this system. In limited cases, these read calls are memoized within controllers for short periods of time. (Such a limited case would be, for instance, the current list of system users.) - +Nova recently moved to using a sql-based central database that is shared by all components in the system. The amount and depth of the data fits into a sql database quite well. For small deployments this seems like an optimal solution. For larger deployments, and especially if security is a concern, nova will be moving towards multiple data stores with some kind of aggregation system. Components ---------- -Below you will find a helpful explanation. +Below you will find a helpful explanation of the different components. :: - [ Auth Manager ] ---- ( LDAP ) - | - | - | - | / [ Storage ] - ( ATAoE/iSCSI ) - [ Web Dashboard ] -> [ nova-api ] < AMQP > - - | \ [ Nodes ] - ( libvirt/kvm ) - < HTTP > - | - [ nova-objectstore ] - - -* API: receives http requests from boto, converts commands to/from API format, and sending requests to cloud controller -* Cloud Controller: global state of system, talks to ldap, s3, and node/storage workers through a queue -* Nodes: worker that spawns instances -* S3: tornado based http/s3 server -* User Manager: create/manage users, which are stored in ldap -* Network Controller: allocate and deallocate IPs and VLANs + /- ( LDAP ) + [ Auth Manager ] --- + | \- ( DB ) + | + | [ scheduler ] - [ volume ] - ( ATAoE/iSCSI ) + | / + [ Web Dashboard ] -> [ api ] -- < AMQP > ------ [ network ] - ( Flat/Vlan ) + | \ + < HTTP > [ scheduler ] - [ compute ] - ( libvirt/xen ) + | | + [ objectstore ] < - retrieves images + +* DB: sql database for data storage. Used by all components (LINKS NOT SHOWN) +* Web Dashboard: potential external component that talks to the api +* api: component that receives http requests, converts commands and communicates with other components via the queue or http (in the case of objectstore) +* Auth Manager: component responsible for users/projects/and roles. Can backend to DB or LDAP. This is not a separate binary, but rather a python class that is used by most components in the system. +* objectstore: twisted http server that replicates s3 api and allows storage and retrieval of images +* scheduler: decides which host gets each vm and volume +* volume: manages dynamically attachable block devices. +* network: manages ip forwarding, bridges, and vlans +* compute: manages communication with hypervisor and virtual machines. diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index 261059d9b..45ebe2f10 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -1,6 +1,6 @@ .. Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. + Administrator of the National Aeronautics and Space Administration. All Rights Reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -31,11 +31,13 @@ Contents .. toctree:: :maxdepth: 1 + architecture + services database volume compute - network - auth + network + auth api scheduler fakes diff --git a/doc/source/getting.started.rst b/doc/source/getting.started.rst index 70f26038f..a0148e4b6 100644 --- a/doc/source/getting.started.rst +++ b/doc/source/getting.started.rst @@ -29,24 +29,37 @@ Related servers we rely on Optional servers -* OpenLDAP: By default, the auth server uses the RDBMS-backed datastore by setting FLAGS.auth_driver to 'nova.auth.dbdriver.DbDriver'. But OpenLDAP (or LDAP) could be configured. -* ReDIS: By default, this is not enabled as the auth driver. +* OpenLDAP: By default, the auth server uses the RDBMS-backed datastore by setting FLAGS.auth_driver to 'nova.auth.dbdriver.DbDriver'. But OpenLDAP (or LDAP) could be configured by specifying 'nova.auth.ldapdriver.LdapDriver'. There is a script in the sources(nova/auth/slap.sh) to install a very basic openldap server on ubuntu. +* ReDIS: There is a fake ldap driver that backends to redis. This was created for testing ldap implementation on systems that don't have an easy means to install ldap. -Python libraries we don't vendor +Python libraries that we use (from pip-requires): + +.. literalinclude:: ../../tools/pip-requires + +Other libraries: -* M2Crypto: python library interface for openssl -* curl * XenAPI: Needed only for Xen Cloud Platform or XenServer support. Available from http://wiki.xensource.com/xenwiki/XCP_SDK or http://community.citrix.com/cdn/xs/sdks. -Vendored python libaries (don't require any installation) +External unix tools that are required: + +* iptables +* ebtables +* gawk +* curl +* kvm +* libvirt +* dnsmasq +* vlan +* open-iscsi and iscsitarget (if you use iscsi volumes) +* aoetools and vblade-persist (if you use aoe-volumes) + +Nova uses cutting-edge versions of many packages. There are ubuntu packages in the nova-core ppa. You can use add this ppa to your sources list on an ubuntu machine with the following commands:: -* Twisted: just for the twisted.internet.defer package -* Tornado: scalable non blocking web server for api requests -* boto: python api for aws api -* IPy: library for managing ip addresses + sudo apt-get install -y python-software-properties + sudo add-apt-repository ppa:nova-core/ppa Recommended ------------------ +----------- * euca2ools: python implementation of aws ec2-tools and ami tools * build tornado to use C module for evented section @@ -55,7 +68,7 @@ Recommended Installation -------------- - Due to many changes it's best to rely on the `OpenStack wiki `_ for installation instructions. +Due to many changes it's best to rely on the `OpenStack wiki `_ for installation instructions. Configuration --------------- @@ -88,3 +101,4 @@ Launch nova components * nova-compute * nova-objectstore * nova-volume +* nova-scheduler -- cgit From 1119f7e0eeffc0bc2f918b0c5fb8eb87cd3e3784 Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Tue, 2 Nov 2010 17:37:11 -0700 Subject: more descriptive title for cloudpipe --- doc/source/cloudpipe.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/source') diff --git a/doc/source/cloudpipe.rst b/doc/source/cloudpipe.rst index 952a75351..d632a40e6 100644 --- a/doc/source/cloudpipe.rst +++ b/doc/source/cloudpipe.rst @@ -18,8 +18,8 @@ .. _cloudpipe: -Cloudpipe -========= +Cloudpipe -- Per Project Vpns +============================= Cloudpipe is a method for connecting end users to their project insnances in vlan mode. -- cgit From 2cbef8ffd80546f1dcd850322621b04395591d69 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Wed, 3 Nov 2010 14:30:13 -0400 Subject: Doc cleanups. --- doc/source/auth.rst | 21 ++++++++++++++------- doc/source/compute.rst | 6 ------ doc/source/development.guide.rst | 10 +++++++--- doc/source/network.rst | 8 ++------ doc/source/nova.rst | 16 ---------------- 5 files changed, 23 insertions(+), 38 deletions(-) (limited to 'doc/source') diff --git a/doc/source/auth.rst b/doc/source/auth.rst index 61db39ec3..77d97f68b 100644 --- a/doc/source/auth.rst +++ b/doc/source/auth.rst @@ -20,12 +20,13 @@ Authentication and Authorization Programming Guide ================================================== -:: - - TOOD(todd): * Authorizztion - * RBAC - * Quotas +The :mod:`quota` Module +----------------------- +.. automodule:: nova.quota + :members: + :undoc-members: + :show-inheritance: Auth Manager ------------ @@ -68,7 +69,6 @@ The :mod:`signer` Module :undoc-members: :show-inheritance: - Related Tests ------------- @@ -81,13 +81,20 @@ The :mod:`auth_unittest` Module :show-inheritance: The :mod:`access_unittest` Module ---------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: nova.tests.access_unittest :members: :undoc-members: :show-inheritance: +The :mod:`quota_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.quota_unittest + :members: + :undoc-members: + :show-inheritance: legacy docs ----------- diff --git a/doc/source/compute.rst b/doc/source/compute.rst index ff6aeab92..e4c6c6ae7 100644 --- a/doc/source/compute.rst +++ b/doc/source/compute.rst @@ -21,12 +21,6 @@ Virtualization Programming Guide This page contains the Compute Package documentation. - -:: - - TODO(todd): * document instance_types and power_states - - Manager ------- diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst index e4af2da56..704cbe399 100644 --- a/doc/source/development.guide.rst +++ b/doc/source/development.guide.rst @@ -20,9 +20,14 @@ Nova Development Guide Nova is written in python. -:: +Prerequisites +------------- +.. toctree:: + :maxdepth: 1 - TODO(todd): * Building packages + concepts.and.introduction + architecture + development.environment Contents @@ -31,7 +36,6 @@ Contents .. toctree:: :maxdepth: 1 - architecture services database volume diff --git a/doc/source/network.rst b/doc/source/network.rst index cbff22d3f..9426ece82 100644 --- a/doc/source/network.rst +++ b/doc/source/network.rst @@ -20,12 +20,8 @@ Networking Programming Guide :: - TODO(todd): - * document how managers work - * document backend (linux-net) - * document hardware specific commands - * pep 257 - * document flags + TODO(todd): * document hardware specific commands (maybe in admin guide?) + * document a map between flags and managers/backends The :mod:`Manager` module ------------------------- diff --git a/doc/source/nova.rst b/doc/source/nova.rst index 3963992ae..58125dc80 100644 --- a/doc/source/nova.rst +++ b/doc/source/nova.rst @@ -58,14 +58,6 @@ The :mod:`process` Module :undoc-members: :show-inheritance: -The :mod:`quota` Module ------------------------ - -.. automodule:: nova.quota - :members: - :undoc-members: - :show-inheritance: - The :mod:`rpc` Module --------------------- @@ -157,14 +149,6 @@ The :mod:`process_unittest` Module :undoc-members: :show-inheritance: -The :mod:`quota_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.quota_unittest - :members: - :undoc-members: - :show-inheritance: - The :mod:`real_flags` Module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -- cgit From 80a6dd72ef48ec4c7dc72b39073800bb7fe1e70a Mon Sep 17 00:00:00 2001 From: Vishvananda Ishaya Date: Wed, 3 Nov 2010 14:04:36 -0700 Subject: prettier theme --- doc/source/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/conf.py b/doc/source/conf.py index 27bc4ea6c..3f6ca2313 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -99,7 +99,7 @@ modindex_common_prefix = ['nova.'] # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. -html_theme = 'default' +html_theme = 'sphinxdoc' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the -- cgit From 6b8ee54df1a77c46b692cf43cc73009684cc9033 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 4 Nov 2010 00:02:56 -0400 Subject: have "contents" look the same as other headings. --- doc/source/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/index.rst b/doc/source/index.rst index 261919629..3c05d721d 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -36,7 +36,7 @@ be found on the `OpenStack wiki`_. Contents --------- +======== .. toctree:: :maxdepth: 1 -- cgit From 8ff07424548ad4d25b1653351d0bffaac7bc0642 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 4 Nov 2010 02:06:10 -0400 Subject: Cleanup nova-manage section. --- doc/source/concepts.and.introduction.rst | 117 +----------------- doc/source/nova.manage.rst | 200 +++++++++++++++++++++++++++++++ 2 files changed, 205 insertions(+), 112 deletions(-) create mode 100644 doc/source/nova.manage.rst (limited to 'doc/source') diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index d73447ba9..205529812 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -35,11 +35,9 @@ focues on describing how Nova's implementation of those concepts is achieved. This page outlines concepts that you will need to understand as a user or administrator of an OpenStack installation. Each section links to more more -detailed information in the `Administration Guide`_, but you'll probably want -to read this section straight-through before tackling the specifics presented -in the administration guide. - -.. _`Administration Guide`: administration.guide.html +detailed information in the :doc:`administration.guide`, +but you'll probably want to read this section straight-through before tackling +the specifics presented in the administration guide. Concept: Users and Projects @@ -142,117 +140,12 @@ Nova is implemented by a number of related binaries. These binaries can run on Concept: nova-manage -------------------- -Introduction -~~~~~~~~~~~~ - The nova-manage command is used to perform many essential functions for administration and ongoing maintenance of nova, such as user creation, vpn management, and much more. -The standard pattern for executing a nova-manage command is: - -``nova-manage []`` - -For example, to obtain a list of all projects: - -``nova-manage project list`` - -User Maintenance -~~~~~~~~~~~~~~~~ - -* user admin: creates a new admin and prints exports - * arguments: name [access] [secret] -* user create: creates a new user and prints exports - * arguments: name [access] [secret] -* user delete: deletes an existing user - * arguments: name -* user exports: prints access and secrets for user in export format - * arguments: name -* user list: lists all users - * arguments: none -* user modify: update a users keys & admin flag - * arguments: accesskey secretkey admin - * leave any field blank to ignore it, admin should be 'T', 'F', or blank - -Project Maintenance -~~~~~~~~~~~~~~~~~~~ - -* project add: Adds user to project - * arguments: project user -* project create: Creates a new project - * arguments: name project_manager [description] -* project delete: Deletes an existing project - * arguments: project_id -* project environment: Exports environment variables to an sourcable file - * arguments: project_id user_id [filename='novarc] -* project list: lists all projects - * arguments: none -* project quota: Set or display quotas for project - * arguments: project_id [key] [value] -* project remove: Removes user from project - * arguments: project user -* project scrub: Deletes data associated with project - * arguments: project -* project zipfile: Exports credentials for project to a zip file - * arguments: project_id user_id [filename='nova.zip] - -User Role Management -~~~~~~~~~~~~~~~~~~~~ - -* role add: adds role to user - * if project is specified, adds project specific role - * arguments: user, role [project] -* role has: checks to see if user has role - * if project is specified, returns True if user has - the global role and the project role - * arguments: user, role [project] -* role remove: removes role from user - * if project is specified, removes project specific role - * arguments: user, role [project] - - -Nova Shell -~~~~~~~~~~ - -* shell bpython - * start a new bpython shell -* shell ipython - * start a new ipython shell -* shell python - * start a new python shell -* shell run - * ??? -* shell script: Runs the script from the specifed path with flags set properly. - * arguments: path - -VPN Management -~~~~~~~~~~~~~~ - -* vpn list: Print a listing of the VPNs for all projects. - * arguments: none -* vpn run: Start the VPN for a given project. - * arguments: project -* vpn spawn: Run all VPNs. - * arguments: none - - -Floating IP Management -~~~~~~~~~~~~~~~~~~~~~~ - -* floating create: Creates floating ips for host by range - * arguments: host ip_range -* floating delete: Deletes floating ips by range - * arguments: range -* floating list: Prints a listing of all floating ips - * arguments: none - -Network Management -~~~~~~~~~~~~~~~~~~ - -* network create: Creates fixed ips for host by range - * arguments: [fixed_range=FLAG], [num_networks=FLAG], - [network_size=FLAG], [vlan_start=FLAG], - [vpn_start=FLAG] +See :doc:`nova.manage` in the Administration Guide for more +details. Concept: Flags diff --git a/doc/source/nova.manage.rst b/doc/source/nova.manage.rst new file mode 100644 index 000000000..eeb463f52 --- /dev/null +++ b/doc/source/nova.manage.rst @@ -0,0 +1,200 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + +nova-manage +=========== + +Introduction +~~~~~~~~~~~~ + +The nova-manage command is used to perform many essential functions for +administration and ongoing maintenance of nova, such as user creation, +vpn management, and much more. + +The standard pattern for executing a nova-manage command is: + +``nova-manage []`` + +For example, to obtain a list of all projects: + +``nova-manage project list`` + +You can run without arguments to see a list of available command categories: + +``nova-manage`` + +You can run with a category argument to see a list of all commands in that +category: + +``nova-manage user`` + +User Maintenance +~~~~~~~~~~~~~~~~ + +Users, including admins, are created through the ``user`` commands. + +* user admin: creates a new admin and prints exports + * arguments: name [access] [secret] +* user create: creates a new user and prints exports + * arguments: name [access] [secret] +* user delete: deletes an existing user + * arguments: name +* user exports: prints access and secrets for user in export format + * arguments: name +* user list: lists all users + * arguments: none +* user modify: update a users keys & admin flag + * arguments: accesskey secretkey admin + * leave any field blank to ignore it, admin should be 'T', 'F', or blank + +Project Maintenance +~~~~~~~~~~~~~~~~~~~ + +* project add: Adds user to project + * arguments: project user +* project create: Creates a new project + * arguments: name project_manager [description] +* project delete: Deletes an existing project + * arguments: project_id +* project environment: Exports environment variables to an sourcable file + * arguments: project_id user_id [filename='novarc] +* project list: lists all projects + * arguments: none +* project quota: Set or display quotas for project + * arguments: project_id [key] [value] +* project remove: Removes user from project + * arguments: project user +* project scrub: Deletes data associated with project + * arguments: project +* project zipfile: Exports credentials for project to a zip file + * arguments: project_id user_id [filename='nova.zip] + +User Role Management +~~~~~~~~~~~~~~~~~~~~ + +* role add: adds role to user + * if project is specified, adds project specific role + * arguments: user, role [project] +* role has: checks to see if user has role + * if project is specified, returns True if user has + the global role and the project role + * arguments: user, role [project] +* role remove: removes role from user + * if project is specified, removes project specific role + * arguments: user, role [project] + + +Nova Shell +~~~~~~~~~~ + +* shell bpython + * start a new bpython shell +* shell ipython + * start a new ipython shell +* shell python + * start a new python shell +* shell run + * ??? +* shell script: Runs the script from the specifed path with flags set properly. + * arguments: path + +VPN Management +~~~~~~~~~~~~~~ + +* vpn list: Print a listing of the VPNs for all projects. + * arguments: none +* vpn run: Start the VPN for a given project. + * arguments: project +* vpn spawn: Run all VPNs. + * arguments: none + + +Floating IP Management +~~~~~~~~~~~~~~~~~~~~~~ + +* floating create: Creates floating ips for host by range + * arguments: host ip_range +* floating delete: Deletes floating ips by range + * arguments: range +* floating list: Prints a listing of all floating ips + * arguments: none + +Network Management +~~~~~~~~~~~~~~~~~~ + +* network create: Creates fixed ips for host by range + * arguments: [fixed_range=FLAG], [num_networks=FLAG], + [network_size=FLAG], [vlan_start=FLAG], + [vpn_start=FLAG] + + +Concept: Flags +-------------- + +python-gflags + + +Concept: Plugins +---------------- + +* Managers/Drivers: utils.import_object from string flag +* virt/connections: conditional loading from string flag +* db: LazyPluggable via string flag +* auth_manager: utils.import_class based on string flag +* Volumes: moving to pluggable driver instead of manager +* Network: pluggable managers +* Compute: same driver used, but pluggable at connection + + +Concept: IPC/RPC +---------------- + +Rabbit! + + +Concept: Fakes +-------------- + +* auth +* ldap + + +Concept: Scheduler +------------------ + +* simple +* random + + +Concept: Security Groups +------------------------ + +Security groups + + +Concept: Certificate Authority +------------------------------ + +Nova does a small amount of certificate management. These certificates are used for :ref:`project vpns ` and decrypting bundled images. + + +Concept: Images +--------------- + +* launching +* bundling -- cgit From d14dafebf80cfc1776be5496b44a1970167114d8 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 4 Nov 2010 02:06:28 -0400 Subject: add missing file. --- doc/source/development.environment.rst | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 doc/source/development.environment.rst (limited to 'doc/source') diff --git a/doc/source/development.environment.rst b/doc/source/development.environment.rst new file mode 100644 index 000000000..ee19ba2bf --- /dev/null +++ b/doc/source/development.environment.rst @@ -0,0 +1,23 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Setting up a development environment +==================================== + +:: + + TODO(anthony) -- cgit From bbd11f550765a91af17cb200a2b857f263765918 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 4 Nov 2010 14:44:22 -0400 Subject: Nova quickstart: move vish's novascript into contrib, and convert reademe.md to a quickstart.rst --- doc/source/quickstart.rst | 79 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 77 insertions(+), 2 deletions(-) (limited to 'doc/source') diff --git a/doc/source/quickstart.rst b/doc/source/quickstart.rst index 7a41d4be6..91e3c9ccb 100644 --- a/doc/source/quickstart.rst +++ b/doc/source/quickstart.rst @@ -16,6 +16,81 @@ under the License. Nova Quickstart -========================= +=============== -http://github.com/vishvananda/novascript +The `contrib/novascript.sh` file in the source distribution is a script that +will quickly set up nova to run on a single machine. It is tested against +Ubuntu only, but other distributions are forthcoming. + +Usage +----- + +Unless you want to spend a lot of time fiddling with permissions and sudoers, +you should probably run nova as root. + +:: + + sudo -i + +If you are concerned about security, nova runs just fine inside a virtual +machine. + +Use the script to install and run the current trunk. You can also specify a +specific branch by putting `lp:~someone/nova/some-branch` after the branch +command + +:: + + ./nova.sh branch + ./nova.sh install + ./nova.sh run + +The run command will drop you into a screen session with all of the workers +running in different windows You can use eucatools to run commands against the +cloud. + +:: + + euca-add-keypair test > test.pem + euca-run-instances -k test -t m1.tiny ami-tiny + euca-describe-instances + +To see output from the various workers, switch screen windows + +:: + + " + +will give you a list of running windows. + +When the instance is running, you should be able to ssh to it. + +:: + + chmod 600 test.pem + ssh -i test.pem root@10.0.0.3 + +When you exit screen + +:: + + + +nova will terminate. It may take a while for nova to finish cleaning up. If +you exit the process before it is done because there were some problems in your +build, you may have to clean up the nova processes manually. If you had any +instances running, you can attempt to kill them through the api: + +:: + + ./nova.sh terminate + +Then you can destroy the screen: + +:: + + ./nova.sh clean + +If things get particularly messed up, you might need to do some more intense +cleanup. Be careful, the following command will manually destroy all runnning +virsh instances and attempt to delete all vlans and bridges. -- cgit From 3a8dadd4d31dd564a08a9e285cc6f3318dc243b3 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 4 Nov 2010 17:36:32 -0400 Subject: Getting Started Guide. --- doc/source/administration.guide.rst | 1 + doc/source/concepts.and.introduction.rst | 3 +- doc/source/getting.started.rst | 83 ++++++++++++++++++++++++++------ doc/source/index.rst | 4 +- doc/source/quickstart.rst | 8 +++ doc/source/reaching.out.rst | 28 +++++++++++ 6 files changed, 110 insertions(+), 17 deletions(-) create mode 100644 doc/source/reaching.out.rst (limited to 'doc/source') diff --git a/doc/source/administration.guide.rst b/doc/source/administration.guide.rst index 8c36d3d44..2d09bed71 100644 --- a/doc/source/administration.guide.rst +++ b/doc/source/administration.guide.rst @@ -30,5 +30,6 @@ Contents quickstart getting.started multi.node.install + nova.manage flags monitoring diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 205529812..5810280b4 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -144,8 +144,7 @@ The nova-manage command is used to perform many essential functions for administration and ongoing maintenance of nova, such as user creation, vpn management, and much more. -See :doc:`nova.manage` in the Administration Guide for more -details. +See doc:`nova.manage` in the Administration Guide for more details. Concept: Flags diff --git a/doc/source/getting.started.rst b/doc/source/getting.started.rst index a0148e4b6..6873c49cd 100644 --- a/doc/source/getting.started.rst +++ b/doc/source/getting.started.rst @@ -18,19 +18,31 @@ Getting Started with Nova ========================= -This code base is continually changing so dependencies also change. +This code base is continually changing, so dependencies also change. If you +encounter any problems, see the :doc:`reaching.out` page. +The `contrib/nova.sh` script should be kept up to date, and may be a good +resource to review when debugging. Dependencies ------------ Related servers we rely on -* RabbitMQ: messaging queue, used for all communication between components +* **RabbitMQ**: messaging queue, used for all communication between components Optional servers -* OpenLDAP: By default, the auth server uses the RDBMS-backed datastore by setting FLAGS.auth_driver to 'nova.auth.dbdriver.DbDriver'. But OpenLDAP (or LDAP) could be configured by specifying 'nova.auth.ldapdriver.LdapDriver'. There is a script in the sources(nova/auth/slap.sh) to install a very basic openldap server on ubuntu. -* ReDIS: There is a fake ldap driver that backends to redis. This was created for testing ldap implementation on systems that don't have an easy means to install ldap. +* **OpenLDAP**: By default, the auth server uses the RDBMS-backed datastore by + setting FLAGS.auth_driver to `nova.auth.dbdriver.DbDriver`. But OpenLDAP + (or LDAP) could be configured by specifying `nova.auth.ldapdriver.LdapDriver`. + There is a script in the sources (`nova/auth/slap.sh`) to install a very basic + openldap server on ubuntu. +* **ReDIS**: There is a fake ldap auth driver + `nova.auth.ldapdriver.FakeLdapDriver` that backends to redis. This was + created for testing ldap implementation on systems that don't have an easy + means to install ldap. +* **MySQL**: Either MySQL or another database supported by sqlalchemy needs to + be avilable. Currently, only sqlite3 an mysql have been tested. Python libraries that we use (from pip-requires): @@ -38,7 +50,9 @@ Python libraries that we use (from pip-requires): Other libraries: -* XenAPI: Needed only for Xen Cloud Platform or XenServer support. Available from http://wiki.xensource.com/xenwiki/XCP_SDK or http://community.citrix.com/cdn/xs/sdks. +* **XenAPI**: Needed only for Xen Cloud Platform or XenServer support. Available + from http://wiki.xensource.com/xenwiki/XCP_SDK or + http://community.citrix.com/cdn/xs/sdks. External unix tools that are required: @@ -53,7 +67,9 @@ External unix tools that are required: * open-iscsi and iscsitarget (if you use iscsi volumes) * aoetools and vblade-persist (if you use aoe-volumes) -Nova uses cutting-edge versions of many packages. There are ubuntu packages in the nova-core ppa. You can use add this ppa to your sources list on an ubuntu machine with the following commands:: +Nova uses cutting-edge versions of many packages. There are ubuntu packages in +the nova-core ppa. You can use add this ppa to your sources list on an ubuntu +machine with the following commands:: sudo apt-get install -y python-software-properties sudo add-apt-repository ppa:nova-core/ppa @@ -68,34 +84,73 @@ Recommended Installation -------------- -Due to many changes it's best to rely on the `OpenStack wiki `_ for installation instructions. +You can install from packages for your particular Linux distribution if they are +available. Otherwise you can install from source by checking out the source +files from the `Nova Source Code Repository `_ +and running:: + + python setup.py install Configuration --------------- -These instructions are incomplete, but we are actively updating the `OpenStack wiki `_ with more configuration information. - -On the volume node +Configuring the host system +~~~~~~~~~~~~~~~~~~~~~~~~~~~ -* Create a volume group (you can use an actual disk for the volume group as well) +As you read through the Administration Guide you will notice configuration hints +inline with documentation on the subsystem you are configuring. Presented in +this "Getting Started with Nova" document, we only provide what you need to +get started as quickly as possible. For a more detailed description of system +configuration, start reading through :doc:`multi.node.install`. -:: +* Create a volume group (you can use an actual disk for the volume group as + well):: # This creates a 1GB file to create volumes out of dd if=/dev/zero of=MY_FILE_PATH bs=100M count=10 losetup --show -f MY_FILE_PATH # replace /dev/loop0 below with whatever losetup returns + # nova-volumes is the default for the --volume_group flag vgcreate nova-volumes /dev/loop0 + +Configuring Nova +~~~~~~~~~~~~~~~~ + +Configuration of the entire system is performed through python-gflags. The +best way to track configuration is through the use of a flagfile. + +A flagfile is specified with the ``--flagfile=FILEPATH`` argument to the binary +when you launch it. Flagfiles for nova are typically stored in +``/etc/nova/nova.conf``, and flags specific to a certain program are stored in +``/etc/nova/nova-COMMAND.conf``. Each configuration file can include another +flagfile, so typically a file like ``nova-manage.conf`` would have as its first +line ``--flagfile=/etc/nova/nova.conf`` to load the common flags before +specifying overrides or additional options. + +A sample configuration to test the system follows:: + + --verbose + --nodaemon + --FAKE_subdomain=ec2 + --auth_driver=nova.auth.dbdriver.DbDriver + Running --------- -Launch servers +There are many parts to the nova system, each with a specific function. They +are built to be highly-available, so there are may configurations they can be +run in (ie: on many machines, many listeners per machine, etc). This part +of the guide only gets you started quickly, to learn about HA options, see +:doc:`multi.node.install`. + +Launch supporting services * rabbitmq * redis (optional) +* mysql (optional) -Launch nova components +Launch nova components, each should have ``--flagfile=/etc/nova/nova.conf`` * nova-api * nova-compute diff --git a/doc/source/index.rst b/doc/source/index.rst index 3c05d721d..9a62570ed 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -30,7 +30,8 @@ Nova is written with the following design guidelines in mind: This documentation is generated by the Sphinx toolkit and lives in the source tree. Additional documentation on Nova and other components of OpenStack can -be found on the `OpenStack wiki`_. +be found on the `OpenStack wiki`_. Also see the :doc:`reaching.out` page for +other ways to interact witht the community. .. _`OpenStack wiki`: http://wiki.openstack.org @@ -44,6 +45,7 @@ Contents concepts.and.introduction administration.guide development.guide + reaching.out Indices and tables diff --git a/doc/source/quickstart.rst b/doc/source/quickstart.rst index 91e3c9ccb..511ed2483 100644 --- a/doc/source/quickstart.rst +++ b/doc/source/quickstart.rst @@ -18,6 +18,14 @@ Nova Quickstart =============== +:: + + TODO(todd): * Document the assumptions about pluggable interfaces + (sqlite3 instead of mysql, etc) + * Document env vars that can change things + (USE_MYSQL, HOST_IP) + + The `contrib/novascript.sh` file in the source distribution is a script that will quickly set up nova to run on a single machine. It is tested against Ubuntu only, but other distributions are forthcoming. diff --git a/doc/source/reaching.out.rst b/doc/source/reaching.out.rst new file mode 100644 index 000000000..0d8f675e0 --- /dev/null +++ b/doc/source/reaching.out.rst @@ -0,0 +1,28 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Reaching Out +============ + +The OpenStack community is a very friendly place. Feel free to ask questions. +This document points you to some of the places you may want to communicate +with people at. + +:: + + TODO(todd): * Write This + * Don't end a sentence with a preposition -- cgit From 74762113966fb873816afa7bc7c0f2e2e9eb2ec5 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 4 Nov 2010 17:49:13 -0400 Subject: document purpose of documentation. --- doc/source/getting.started.rst | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'doc/source') diff --git a/doc/source/getting.started.rst b/doc/source/getting.started.rst index 6873c49cd..79d2cf204 100644 --- a/doc/source/getting.started.rst +++ b/doc/source/getting.started.rst @@ -23,6 +23,14 @@ encounter any problems, see the :doc:`reaching.out` page. The `contrib/nova.sh` script should be kept up to date, and may be a good resource to review when debugging. +The purpose of this document is to get a system installed that you can use to +test your setup assumptions. Working from this base installtion you can +tweak configurations and work with different flags to monitor interaction with +your hardware, network, and other factors that will allow you to determine +suitability for your deployment. After following this setup method, you should +be able to experiment with different managers, drivers, and flags to get the +best performance. + Dependencies ------------ -- cgit From 0de105e28ece8e742943abd3e6332d6e5694642c Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 4 Nov 2010 18:17:02 -0400 Subject: start adding info to multi-node admin guide. --- doc/source/multi.node.install.rst | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) (limited to 'doc/source') diff --git a/doc/source/multi.node.install.rst b/doc/source/multi.node.install.rst index dad9dcd01..1d484e779 100644 --- a/doc/source/multi.node.install.rst +++ b/doc/source/multi.node.install.rst @@ -18,8 +18,40 @@ Multi-Node Nova =============== -http://etherpad.openstack.org/NovaMultinodeInstall +When you move beyond evaluating the technology and into building an actual +production environemnt, you will need to know how to configure your datacenter +and how to deploy components across your clusters. This guide should help you +through that process. + +Bare-metal Provisioning +----------------------- + +To install the base operating system you can use PXE booting. + +Deployment Technologies +----------------------- + +Once you have machines with a base operating system installation, you can deploy +code and configuration with your favorite tools: * Puppet * Chef -* PXE + +Types of Hosts +-------------- + +A single machine in your cluster can act as one or more of the following types +of host: + +Nova Services + +* Network +* Compute +* Volume +* API +* Objectstore + +Other supporting services + +* Datastore +* Message Queue -- cgit From 4ea4eac3f1a33c1f618c82c5c2312b4626aa7244 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Thu, 4 Nov 2010 19:00:46 -0400 Subject: Change directory structure for great justice! --- doc/source/adminguide/binaries.rst | 62 ++++++ doc/source/adminguide/flags.rst | 23 ++ doc/source/adminguide/getting.started.rst | 167 +++++++++++++++ doc/source/adminguide/index.rst | 36 ++++ doc/source/adminguide/monitoring.rst | 27 +++ doc/source/adminguide/multi.node.install.rst | 57 +++++ doc/source/adminguide/nova.manage.rst | 200 +++++++++++++++++ doc/source/adminguide/quickstart.rst | 104 +++++++++ doc/source/administration.guide.rst | 35 --- doc/source/api.rst | 271 ------------------------ doc/source/architecture.rst | 52 ----- doc/source/auth.rst | 258 ---------------------- doc/source/binaries.rst | 62 ------ doc/source/cloudpipe.rst | 77 ------- doc/source/compute.rst | 146 ------------- doc/source/concepts.and.introduction.rst | 2 +- doc/source/database.rst | 61 ------ doc/source/development.environment.rst | 23 -- doc/source/development.guide.rst | 60 ------ doc/source/devguide/api.rst | 271 ++++++++++++++++++++++++ doc/source/devguide/architecture.rst | 52 +++++ doc/source/devguide/auth.rst | 258 ++++++++++++++++++++++ doc/source/devguide/cloudpipe.rst | 77 +++++++ doc/source/devguide/compute.rst | 146 +++++++++++++ doc/source/devguide/database.rst | 61 ++++++ doc/source/devguide/development.environment.rst | 23 ++ doc/source/devguide/fakes.rst | 74 +++++++ doc/source/devguide/glance.rst | 27 +++ doc/source/devguide/index.rst | 60 ++++++ doc/source/devguide/network.rst | 124 +++++++++++ doc/source/devguide/nova.rst | 190 +++++++++++++++++ doc/source/devguide/objectstore.rst | 62 ++++++ doc/source/devguide/scheduler.rst | 62 ++++++ doc/source/devguide/services.rst | 72 +++++++ doc/source/devguide/volume.rst | 62 ++++++ doc/source/fakes.rst | 74 ------- doc/source/flags.rst | 23 -- doc/source/getting.started.rst | 167 --------------- doc/source/glance.rst | 27 --- doc/source/index.rst | 4 +- doc/source/modules.rst | 38 ---- doc/source/monitoring.rst | 27 --- doc/source/multi.node.install.rst | 57 ----- doc/source/network.rst | 124 ----------- doc/source/nova.manage.rst | 200 ----------------- doc/source/nova.rst | 190 ----------------- doc/source/objectstore.rst | 62 ------ doc/source/quickstart.rst | 104 --------- doc/source/scheduler.rst | 62 ------ doc/source/services.rst | 72 ------- doc/source/volume.rst | 62 ------ 51 files changed, 2300 insertions(+), 2337 deletions(-) create mode 100644 doc/source/adminguide/binaries.rst create mode 100644 doc/source/adminguide/flags.rst create mode 100644 doc/source/adminguide/getting.started.rst create mode 100644 doc/source/adminguide/index.rst create mode 100644 doc/source/adminguide/monitoring.rst create mode 100644 doc/source/adminguide/multi.node.install.rst create mode 100644 doc/source/adminguide/nova.manage.rst create mode 100644 doc/source/adminguide/quickstart.rst delete mode 100644 doc/source/administration.guide.rst delete mode 100644 doc/source/api.rst delete mode 100644 doc/source/architecture.rst delete mode 100644 doc/source/auth.rst delete mode 100644 doc/source/binaries.rst delete mode 100644 doc/source/cloudpipe.rst delete mode 100644 doc/source/compute.rst delete mode 100644 doc/source/database.rst delete mode 100644 doc/source/development.environment.rst delete mode 100644 doc/source/development.guide.rst create mode 100644 doc/source/devguide/api.rst create mode 100644 doc/source/devguide/architecture.rst create mode 100644 doc/source/devguide/auth.rst create mode 100644 doc/source/devguide/cloudpipe.rst create mode 100644 doc/source/devguide/compute.rst create mode 100644 doc/source/devguide/database.rst create mode 100644 doc/source/devguide/development.environment.rst create mode 100644 doc/source/devguide/fakes.rst create mode 100644 doc/source/devguide/glance.rst create mode 100644 doc/source/devguide/index.rst create mode 100644 doc/source/devguide/network.rst create mode 100644 doc/source/devguide/nova.rst create mode 100644 doc/source/devguide/objectstore.rst create mode 100644 doc/source/devguide/scheduler.rst create mode 100644 doc/source/devguide/services.rst create mode 100644 doc/source/devguide/volume.rst delete mode 100644 doc/source/fakes.rst delete mode 100644 doc/source/flags.rst delete mode 100644 doc/source/getting.started.rst delete mode 100644 doc/source/glance.rst delete mode 100644 doc/source/modules.rst delete mode 100644 doc/source/monitoring.rst delete mode 100644 doc/source/multi.node.install.rst delete mode 100644 doc/source/network.rst delete mode 100644 doc/source/nova.manage.rst delete mode 100644 doc/source/nova.rst delete mode 100644 doc/source/objectstore.rst delete mode 100644 doc/source/quickstart.rst delete mode 100644 doc/source/scheduler.rst delete mode 100644 doc/source/services.rst delete mode 100644 doc/source/volume.rst (limited to 'doc/source') diff --git a/doc/source/adminguide/binaries.rst b/doc/source/adminguide/binaries.rst new file mode 100644 index 000000000..3ea07d9c2 --- /dev/null +++ b/doc/source/adminguide/binaries.rst @@ -0,0 +1,62 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _binaries: + +Nova Binaries +=============== + +The configuration of these binaries relies on "flagfiles" using the google +gflags package:: + + $ nova-xxxxx --flagfile flagfile + +The binaries can all run on the same machine or be spread out amongst multiple boxes in a large deployment. + +nova-manage +----------- + +Nova manage is a command line utility to administer the system. It will autmatically try to load a flagfile from /etc/nova/nova-manage.conf to save you having to type it. Info on the commands can be found :ref:`here `. + +nova-api +-------- + +Nova api receives xml requests and sends them to the rest of the system. It is a wsgi app that routes and authenticate requests. It supports the ec2 and openstack apis. + +nova-objectstore +---------------- + +Nova objectstore is an ultra simple file-based storage system for images that replicates most of the S3 Api. It will soon be replaced with glance and a simple image manager. + +nova-compute +------------ + +Nova compute is responsible for managing virtual machines. It loads a Service object which exposes the public methods on ComputeManager via rpc. + +nova-volume +----------- + +Nova volume is responsible for managing attachable block storage devices. It loads a Service object which exposes the public methods on VolumeManager via rpc. + +nova-network +------------ + +Nova network is responsible for managing floating and fixed ips, dhcp, bridging and vlans. It loads a Service object which exposes the public methods on one of the subclasses of NetworkManager. Different networking strategies are as simple as changing the network_manager flag:: + + $ nova-network --network_manager=nova.network.manager.FlatManager + +IMPORTANT: Make sure that you also set the network_manager on nova-api and nova_compute, since make some calls to network manager in process instead of through rpc. More information on the interactions between services, managers, and drivers can be found :ref:`here ` diff --git a/doc/source/adminguide/flags.rst b/doc/source/adminguide/flags.rst new file mode 100644 index 000000000..4c950aa88 --- /dev/null +++ b/doc/source/adminguide/flags.rst @@ -0,0 +1,23 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Flags and Flagfiles +=================== + +* python-gflags +* flagfiles +* list of flags by component (see concepts list) diff --git a/doc/source/adminguide/getting.started.rst b/doc/source/adminguide/getting.started.rst new file mode 100644 index 000000000..65ba2d57e --- /dev/null +++ b/doc/source/adminguide/getting.started.rst @@ -0,0 +1,167 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Getting Started with Nova +========================= + +This code base is continually changing, so dependencies also change. If you +encounter any problems, see the :doc:`../reaching.out` page. +The `contrib/nova.sh` script should be kept up to date, and may be a good +resource to review when debugging. + +The purpose of this document is to get a system installed that you can use to +test your setup assumptions. Working from this base installtion you can +tweak configurations and work with different flags to monitor interaction with +your hardware, network, and other factors that will allow you to determine +suitability for your deployment. After following this setup method, you should +be able to experiment with different managers, drivers, and flags to get the +best performance. + +Dependencies +------------ + +Related servers we rely on + +* **RabbitMQ**: messaging queue, used for all communication between components + +Optional servers + +* **OpenLDAP**: By default, the auth server uses the RDBMS-backed datastore by + setting FLAGS.auth_driver to `nova.auth.dbdriver.DbDriver`. But OpenLDAP + (or LDAP) could be configured by specifying `nova.auth.ldapdriver.LdapDriver`. + There is a script in the sources (`nova/auth/slap.sh`) to install a very basic + openldap server on ubuntu. +* **ReDIS**: There is a fake ldap auth driver + `nova.auth.ldapdriver.FakeLdapDriver` that backends to redis. This was + created for testing ldap implementation on systems that don't have an easy + means to install ldap. +* **MySQL**: Either MySQL or another database supported by sqlalchemy needs to + be avilable. Currently, only sqlite3 an mysql have been tested. + +Python libraries that we use (from pip-requires): + +.. literalinclude:: ../../../tools/pip-requires + +Other libraries: + +* **XenAPI**: Needed only for Xen Cloud Platform or XenServer support. Available + from http://wiki.xensource.com/xenwiki/XCP_SDK or + http://community.citrix.com/cdn/xs/sdks. + +External unix tools that are required: + +* iptables +* ebtables +* gawk +* curl +* kvm +* libvirt +* dnsmasq +* vlan +* open-iscsi and iscsitarget (if you use iscsi volumes) +* aoetools and vblade-persist (if you use aoe-volumes) + +Nova uses cutting-edge versions of many packages. There are ubuntu packages in +the nova-core ppa. You can use add this ppa to your sources list on an ubuntu +machine with the following commands:: + + sudo apt-get install -y python-software-properties + sudo add-apt-repository ppa:nova-core/ppa + +Recommended +----------- + +* euca2ools: python implementation of aws ec2-tools and ami tools +* build tornado to use C module for evented section + + +Installation +-------------- + +You can install from packages for your particular Linux distribution if they are +available. Otherwise you can install from source by checking out the source +files from the `Nova Source Code Repository `_ +and running:: + + python setup.py install + +Configuration +--------------- + +Configuring the host system +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As you read through the Administration Guide you will notice configuration hints +inline with documentation on the subsystem you are configuring. Presented in +this "Getting Started with Nova" document, we only provide what you need to +get started as quickly as possible. For a more detailed description of system +configuration, start reading through :doc:`multi.node.install`. + +* Create a volume group (you can use an actual disk for the volume group as + well):: + + # This creates a 1GB file to create volumes out of + dd if=/dev/zero of=MY_FILE_PATH bs=100M count=10 + losetup --show -f MY_FILE_PATH + # replace /dev/loop0 below with whatever losetup returns + # nova-volumes is the default for the --volume_group flag + vgcreate nova-volumes /dev/loop0 + + +Configuring Nova +~~~~~~~~~~~~~~~~ + +Configuration of the entire system is performed through python-gflags. The +best way to track configuration is through the use of a flagfile. + +A flagfile is specified with the ``--flagfile=FILEPATH`` argument to the binary +when you launch it. Flagfiles for nova are typically stored in +``/etc/nova/nova.conf``, and flags specific to a certain program are stored in +``/etc/nova/nova-COMMAND.conf``. Each configuration file can include another +flagfile, so typically a file like ``nova-manage.conf`` would have as its first +line ``--flagfile=/etc/nova/nova.conf`` to load the common flags before +specifying overrides or additional options. + +A sample configuration to test the system follows:: + + --verbose + --nodaemon + --FAKE_subdomain=ec2 + --auth_driver=nova.auth.dbdriver.DbDriver + +Running +--------- + +There are many parts to the nova system, each with a specific function. They +are built to be highly-available, so there are may configurations they can be +run in (ie: on many machines, many listeners per machine, etc). This part +of the guide only gets you started quickly, to learn about HA options, see +:doc:`multi.node.install`. + +Launch supporting services + +* rabbitmq +* redis (optional) +* mysql (optional) + +Launch nova components, each should have ``--flagfile=/etc/nova/nova.conf`` + +* nova-api +* nova-compute +* nova-objectstore +* nova-volume +* nova-scheduler diff --git a/doc/source/adminguide/index.rst b/doc/source/adminguide/index.rst new file mode 100644 index 000000000..ad1788b8b --- /dev/null +++ b/doc/source/adminguide/index.rst @@ -0,0 +1,36 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Administration Guide +==================== + +How to deploy, monitor, and debug Nova. + + +Contents +-------- + +.. toctree:: + :maxdepth: 1 + + quickstart + getting.started + binaries + multi.node.install + nova.manage + flags + monitoring diff --git a/doc/source/adminguide/monitoring.rst b/doc/source/adminguide/monitoring.rst new file mode 100644 index 000000000..e7766a6e7 --- /dev/null +++ b/doc/source/adminguide/monitoring.rst @@ -0,0 +1,27 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Monitoring +========== + +* components +* throughput +* exceptions +* hardware + +* ganglia +* syslog diff --git a/doc/source/adminguide/multi.node.install.rst b/doc/source/adminguide/multi.node.install.rst new file mode 100644 index 000000000..1d484e779 --- /dev/null +++ b/doc/source/adminguide/multi.node.install.rst @@ -0,0 +1,57 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Multi-Node Nova +=============== + +When you move beyond evaluating the technology and into building an actual +production environemnt, you will need to know how to configure your datacenter +and how to deploy components across your clusters. This guide should help you +through that process. + +Bare-metal Provisioning +----------------------- + +To install the base operating system you can use PXE booting. + +Deployment Technologies +----------------------- + +Once you have machines with a base operating system installation, you can deploy +code and configuration with your favorite tools: + +* Puppet +* Chef + +Types of Hosts +-------------- + +A single machine in your cluster can act as one or more of the following types +of host: + +Nova Services + +* Network +* Compute +* Volume +* API +* Objectstore + +Other supporting services + +* Datastore +* Message Queue diff --git a/doc/source/adminguide/nova.manage.rst b/doc/source/adminguide/nova.manage.rst new file mode 100644 index 000000000..683cf5e14 --- /dev/null +++ b/doc/source/adminguide/nova.manage.rst @@ -0,0 +1,200 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + +nova-manage +=========== + +Introduction +~~~~~~~~~~~~ + +The nova-manage command is used to perform many essential functions for +administration and ongoing maintenance of nova, such as user creation, +vpn management, and much more. + +The standard pattern for executing a nova-manage command is: + +``nova-manage []`` + +For example, to obtain a list of all projects: + +``nova-manage project list`` + +You can run without arguments to see a list of available command categories: + +``nova-manage`` + +You can run with a category argument to see a list of all commands in that +category: + +``nova-manage user`` + +User Maintenance +~~~~~~~~~~~~~~~~ + +Users, including admins, are created through the ``user`` commands. + +* user admin: creates a new admin and prints exports + * arguments: name [access] [secret] +* user create: creates a new user and prints exports + * arguments: name [access] [secret] +* user delete: deletes an existing user + * arguments: name +* user exports: prints access and secrets for user in export format + * arguments: name +* user list: lists all users + * arguments: none +* user modify: update a users keys & admin flag + * arguments: accesskey secretkey admin + * leave any field blank to ignore it, admin should be 'T', 'F', or blank + +Project Maintenance +~~~~~~~~~~~~~~~~~~~ + +* project add: Adds user to project + * arguments: project user +* project create: Creates a new project + * arguments: name project_manager [description] +* project delete: Deletes an existing project + * arguments: project_id +* project environment: Exports environment variables to an sourcable file + * arguments: project_id user_id [filename='novarc] +* project list: lists all projects + * arguments: none +* project quota: Set or display quotas for project + * arguments: project_id [key] [value] +* project remove: Removes user from project + * arguments: project user +* project scrub: Deletes data associated with project + * arguments: project +* project zipfile: Exports credentials for project to a zip file + * arguments: project_id user_id [filename='nova.zip] + +User Role Management +~~~~~~~~~~~~~~~~~~~~ + +* role add: adds role to user + * if project is specified, adds project specific role + * arguments: user, role [project] +* role has: checks to see if user has role + * if project is specified, returns True if user has + the global role and the project role + * arguments: user, role [project] +* role remove: removes role from user + * if project is specified, removes project specific role + * arguments: user, role [project] + + +Nova Shell +~~~~~~~~~~ + +* shell bpython + * start a new bpython shell +* shell ipython + * start a new ipython shell +* shell python + * start a new python shell +* shell run + * ??? +* shell script: Runs the script from the specifed path with flags set properly. + * arguments: path + +VPN Management +~~~~~~~~~~~~~~ + +* vpn list: Print a listing of the VPNs for all projects. + * arguments: none +* vpn run: Start the VPN for a given project. + * arguments: project +* vpn spawn: Run all VPNs. + * arguments: none + + +Floating IP Management +~~~~~~~~~~~~~~~~~~~~~~ + +* floating create: Creates floating ips for host by range + * arguments: host ip_range +* floating delete: Deletes floating ips by range + * arguments: range +* floating list: Prints a listing of all floating ips + * arguments: none + +Network Management +~~~~~~~~~~~~~~~~~~ + +* network create: Creates fixed ips for host by range + * arguments: [fixed_range=FLAG], [num_networks=FLAG], + [network_size=FLAG], [vlan_start=FLAG], + [vpn_start=FLAG] + + +Concept: Flags +-------------- + +python-gflags + + +Concept: Plugins +---------------- + +* Managers/Drivers: utils.import_object from string flag +* virt/connections: conditional loading from string flag +* db: LazyPluggable via string flag +* auth_manager: utils.import_class based on string flag +* Volumes: moving to pluggable driver instead of manager +* Network: pluggable managers +* Compute: same driver used, but pluggable at connection + + +Concept: IPC/RPC +---------------- + +Rabbit! + + +Concept: Fakes +-------------- + +* auth +* ldap + + +Concept: Scheduler +------------------ + +* simple +* random + + +Concept: Security Groups +------------------------ + +Security groups + + +Concept: Certificate Authority +------------------------------ + +Nova does a small amount of certificate management. These certificates are used for :ref:`project vpns <../cloudpipe>` and decrypting bundled images. + + +Concept: Images +--------------- + +* launching +* bundling diff --git a/doc/source/adminguide/quickstart.rst b/doc/source/adminguide/quickstart.rst new file mode 100644 index 000000000..511ed2483 --- /dev/null +++ b/doc/source/adminguide/quickstart.rst @@ -0,0 +1,104 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Nova Quickstart +=============== + +:: + + TODO(todd): * Document the assumptions about pluggable interfaces + (sqlite3 instead of mysql, etc) + * Document env vars that can change things + (USE_MYSQL, HOST_IP) + + +The `contrib/novascript.sh` file in the source distribution is a script that +will quickly set up nova to run on a single machine. It is tested against +Ubuntu only, but other distributions are forthcoming. + +Usage +----- + +Unless you want to spend a lot of time fiddling with permissions and sudoers, +you should probably run nova as root. + +:: + + sudo -i + +If you are concerned about security, nova runs just fine inside a virtual +machine. + +Use the script to install and run the current trunk. You can also specify a +specific branch by putting `lp:~someone/nova/some-branch` after the branch +command + +:: + + ./nova.sh branch + ./nova.sh install + ./nova.sh run + +The run command will drop you into a screen session with all of the workers +running in different windows You can use eucatools to run commands against the +cloud. + +:: + + euca-add-keypair test > test.pem + euca-run-instances -k test -t m1.tiny ami-tiny + euca-describe-instances + +To see output from the various workers, switch screen windows + +:: + + " + +will give you a list of running windows. + +When the instance is running, you should be able to ssh to it. + +:: + + chmod 600 test.pem + ssh -i test.pem root@10.0.0.3 + +When you exit screen + +:: + + + +nova will terminate. It may take a while for nova to finish cleaning up. If +you exit the process before it is done because there were some problems in your +build, you may have to clean up the nova processes manually. If you had any +instances running, you can attempt to kill them through the api: + +:: + + ./nova.sh terminate + +Then you can destroy the screen: + +:: + + ./nova.sh clean + +If things get particularly messed up, you might need to do some more intense +cleanup. Be careful, the following command will manually destroy all runnning +virsh instances and attempt to delete all vlans and bridges. diff --git a/doc/source/administration.guide.rst b/doc/source/administration.guide.rst deleted file mode 100644 index 2d09bed71..000000000 --- a/doc/source/administration.guide.rst +++ /dev/null @@ -1,35 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Administration Guide -==================== - -How to deploy, monitor, and debug Nova. - - -Contents --------- - -.. toctree:: - :maxdepth: 1 - - quickstart - getting.started - multi.node.install - nova.manage - flags - monitoring diff --git a/doc/source/api.rst b/doc/source/api.rst deleted file mode 100644 index 0908d9e57..000000000 --- a/doc/source/api.rst +++ /dev/null @@ -1,271 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -API Endpoint Programming Guide -============================== - -:: - - TODO(todd): get actual docstrings from ec2/osapi_verions instead of @wsgify - -Nova has a system for managing multiple APIs on different subdomains. -Currently there is support for the OpenStack API, as well as the Amazon EC2 -API. - -Common Components ------------------ - -The :mod:`nova.api` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.api - :members: - :undoc-members: - :show-inheritance: - -The :mod:`cloud` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.api.cloud - :members: - :undoc-members: - :show-inheritance: - -OpenStack API -------------- - -The :mod:`openstack` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack - :members: - :undoc-members: - :show-inheritance: - -The :mod:`auth` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack.auth - :members: - :undoc-members: - :show-inheritance: - -The :mod:`backup_schedules` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack.backup_schedules - :members: - :undoc-members: - :show-inheritance: - -The :mod:`faults` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack.faults - :members: - :undoc-members: - :show-inheritance: - -The :mod:`flavors` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack.flavors - :members: - :undoc-members: - :show-inheritance: - -The :mod:`images` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack.images - :members: - :undoc-members: - :show-inheritance: - -The :mod:`ratelimiting` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack.ratelimiting - :members: - :undoc-members: - :show-inheritance: - -The :mod:`servers` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack.servers - :members: - :undoc-members: - :show-inheritance: - -The :mod:`sharedipgroups` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: nova.api.openstack.sharedipgroups - :members: - :undoc-members: - :show-inheritance: - -EC2 API -------- - -The :mod:`nova.api.ec2` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.api.ec2 - :members: - :undoc-members: - :show-inheritance: - -The :mod:`admin` Module -~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.api.ec2.admin - :members: - :undoc-members: - :show-inheritance: - -The :mod:`apirequest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.api.ec2.apirequest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`cloud` Module -~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.api.ec2.cloud - :members: - :undoc-members: - :show-inheritance: - -The :mod:`images` Module -~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.api.ec2.images - :members: - :undoc-members: - :show-inheritance: - -The :mod:`metadatarequesthandler` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.api.ec2.metadatarequesthandler - :members: - :undoc-members: - :show-inheritance: - -Tests ------ - -The :mod:`api_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`api_integration` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api_integration - :members: - :undoc-members: - :show-inheritance: - -The :mod:`cloud_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.cloud_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`api.fakes` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.fakes - :members: - :undoc-members: - :show-inheritance: - -The :mod:`api.test_wsgi` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.test_wsgi - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test_api` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.openstack.test_api - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test_auth` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.openstack.test_auth - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test_faults` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.openstack.test_faults - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test_flavors` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.openstack.test_flavors - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test_images` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.openstack.test_images - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test_ratelimiting` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.openstack.test_ratelimiting - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test_servers` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.openstack.test_servers - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test_sharedipgroups` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.api.openstack.test_sharedipgroups - :members: - :undoc-members: - :show-inheritance: - diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst deleted file mode 100644 index 1e23e1361..000000000 --- a/doc/source/architecture.rst +++ /dev/null @@ -1,52 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Nova System Architecture -======================== - -Nova is built on a shared-nothing, messaging-based architecture. All of the major nova components can be run on multiple servers. This means that most component to component communication must go via message queue. In order to avoid blocking each component while waiting for a response, we use deferred objects, with a callback that gets triggered when a response is received. - -Nova recently moved to using a sql-based central database that is shared by all components in the system. The amount and depth of the data fits into a sql database quite well. For small deployments this seems like an optimal solution. For larger deployments, and especially if security is a concern, nova will be moving towards multiple data stores with some kind of aggregation system. - -Components ----------- - -Below you will find a helpful explanation of the different components. - -:: - - /- ( LDAP ) - [ Auth Manager ] --- - | \- ( DB ) - | - | [ scheduler ] - [ volume ] - ( ATAoE/iSCSI ) - | / - [ Web Dashboard ] -> [ api ] -- < AMQP > ------ [ network ] - ( Flat/Vlan ) - | \ - < HTTP > [ scheduler ] - [ compute ] - ( libvirt/xen ) - | | - [ objectstore ] < - retrieves images - -* DB: sql database for data storage. Used by all components (LINKS NOT SHOWN) -* Web Dashboard: potential external component that talks to the api -* api: component that receives http requests, converts commands and communicates with other components via the queue or http (in the case of objectstore) -* Auth Manager: component responsible for users/projects/and roles. Can backend to DB or LDAP. This is not a separate binary, but rather a python class that is used by most components in the system. -* objectstore: twisted http server that replicates s3 api and allows storage and retrieval of images -* scheduler: decides which host gets each vm and volume -* volume: manages dynamically attachable block devices. -* network: manages ip forwarding, bridges, and vlans -* compute: manages communication with hypervisor and virtual machines. diff --git a/doc/source/auth.rst b/doc/source/auth.rst deleted file mode 100644 index 77d97f68b..000000000 --- a/doc/source/auth.rst +++ /dev/null @@ -1,258 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _auth: - -Authentication and Authorization Programming Guide -================================================== - -The :mod:`quota` Module ------------------------ - -.. automodule:: nova.quota - :members: - :undoc-members: - :show-inheritance: - -Auth Manager ------------- - -The :mod:`manager` Module -~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.auth.manager - :members: - :undoc-members: - :show-inheritance: - -Drivers -------- - -The :mod:`ldapdriver` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.auth.ldapdriver - :members: - :undoc-members: - :show-inheritance: - -The :mod:`dbdriver` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.auth.dbdriver - :members: - :undoc-members: - :show-inheritance: - -API Integration ---------------- - -The :mod:`signer` Module -~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.auth.signer - :members: - :undoc-members: - :show-inheritance: - -Related Tests -------------- - -The :mod:`auth_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.auth_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`access_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.access_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`quota_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.quota_unittest - :members: - :undoc-members: - :show-inheritance: - -legacy docs ------------ - -Nova provides RBAC (Role-based access control) of the AWS-type APIs. We define the following roles: - -Roles-Based Access Control of AWS-style APIs using SAML Assertions -“Achieving FIPS 199 Moderate certification of a hybrid cloud environment using CloudAudit and declarative C.I.A. classifications” - -Introduction ------------- - -We will investigate one method for integrating an AWS-style API with US eAuthentication-compatible federated authentication systems, to achieve access controls and limits based on traditional operational roles. -Additionally, we will look at how combining this approach, with an implementation of the CloudAudit APIs, will allow us to achieve a certification under FIPS 199 Moderate classification for a hybrid cloud environment. - -Relationship of US eAuth to RBAC --------------------------------- - -Typical implementations of US eAuth authentication systems are structured as follows:: - - [ MS Active Directory or other federated LDAP user store ] - --> backends to… - [ SUN Identity Manager or other SAML Policy Controller ] - --> maps URLs to groups… - [ Apache Policy Agent in front of eAuth-secured Web Application ] - -In more ideal implementations, the remainder of the application-specific account information is stored either in extended schema on the LDAP server itself, via the use of a translucent LDAP proxy, or in an independent datastore keyed off of the UID provided via SAML assertion. - -.. _auth_roles: - -Roles ------ - -AWS API calls are traditionally secured via Access and Secret Keys, which are used to sign API calls, along with traditional timestamps to prevent replay attacks. The APIs can be logically grouped into sets that align with five typical roles: - -* Base User -* System Administrator/Developer (currently have the same permissions) -* Network Administrator -* Project Manager -* Cloud Administrator/IT-Security (currently have the same permissions) - -There is an additional, conceptual end-user that may or may not have API access: - -* (EXTERNAL) End-user / Third-party User - -Basic operations are available to any : - -* Describe Instances -* Describe Images -* Describe Volumes -* Describe Keypairs -* Create Keypair -* Delete Keypair -* Create, Upload, Delete: Buckets and Keys (Object Store) - -System Administrators/Developers/Project Manager: - -* Create, Attach, Delete Volume (Block Store) -* Launch, Reboot, Terminate Instance -* Register/Unregister Machine Image (project-wide) -* Request / Review CloudAudit Scans - -Project Manager: - -* Add and remove other users (currently no api) -* Set roles (currently no api) - -Network Administrator: - -* Change Machine Image properties (public / private) -* Change Firewall Rules, define Security Groups -* Allocate, Associate, Deassociate Public IP addresses - -Cloud Administrator/IT-Security: - -* All permissions - -Enhancements ------------- - -* SAML Token passing -* REST interfaces -* SOAP interfaces - -Wrapping the SAML token into the API calls. -Then store the UID (fetched via backchannel) into the instance metadata, providing end-to-end auditability of ownership and responsibility, without PII. - -CloudAudit APIs ---------------- - -* Request formats -* Response formats -* Stateless asynchronous queries - -CloudAudit queries may spawn long-running processes (similar to launching instances, etc.) They need to return a ReservationId in the same fashion, which can be returned in further queries for updates. -RBAC of CloudAudit API calls is critical, since detailed system information is a system vulnerability. - -Type declarations ------------------ -* Data declarations – Volumes and Objects -* System declarations – Instances - -Existing API calls to launch instances specific a single, combined “type” flag. We propose to extend this with three additional type declarations, mapping to the “Confidentiality, Integrity, Availability” classifications of FIPS 199. An example API call would look like:: - - RunInstances type=m1.large number=1 secgroup=default key=mykey confidentiality=low integrity=low availability=low - -These additional parameters would also apply to creation of block storage volumes (along with the existing parameter of ‘size’), and creation of object storage ‘buckets’. (C.I.A. classifications on a bucket would be inherited by the keys within this bucket.) - -Request Brokering ------------------ - - * Cloud Interop - * IMF Registration / PubSub - * Digital C&A - -Establishing declarative semantics for individual API calls will allow the cloud environment to seamlessly proxy these API calls to external, third-party vendors – when the requested CIA levels match. - -See related work within the Infrastructure 2.0 working group for more information on how the IMF Metadata specification could be utilized to manage registration of these vendors and their C&A credentials. - -Dirty Cloud – Hybrid Data Centers ---------------------------------- - -* CloudAudit bridge interfaces -* Anything in the ARP table - -A hybrid cloud environment provides dedicated, potentially co-located physical hardware with a network interconnect to the project or users’ cloud virtual network. - -This interconnect is typically a bridged VPN connection. Any machines that can be bridged into a hybrid environment in this fashion (at Layer 2) must implement a minimum version of the CloudAudit spec, such that they can be queried to provide a complete picture of the IT-sec runtime environment. - -Network discovery protocols (ARP, CDP) can be applied in this case, and existing protocols (SNMP location data, DNS LOC records) overloaded to provide CloudAudit information. - -The Details ------------ - - * Preliminary Roles Definitions - * Categorization of available API calls - * SAML assertion vocabulary - -System limits -------------- - -The following limits need to be defined and enforced: - -* Total number of instances allowed (user / project) -* Total number of instances, per instance type (user / project) -* Total number of volumes (user / project) -* Maximum size of volume -* Cumulative size of all volumes -* Total use of object storage (GB) -* Total number of Public IPs - - -Further Challenges ------------------- - * Prioritization of users / jobs in shared computing environments - * Incident response planning - * Limit launch of instances to specific security groups based on AMI - * Store AMIs in LDAP for added property control - - - diff --git a/doc/source/binaries.rst b/doc/source/binaries.rst deleted file mode 100644 index 3ea07d9c2..000000000 --- a/doc/source/binaries.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _binaries: - -Nova Binaries -=============== - -The configuration of these binaries relies on "flagfiles" using the google -gflags package:: - - $ nova-xxxxx --flagfile flagfile - -The binaries can all run on the same machine or be spread out amongst multiple boxes in a large deployment. - -nova-manage ------------ - -Nova manage is a command line utility to administer the system. It will autmatically try to load a flagfile from /etc/nova/nova-manage.conf to save you having to type it. Info on the commands can be found :ref:`here `. - -nova-api --------- - -Nova api receives xml requests and sends them to the rest of the system. It is a wsgi app that routes and authenticate requests. It supports the ec2 and openstack apis. - -nova-objectstore ----------------- - -Nova objectstore is an ultra simple file-based storage system for images that replicates most of the S3 Api. It will soon be replaced with glance and a simple image manager. - -nova-compute ------------- - -Nova compute is responsible for managing virtual machines. It loads a Service object which exposes the public methods on ComputeManager via rpc. - -nova-volume ------------ - -Nova volume is responsible for managing attachable block storage devices. It loads a Service object which exposes the public methods on VolumeManager via rpc. - -nova-network ------------- - -Nova network is responsible for managing floating and fixed ips, dhcp, bridging and vlans. It loads a Service object which exposes the public methods on one of the subclasses of NetworkManager. Different networking strategies are as simple as changing the network_manager flag:: - - $ nova-network --network_manager=nova.network.manager.FlatManager - -IMPORTANT: Make sure that you also set the network_manager on nova-api and nova_compute, since make some calls to network manager in process instead of through rpc. More information on the interactions between services, managers, and drivers can be found :ref:`here ` diff --git a/doc/source/cloudpipe.rst b/doc/source/cloudpipe.rst deleted file mode 100644 index d632a40e6..000000000 --- a/doc/source/cloudpipe.rst +++ /dev/null @@ -1,77 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - -.. _cloudpipe: - -Cloudpipe -- Per Project Vpns -============================= - -Cloudpipe is a method for connecting end users to their project insnances in vlan mode. - -Overview --------- - -The support code for cloudpipe implements admin commands (via nova-manage) to automatically create a vm for a project that allows users to vpn into the private network of their project. Access to this vpn is provided through a public port on the network host for the project. This allows users to have free access to the virtual machines in their project without exposing those machines to the public internet. - -Cloudpipe Image ---------------- - -The cloudpipe image is basically just a linux instance with openvpn installed. It needs a simple script to grab user data from the metadata server, b64 decode it into a zip file, and run the autorun.sh script from inside the zip. The autorun script will configure and run openvpn to run using the data from nova. - -It is also useful to have a cron script that will periodically redownload the metadata and copy the new crl. This will keep revoked users from connecting and will disconnect any users that are connected with revoked certificates when their connection is renegotiated (every hour). - -Cloudpipe Launch ----------------- - -When you use nova-manage to launch a cloudpipe for a user, it goes through the following process: - -#. creates a keypair called -vpn and saves it in the keys directory -#. creates a security group -vpn and opens up 1194 and icmp -#. creates a cert and private key for the vpn instance and saves it in the CA/projects// directory -#. zips up the info and puts it b64 encoded as user data -#. launches an m1.tiny instance with the above settings using the flag-specified vpn image - -Vpn Access ----------- - -In vlan networking mode, the second ip in each private network is reserved for the cloudpipe instance. This gives a consistent ip to the instance so that nova-network can create forwarding rules for access from the outside world. The network for each project is given a specific high-numbered port on the public ip of the network host. This port is automatically forwarded to 1194 on the vpn instance. - -If specific high numbered ports do not work for your users, you can always allocate and associate a public ip to the instance, and then change the vpn_public_ip and vpn_public_port in the database. This will be turned into a nova-manage command or a flag soon. - - -Certificates and Revocation ---------------------------- - -If the use_project_ca flag is set (required to for cloudpipes to work securely), then each project has its own ca. This ca is used to sign the certificate for the vpn, and is also passed to the user for bundling images. When a certificate is revoked using nova-manage, a new Certificate Revocation List (crl) is generated. As long as cloudpipe has an updated crl, it will block revoked users from connecting to the vpn. - -The :mod:`cloudpipe` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.cloudpipe.pipelib - :members: - :undoc-members: - :show-inheritance: - -The :mod:`crypto` Module -~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.crypto - :members: - :undoc-members: - :show-inheritance: - diff --git a/doc/source/compute.rst b/doc/source/compute.rst deleted file mode 100644 index e4c6c6ae7..000000000 --- a/doc/source/compute.rst +++ /dev/null @@ -1,146 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - -Virtualization Programming Guide -================================ - -This page contains the Compute Package documentation. - -Manager -------- - -Documentation for the compute manager and related files. For reading about -a specific virtualization backend, read Drivers_. - - -The :mod:`manager` Module -~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.compute.manager - :members: - :undoc-members: - :show-inheritance: - -The :mod:`connection` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.virt.connection - :members: - :undoc-members: - :show-inheritance: - -The :mod:`disk` Module -~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.compute.disk - :members: - :undoc-members: - :show-inheritance: - -The :mod:`images` Module -~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.virt.images - :members: - :undoc-members: - :show-inheritance: - - -The :mod:`instance_types` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.compute.instance_types - :members: - :undoc-members: - :show-inheritance: - - -The :mod:`power_state` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.compute.power_state - :members: - :undoc-members: - :show-inheritance: - - -Drivers -------- - -Libvirt Implementations -~~~~~~~~~~~~~~~~~~~~~~~ - -The libvirt driver is capable of supporting KVM, QEMU, and UML. - -The :mod:`libvirt_conn` Module -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: nova.virt.libvirt_conn - :members: - :undoc-members: - :show-inheritance: - -XEN -~~~ - -The :mod:`xenapi` Module -^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: nova.virt.xenapi - :members: - :undoc-members: - :show-inheritance: - -FAKE -~~~~ - -.. automodule:: nova.virt.fake - :members: - :undoc-members: - :show-inheritance: - -Monitoring ----------- - -The :mod:`monitor` Module -~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.compute.monitor - :members: - :undoc-members: - :show-inheritance: - - -Tests ------ - -The :mod:`compute_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.compute_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`virt_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.virt_unittest - :members: - :undoc-members: - :show-inheritance: diff --git a/doc/source/concepts.and.introduction.rst b/doc/source/concepts.and.introduction.rst index 5810280b4..d409fbaec 100644 --- a/doc/source/concepts.and.introduction.rst +++ b/doc/source/concepts.and.introduction.rst @@ -35,7 +35,7 @@ focues on describing how Nova's implementation of those concepts is achieved. This page outlines concepts that you will need to understand as a user or administrator of an OpenStack installation. Each section links to more more -detailed information in the :doc:`administration.guide`, +detailed information in the :doc:`adminguide/index`, but you'll probably want to read this section straight-through before tackling the specifics presented in the administration guide. diff --git a/doc/source/database.rst b/doc/source/database.rst deleted file mode 100644 index b58ea147d..000000000 --- a/doc/source/database.rst +++ /dev/null @@ -1,61 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Database Programming Guide -========================== - -The :mod:`api` Module ---------------------- - -.. automodule:: nova.db.api - :members: - :undoc-members: - :show-inheritance: - - -Drivers -------- - -Driver: Sqlalchemy -~~~~~~~~~~~~~~~~~~ - -The :mod:`sqlalchemy.api` Module -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: nova.db.sqlalchemy.api - -The :mod:`sqlalchemy.models` Module -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: nova.db.sqlalchemy.models - :members: - :undoc-members: - :show-inheritance: - -The :mod:`sqlalchemy.session` Module -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. automodule:: nova.db.sqlalchemy.session - :members: - :undoc-members: - :show-inheritance: - -Tests ------ - -Tests are lacking for the db api layer and for the sqlalchemy driver. -Failures in the drivers would be dectected in other test cases, though. diff --git a/doc/source/development.environment.rst b/doc/source/development.environment.rst deleted file mode 100644 index ee19ba2bf..000000000 --- a/doc/source/development.environment.rst +++ /dev/null @@ -1,23 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Setting up a development environment -==================================== - -:: - - TODO(anthony) diff --git a/doc/source/development.guide.rst b/doc/source/development.guide.rst deleted file mode 100644 index 704cbe399..000000000 --- a/doc/source/development.guide.rst +++ /dev/null @@ -1,60 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Nova Development Guide -====================== - -Nova is written in python. - -Prerequisites -------------- -.. toctree:: - :maxdepth: 1 - - concepts.and.introduction - architecture - development.environment - - -Contents --------- - -.. toctree:: - :maxdepth: 1 - - services - database - volume - compute - network - auth - api - scheduler - fakes - nova - cloudpipe - objectstore - glance - - -Indices and tables ------------------- - -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` - diff --git a/doc/source/devguide/api.rst b/doc/source/devguide/api.rst new file mode 100644 index 000000000..0908d9e57 --- /dev/null +++ b/doc/source/devguide/api.rst @@ -0,0 +1,271 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +API Endpoint Programming Guide +============================== + +:: + + TODO(todd): get actual docstrings from ec2/osapi_verions instead of @wsgify + +Nova has a system for managing multiple APIs on different subdomains. +Currently there is support for the OpenStack API, as well as the Amazon EC2 +API. + +Common Components +----------------- + +The :mod:`nova.api` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api + :members: + :undoc-members: + :show-inheritance: + +The :mod:`cloud` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.cloud + :members: + :undoc-members: + :show-inheritance: + +OpenStack API +------------- + +The :mod:`openstack` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack + :members: + :undoc-members: + :show-inheritance: + +The :mod:`auth` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.auth + :members: + :undoc-members: + :show-inheritance: + +The :mod:`backup_schedules` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.backup_schedules + :members: + :undoc-members: + :show-inheritance: + +The :mod:`faults` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.faults + :members: + :undoc-members: + :show-inheritance: + +The :mod:`flavors` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.flavors + :members: + :undoc-members: + :show-inheritance: + +The :mod:`images` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.images + :members: + :undoc-members: + :show-inheritance: + +The :mod:`ratelimiting` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.ratelimiting + :members: + :undoc-members: + :show-inheritance: + +The :mod:`servers` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.servers + :members: + :undoc-members: + :show-inheritance: + +The :mod:`sharedipgroups` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. automodule:: nova.api.openstack.sharedipgroups + :members: + :undoc-members: + :show-inheritance: + +EC2 API +------- + +The :mod:`nova.api.ec2` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2 + :members: + :undoc-members: + :show-inheritance: + +The :mod:`admin` Module +~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.admin + :members: + :undoc-members: + :show-inheritance: + +The :mod:`apirequest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.apirequest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`cloud` Module +~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.cloud + :members: + :undoc-members: + :show-inheritance: + +The :mod:`images` Module +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.images + :members: + :undoc-members: + :show-inheritance: + +The :mod:`metadatarequesthandler` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.api.ec2.metadatarequesthandler + :members: + :undoc-members: + :show-inheritance: + +Tests +----- + +The :mod:`api_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`api_integration` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api_integration + :members: + :undoc-members: + :show-inheritance: + +The :mod:`cloud_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.cloud_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`api.fakes` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.fakes + :members: + :undoc-members: + :show-inheritance: + +The :mod:`api.test_wsgi` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.test_wsgi + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_api` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_api + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_auth` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_auth + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_faults` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_faults + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_flavors` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_flavors + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_images` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_images + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_ratelimiting` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_ratelimiting + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_servers` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_servers + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test_sharedipgroups` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.api.openstack.test_sharedipgroups + :members: + :undoc-members: + :show-inheritance: + diff --git a/doc/source/devguide/architecture.rst b/doc/source/devguide/architecture.rst new file mode 100644 index 000000000..1e23e1361 --- /dev/null +++ b/doc/source/devguide/architecture.rst @@ -0,0 +1,52 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Nova System Architecture +======================== + +Nova is built on a shared-nothing, messaging-based architecture. All of the major nova components can be run on multiple servers. This means that most component to component communication must go via message queue. In order to avoid blocking each component while waiting for a response, we use deferred objects, with a callback that gets triggered when a response is received. + +Nova recently moved to using a sql-based central database that is shared by all components in the system. The amount and depth of the data fits into a sql database quite well. For small deployments this seems like an optimal solution. For larger deployments, and especially if security is a concern, nova will be moving towards multiple data stores with some kind of aggregation system. + +Components +---------- + +Below you will find a helpful explanation of the different components. + +:: + + /- ( LDAP ) + [ Auth Manager ] --- + | \- ( DB ) + | + | [ scheduler ] - [ volume ] - ( ATAoE/iSCSI ) + | / + [ Web Dashboard ] -> [ api ] -- < AMQP > ------ [ network ] - ( Flat/Vlan ) + | \ + < HTTP > [ scheduler ] - [ compute ] - ( libvirt/xen ) + | | + [ objectstore ] < - retrieves images + +* DB: sql database for data storage. Used by all components (LINKS NOT SHOWN) +* Web Dashboard: potential external component that talks to the api +* api: component that receives http requests, converts commands and communicates with other components via the queue or http (in the case of objectstore) +* Auth Manager: component responsible for users/projects/and roles. Can backend to DB or LDAP. This is not a separate binary, but rather a python class that is used by most components in the system. +* objectstore: twisted http server that replicates s3 api and allows storage and retrieval of images +* scheduler: decides which host gets each vm and volume +* volume: manages dynamically attachable block devices. +* network: manages ip forwarding, bridges, and vlans +* compute: manages communication with hypervisor and virtual machines. diff --git a/doc/source/devguide/auth.rst b/doc/source/devguide/auth.rst new file mode 100644 index 000000000..77d97f68b --- /dev/null +++ b/doc/source/devguide/auth.rst @@ -0,0 +1,258 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _auth: + +Authentication and Authorization Programming Guide +================================================== + +The :mod:`quota` Module +----------------------- + +.. automodule:: nova.quota + :members: + :undoc-members: + :show-inheritance: + +Auth Manager +------------ + +The :mod:`manager` Module +~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.auth.manager + :members: + :undoc-members: + :show-inheritance: + +Drivers +------- + +The :mod:`ldapdriver` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.auth.ldapdriver + :members: + :undoc-members: + :show-inheritance: + +The :mod:`dbdriver` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.auth.dbdriver + :members: + :undoc-members: + :show-inheritance: + +API Integration +--------------- + +The :mod:`signer` Module +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.auth.signer + :members: + :undoc-members: + :show-inheritance: + +Related Tests +------------- + +The :mod:`auth_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.auth_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`access_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.access_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`quota_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.quota_unittest + :members: + :undoc-members: + :show-inheritance: + +legacy docs +----------- + +Nova provides RBAC (Role-based access control) of the AWS-type APIs. We define the following roles: + +Roles-Based Access Control of AWS-style APIs using SAML Assertions +“Achieving FIPS 199 Moderate certification of a hybrid cloud environment using CloudAudit and declarative C.I.A. classifications” + +Introduction +------------ + +We will investigate one method for integrating an AWS-style API with US eAuthentication-compatible federated authentication systems, to achieve access controls and limits based on traditional operational roles. +Additionally, we will look at how combining this approach, with an implementation of the CloudAudit APIs, will allow us to achieve a certification under FIPS 199 Moderate classification for a hybrid cloud environment. + +Relationship of US eAuth to RBAC +-------------------------------- + +Typical implementations of US eAuth authentication systems are structured as follows:: + + [ MS Active Directory or other federated LDAP user store ] + --> backends to… + [ SUN Identity Manager or other SAML Policy Controller ] + --> maps URLs to groups… + [ Apache Policy Agent in front of eAuth-secured Web Application ] + +In more ideal implementations, the remainder of the application-specific account information is stored either in extended schema on the LDAP server itself, via the use of a translucent LDAP proxy, or in an independent datastore keyed off of the UID provided via SAML assertion. + +.. _auth_roles: + +Roles +----- + +AWS API calls are traditionally secured via Access and Secret Keys, which are used to sign API calls, along with traditional timestamps to prevent replay attacks. The APIs can be logically grouped into sets that align with five typical roles: + +* Base User +* System Administrator/Developer (currently have the same permissions) +* Network Administrator +* Project Manager +* Cloud Administrator/IT-Security (currently have the same permissions) + +There is an additional, conceptual end-user that may or may not have API access: + +* (EXTERNAL) End-user / Third-party User + +Basic operations are available to any : + +* Describe Instances +* Describe Images +* Describe Volumes +* Describe Keypairs +* Create Keypair +* Delete Keypair +* Create, Upload, Delete: Buckets and Keys (Object Store) + +System Administrators/Developers/Project Manager: + +* Create, Attach, Delete Volume (Block Store) +* Launch, Reboot, Terminate Instance +* Register/Unregister Machine Image (project-wide) +* Request / Review CloudAudit Scans + +Project Manager: + +* Add and remove other users (currently no api) +* Set roles (currently no api) + +Network Administrator: + +* Change Machine Image properties (public / private) +* Change Firewall Rules, define Security Groups +* Allocate, Associate, Deassociate Public IP addresses + +Cloud Administrator/IT-Security: + +* All permissions + +Enhancements +------------ + +* SAML Token passing +* REST interfaces +* SOAP interfaces + +Wrapping the SAML token into the API calls. +Then store the UID (fetched via backchannel) into the instance metadata, providing end-to-end auditability of ownership and responsibility, without PII. + +CloudAudit APIs +--------------- + +* Request formats +* Response formats +* Stateless asynchronous queries + +CloudAudit queries may spawn long-running processes (similar to launching instances, etc.) They need to return a ReservationId in the same fashion, which can be returned in further queries for updates. +RBAC of CloudAudit API calls is critical, since detailed system information is a system vulnerability. + +Type declarations +----------------- +* Data declarations – Volumes and Objects +* System declarations – Instances + +Existing API calls to launch instances specific a single, combined “type” flag. We propose to extend this with three additional type declarations, mapping to the “Confidentiality, Integrity, Availability” classifications of FIPS 199. An example API call would look like:: + + RunInstances type=m1.large number=1 secgroup=default key=mykey confidentiality=low integrity=low availability=low + +These additional parameters would also apply to creation of block storage volumes (along with the existing parameter of ‘size’), and creation of object storage ‘buckets’. (C.I.A. classifications on a bucket would be inherited by the keys within this bucket.) + +Request Brokering +----------------- + + * Cloud Interop + * IMF Registration / PubSub + * Digital C&A + +Establishing declarative semantics for individual API calls will allow the cloud environment to seamlessly proxy these API calls to external, third-party vendors – when the requested CIA levels match. + +See related work within the Infrastructure 2.0 working group for more information on how the IMF Metadata specification could be utilized to manage registration of these vendors and their C&A credentials. + +Dirty Cloud – Hybrid Data Centers +--------------------------------- + +* CloudAudit bridge interfaces +* Anything in the ARP table + +A hybrid cloud environment provides dedicated, potentially co-located physical hardware with a network interconnect to the project or users’ cloud virtual network. + +This interconnect is typically a bridged VPN connection. Any machines that can be bridged into a hybrid environment in this fashion (at Layer 2) must implement a minimum version of the CloudAudit spec, such that they can be queried to provide a complete picture of the IT-sec runtime environment. + +Network discovery protocols (ARP, CDP) can be applied in this case, and existing protocols (SNMP location data, DNS LOC records) overloaded to provide CloudAudit information. + +The Details +----------- + + * Preliminary Roles Definitions + * Categorization of available API calls + * SAML assertion vocabulary + +System limits +------------- + +The following limits need to be defined and enforced: + +* Total number of instances allowed (user / project) +* Total number of instances, per instance type (user / project) +* Total number of volumes (user / project) +* Maximum size of volume +* Cumulative size of all volumes +* Total use of object storage (GB) +* Total number of Public IPs + + +Further Challenges +------------------ + * Prioritization of users / jobs in shared computing environments + * Incident response planning + * Limit launch of instances to specific security groups based on AMI + * Store AMIs in LDAP for added property control + + + diff --git a/doc/source/devguide/cloudpipe.rst b/doc/source/devguide/cloudpipe.rst new file mode 100644 index 000000000..d632a40e6 --- /dev/null +++ b/doc/source/devguide/cloudpipe.rst @@ -0,0 +1,77 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + +.. _cloudpipe: + +Cloudpipe -- Per Project Vpns +============================= + +Cloudpipe is a method for connecting end users to their project insnances in vlan mode. + +Overview +-------- + +The support code for cloudpipe implements admin commands (via nova-manage) to automatically create a vm for a project that allows users to vpn into the private network of their project. Access to this vpn is provided through a public port on the network host for the project. This allows users to have free access to the virtual machines in their project without exposing those machines to the public internet. + +Cloudpipe Image +--------------- + +The cloudpipe image is basically just a linux instance with openvpn installed. It needs a simple script to grab user data from the metadata server, b64 decode it into a zip file, and run the autorun.sh script from inside the zip. The autorun script will configure and run openvpn to run using the data from nova. + +It is also useful to have a cron script that will periodically redownload the metadata and copy the new crl. This will keep revoked users from connecting and will disconnect any users that are connected with revoked certificates when their connection is renegotiated (every hour). + +Cloudpipe Launch +---------------- + +When you use nova-manage to launch a cloudpipe for a user, it goes through the following process: + +#. creates a keypair called -vpn and saves it in the keys directory +#. creates a security group -vpn and opens up 1194 and icmp +#. creates a cert and private key for the vpn instance and saves it in the CA/projects// directory +#. zips up the info and puts it b64 encoded as user data +#. launches an m1.tiny instance with the above settings using the flag-specified vpn image + +Vpn Access +---------- + +In vlan networking mode, the second ip in each private network is reserved for the cloudpipe instance. This gives a consistent ip to the instance so that nova-network can create forwarding rules for access from the outside world. The network for each project is given a specific high-numbered port on the public ip of the network host. This port is automatically forwarded to 1194 on the vpn instance. + +If specific high numbered ports do not work for your users, you can always allocate and associate a public ip to the instance, and then change the vpn_public_ip and vpn_public_port in the database. This will be turned into a nova-manage command or a flag soon. + + +Certificates and Revocation +--------------------------- + +If the use_project_ca flag is set (required to for cloudpipes to work securely), then each project has its own ca. This ca is used to sign the certificate for the vpn, and is also passed to the user for bundling images. When a certificate is revoked using nova-manage, a new Certificate Revocation List (crl) is generated. As long as cloudpipe has an updated crl, it will block revoked users from connecting to the vpn. + +The :mod:`cloudpipe` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.cloudpipe.pipelib + :members: + :undoc-members: + :show-inheritance: + +The :mod:`crypto` Module +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.crypto + :members: + :undoc-members: + :show-inheritance: + diff --git a/doc/source/devguide/compute.rst b/doc/source/devguide/compute.rst new file mode 100644 index 000000000..e4c6c6ae7 --- /dev/null +++ b/doc/source/devguide/compute.rst @@ -0,0 +1,146 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + + +Virtualization Programming Guide +================================ + +This page contains the Compute Package documentation. + +Manager +------- + +Documentation for the compute manager and related files. For reading about +a specific virtualization backend, read Drivers_. + + +The :mod:`manager` Module +~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.compute.manager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`connection` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.virt.connection + :members: + :undoc-members: + :show-inheritance: + +The :mod:`disk` Module +~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.compute.disk + :members: + :undoc-members: + :show-inheritance: + +The :mod:`images` Module +~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.virt.images + :members: + :undoc-members: + :show-inheritance: + + +The :mod:`instance_types` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.compute.instance_types + :members: + :undoc-members: + :show-inheritance: + + +The :mod:`power_state` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.compute.power_state + :members: + :undoc-members: + :show-inheritance: + + +Drivers +------- + +Libvirt Implementations +~~~~~~~~~~~~~~~~~~~~~~~ + +The libvirt driver is capable of supporting KVM, QEMU, and UML. + +The :mod:`libvirt_conn` Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. automodule:: nova.virt.libvirt_conn + :members: + :undoc-members: + :show-inheritance: + +XEN +~~~ + +The :mod:`xenapi` Module +^^^^^^^^^^^^^^^^^^^^^^^^ + +.. automodule:: nova.virt.xenapi + :members: + :undoc-members: + :show-inheritance: + +FAKE +~~~~ + +.. automodule:: nova.virt.fake + :members: + :undoc-members: + :show-inheritance: + +Monitoring +---------- + +The :mod:`monitor` Module +~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.compute.monitor + :members: + :undoc-members: + :show-inheritance: + + +Tests +----- + +The :mod:`compute_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.compute_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`virt_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.virt_unittest + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/devguide/database.rst b/doc/source/devguide/database.rst new file mode 100644 index 000000000..b58ea147d --- /dev/null +++ b/doc/source/devguide/database.rst @@ -0,0 +1,61 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Database Programming Guide +========================== + +The :mod:`api` Module +--------------------- + +.. automodule:: nova.db.api + :members: + :undoc-members: + :show-inheritance: + + +Drivers +------- + +Driver: Sqlalchemy +~~~~~~~~~~~~~~~~~~ + +The :mod:`sqlalchemy.api` Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. automodule:: nova.db.sqlalchemy.api + +The :mod:`sqlalchemy.models` Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. automodule:: nova.db.sqlalchemy.models + :members: + :undoc-members: + :show-inheritance: + +The :mod:`sqlalchemy.session` Module +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. automodule:: nova.db.sqlalchemy.session + :members: + :undoc-members: + :show-inheritance: + +Tests +----- + +Tests are lacking for the db api layer and for the sqlalchemy driver. +Failures in the drivers would be dectected in other test cases, though. diff --git a/doc/source/devguide/development.environment.rst b/doc/source/devguide/development.environment.rst new file mode 100644 index 000000000..ee19ba2bf --- /dev/null +++ b/doc/source/devguide/development.environment.rst @@ -0,0 +1,23 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Setting up a development environment +==================================== + +:: + + TODO(anthony) diff --git a/doc/source/devguide/fakes.rst b/doc/source/devguide/fakes.rst new file mode 100644 index 000000000..5988e2ab8 --- /dev/null +++ b/doc/source/devguide/fakes.rst @@ -0,0 +1,74 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Fakes Programming Guide +======================= + +:: + + TODO(todd): * document general info about fakes + + +The :mod:`virt.fake` Module +--------------------------- + +.. automodule:: nova.virt.fake + :members: + :undoc-members: + :show-inheritance: + :noindex: + +The :mod:`fakeldap` Module +-------------------------- + +.. automodule:: nova.auth.fakeldap + :members: + :undoc-members: + :show-inheritance: + +The :mod:`fakerabbit` Module +---------------------------- + +.. automodule:: nova.fakerabbit + :members: + :undoc-members: + :show-inheritance: + +The :class:`FakeAOEDriver` Class +-------------------------------- + +.. autoclass:: nova.volume.driver.FakeAOEDriver + :members: + :undoc-members: + :show-inheritance: + +The :class:`service_unittest.FakeManager` Class +----------------------------------------------- + +.. autoclass:: nova.tests.service_unittest.FakeManager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`api.openstack.fakes` Module +------------------------------------------------ + +.. automodule:: nova.tests.api.openstack.fakes + :members: + :undoc-members: + :show-inheritance: + diff --git a/doc/source/devguide/glance.rst b/doc/source/devguide/glance.rst new file mode 100644 index 000000000..3d0f4ebec --- /dev/null +++ b/doc/source/devguide/glance.rst @@ -0,0 +1,27 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Glance Integration Programming Guide +==================================== + +The :mod:`image.service` Module +------------------------------- + +.. automodule:: nova.image.service + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/devguide/index.rst b/doc/source/devguide/index.rst new file mode 100644 index 000000000..6232a3181 --- /dev/null +++ b/doc/source/devguide/index.rst @@ -0,0 +1,60 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Nova Development Guide +====================== + +Nova is written in python. + +Prerequisites +------------- +.. toctree:: + :maxdepth: 1 + + ../concepts.and.introduction + architecture + development.environment + + +Contents +-------- + +.. toctree:: + :maxdepth: 1 + + services + database + volume + compute + network + auth + api + scheduler + fakes + nova + cloudpipe + objectstore + glance + + +Indices and tables +------------------ + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` + diff --git a/doc/source/devguide/network.rst b/doc/source/devguide/network.rst new file mode 100644 index 000000000..9426ece82 --- /dev/null +++ b/doc/source/devguide/network.rst @@ -0,0 +1,124 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Networking Programming Guide +============================ + +:: + + TODO(todd): * document hardware specific commands (maybe in admin guide?) + * document a map between flags and managers/backends + +The :mod:`Manager` module +------------------------- + +.. automodule:: nova.network.manager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`LinuxNet` driver +-------------------------- + +.. automodule:: nova.network.linux_net + :members: + :undoc-members: + :show-inheritance: + +Tests +----- + +The :mod:`network_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.network_unittest + :members: + :undoc-members: + :show-inheritance: + + +legacy docs +----------- + +The nova networking components manage private networks, public IP addressing, VPN connectivity, and firewall rules. + +Components +---------- +There are several key components: + +* NetworkController (Manages address and vlan allocation) +* RoutingNode (NATs public IPs to private IPs, and enforces firewall rules) +* AddressingNode (runs DHCP services for private networks) +* BridgingNode (a subclass of the basic nova ComputeNode) +* TunnelingNode (provides VPN connectivity) + +Component Diagram +----------------- + +Overview:: + + (PUBLIC INTERNET) + | \ + / \ / \ + [RoutingNode] ... [RN] [TunnelingNode] ... [TN] + | \ / | | + | < AMQP > | | + [AddressingNode]-- (VLAN) ... | (VLAN)... (VLAN) --- [AddressingNode] + \ | \ / + / \ / \ / \ / \ + [BridgingNode] ... [BridgingNode] + + + [NetworkController] ... [NetworkController] + \ / + < AMQP > + | + / \ + [CloudController]...[CloudController] + +While this diagram may not make this entirely clear, nodes and controllers communicate exclusively across the message bus (AMQP, currently). + +State Model +----------- +Network State consists of the following facts: + +* VLAN assignment (to a project) +* Private Subnet assignment (to a security group) in a VLAN +* Private IP assignments (to running instances) +* Public IP allocations (to a project) +* Public IP associations (to a private IP / running instance) + +While copies of this state exist in many places (expressed in IPTables rule chains, DHCP hosts files, etc), the controllers rely only on the distributed "fact engine" for state, queried over RPC (currently AMQP). The NetworkController inserts most records into this datastore (allocating addresses, etc) - however, individual nodes update state e.g. when running instances crash. + +The Public Traffic Path +----------------------- + +Public Traffic:: + + (PUBLIC INTERNET) + | + <-- [RoutingNode] + | + [AddressingNode] --> | + ( VLAN ) + | <-- [BridgingNode] + | + + +The RoutingNode is currently implemented using IPTables rules, which implement both NATing of public IP addresses, and the appropriate firewall chains. We are also looking at using Netomata / Clusto to manage NATting within a switch or router, and/or to manage firewall rules within a hardware firewall appliance. + +Similarly, the AddressingNode currently manages running DNSMasq instances for DHCP services. However, we could run an internal DHCP server (using Scapy ala Clusto), or even switch to static addressing by inserting the private address into the disk image the same way we insert the SSH keys. (See compute for more details). diff --git a/doc/source/devguide/nova.rst b/doc/source/devguide/nova.rst new file mode 100644 index 000000000..58125dc80 --- /dev/null +++ b/doc/source/devguide/nova.rst @@ -0,0 +1,190 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Nova Libraries Programming Guide +================================ + +The :mod:`adminclient` Module +----------------------------- + +.. automodule:: nova.adminclient + :members: + :undoc-members: + :show-inheritance: + +The :mod:`context` Module +------------------------- + +.. automodule:: nova.context + :members: + :undoc-members: + :show-inheritance: + +The :mod:`exception` Module +--------------------------- + +.. automodule:: nova.exception + :members: + :undoc-members: + :show-inheritance: + +The :mod:`flags` Module +----------------------- + +.. automodule:: nova.flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`process` Module +------------------------- + +.. automodule:: nova.process + :members: + :undoc-members: + :show-inheritance: + +The :mod:`rpc` Module +--------------------- + +.. automodule:: nova.rpc + :members: + :undoc-members: + :show-inheritance: + +The :mod:`server` Module +------------------------ + +.. automodule:: nova.server + :members: + :undoc-members: + :show-inheritance: + +The :mod:`test` Module +---------------------- + +.. automodule:: nova.test + :members: + :undoc-members: + :show-inheritance: + +The :mod:`twistd` Module +------------------------ + +.. automodule:: nova.twistd + :members: + :undoc-members: + :show-inheritance: + +The :mod:`utils` Module +----------------------- + +.. automodule:: nova.utils + :members: + :undoc-members: + :show-inheritance: + +The :mod:`validate` Module +-------------------------- + +.. automodule:: nova.validate + :members: + :undoc-members: + :show-inheritance: + +The :mod:`wsgi` Module +---------------------- + +.. automodule:: nova.wsgi + :members: + :undoc-members: + :show-inheritance: + +Tests +----- + +The :mod:`declare_flags` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.declare_flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`fake_flags` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.fake_flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`flags_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.flags_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`process_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.process_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`real_flags` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.real_flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`rpc_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.rpc_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`runtime_flags` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.runtime_flags + :members: + :undoc-members: + :show-inheritance: + +The :mod:`twistd_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.twistd_unittest + :members: + :undoc-members: + :show-inheritance: + +The :mod:`validator_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.validator_unittest + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/devguide/objectstore.rst b/doc/source/devguide/objectstore.rst new file mode 100644 index 000000000..4087b5dd0 --- /dev/null +++ b/doc/source/devguide/objectstore.rst @@ -0,0 +1,62 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Objectstore Programming Guide +============================= + +The :mod:`handler` Module +------------------------- + +.. automodule:: nova.objectstore.handler + :members: + :undoc-members: + :show-inheritance: + +The :mod:`bucket` Module +------------------------ + +.. automodule:: nova.objectstore.bucket + :members: + :undoc-members: + :show-inheritance: + +The :mod:`stored` Module +------------------------ + +.. automodule:: nova.objectstore.stored + :members: + :undoc-members: + :show-inheritance: + +The :mod:`image` Module +----------------------- + +.. automodule:: nova.objectstore.image + :members: + :undoc-members: + :show-inheritance: + +Tests +----- + +The :mod:`objectstore_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.objectstore_unittest + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/devguide/scheduler.rst b/doc/source/devguide/scheduler.rst new file mode 100644 index 000000000..df820c2f2 --- /dev/null +++ b/doc/source/devguide/scheduler.rst @@ -0,0 +1,62 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Scheduler Programming Guide +=========================== + +The :mod:`manager` Module +------------------------- + +.. automodule:: nova.scheduler.manager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`driver` Module +------------------------ + +.. automodule:: nova.scheduler.driver + :members: + :undoc-members: + :show-inheritance: + +The :mod:`chance` Module +------------------------ + +.. automodule:: nova.scheduler.chance + :members: + :undoc-members: + :show-inheritance: + +The :mod:`simple` Module +------------------------ + +.. automodule:: nova.scheduler.simple + :members: + :undoc-members: + :show-inheritance: + +Tests +----- + +The :mod:`scheduler_unittest` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.tests.scheduler_unittest + :members: + :undoc-members: + :show-inheritance: diff --git a/doc/source/devguide/services.rst b/doc/source/devguide/services.rst new file mode 100644 index 000000000..517a51901 --- /dev/null +++ b/doc/source/devguide/services.rst @@ -0,0 +1,72 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _service_manager_driver: + +Services Managers and Drivers +============================= + +The responsibilities of Services, Managers, and Drivers, can be a bit confusing to people that are new to nova. This document attempts to outline the division of responsibilities to make understanding the system a little bit easier. + +Currently, Managers and Drivers are specified by flags and loaded using utils.load_object(). This method allows for them to be implemented as singletons, classes, modules or objects. As long as the path specified by the flag leads to an object (or a callable that returns an object) that responds to getattr, it should work as a manager or driver. + +Service +------- + +A service is a very thin wrapper around a Manager object. It exposes the manager's public methods to other components of the system via rpc. It will report state periodically to the database and is responsible for initiating any periodic tasts that need to be executed on a given host. + +The :mod:`service` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.service + :members: + :undoc-members: + :show-inheritance: + +Manager +------- + +Managers are responsible for a certain aspect of the sytem. It is a logical grouping of code relating to a portion of the system. In general other components should be using the manager to make changes to the components that it is responsible for. + +For example, other components that need to deal with volumes in some way, should do so by calling methods on the VolumeManager instead of directly changing fields in the database. This allows us to keep all of the code relating to volumes in the same place. + +We have adopted a basic strategy of Smart managers and dumb data, which means rather than attaching methods to data objects, components should call manager methods that act on the data. + +Methods on managers that can be executed locally should be called directly. If a particular method must execute on a remote host, this should be done via rpc to the service that wraps the manager + +Managers should be responsible for most of the db access, and non-implementation specific data. Anything implementation specific that can't be generalized should be done by the Driver. + +In general, we prefer to have one manager with multiple drivers for different implementations, but sometimes it makes sense to have multiple managers. You can think of it this way: Abstract different overall strategies at the manager level(FlatNetwork vs VlanNetwork), and different implementations at the driver level(LinuxNetDriver vs CiscoNetDriver). + +Managers will often provide methods for initial setup of a host or periodic tasksto a wrapping service. + +The :mod:`manager` Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: nova.manager + :members: + :undoc-members: + :show-inheritance: + +Driver +------ + +A manager will generally load a driver for some of its tasks. The driver is responsible for specific implementation details. Anything running shell commands on a host, or dealing with other non-python code should probably be happening in a driver. + +Drivers should minimize touching the database, although it is currently acceptable for implementation specific data. This may be reconsidered at some point. + +It usually makes sense to define an Abstract Base Class for the specific driver (i.e. VolumeDriver), to define the methods that a different driver would need to implement. diff --git a/doc/source/devguide/volume.rst b/doc/source/devguide/volume.rst new file mode 100644 index 000000000..39b33a500 --- /dev/null +++ b/doc/source/devguide/volume.rst @@ -0,0 +1,62 @@ +.. + Copyright 2010 United States Government as represented by the + Administrator of the National Aeronautics and Space Administration. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Volume Programming Guide +========================= + +:: + + TODO(todd): rework after iSCSI merge (see 'Old Docs') + + +The :mod:`nova.volume.manager` Module +------------------------------------- + +.. automodule:: nova.volume.manager + :members: + :undoc-members: + :show-inheritance: + +The :mod:`nova.volume.driver` Module +------------------------------------- + +.. automodule:: nova.volume.driver + :members: + :undoc-members: + :show-inheritance: + :exclude-members: FakeAOEDriver + +Tests +----- + +.. automodule:: nova.tests.volume_unittest + :members: + :undoc-members: + :show-inheritance: + +Old Docs +-------- + +Nova uses ata-over-ethernet (AoE) to export storage volumes from multiple storage nodes. These AoE exports are attached (using libvirt) directly to running instances. + +Nova volumes are exported over the primary system VLAN (usually VLAN 1), and not over individual VLANs. + +AoE exports are numbered according to a "shelf and blade" syntax. In order to avoid collisions, we currently perform an AoE-discover of existing exports, and then grab the next unused number. (This obviously has race condition problems, and should be replaced by allocating a shelf-id to each storage node.) + +The underlying volumes are LVM logical volumes, created on demand within a single large volume group. + + diff --git a/doc/source/fakes.rst b/doc/source/fakes.rst deleted file mode 100644 index 5988e2ab8..000000000 --- a/doc/source/fakes.rst +++ /dev/null @@ -1,74 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Fakes Programming Guide -======================= - -:: - - TODO(todd): * document general info about fakes - - -The :mod:`virt.fake` Module ---------------------------- - -.. automodule:: nova.virt.fake - :members: - :undoc-members: - :show-inheritance: - :noindex: - -The :mod:`fakeldap` Module --------------------------- - -.. automodule:: nova.auth.fakeldap - :members: - :undoc-members: - :show-inheritance: - -The :mod:`fakerabbit` Module ----------------------------- - -.. automodule:: nova.fakerabbit - :members: - :undoc-members: - :show-inheritance: - -The :class:`FakeAOEDriver` Class --------------------------------- - -.. autoclass:: nova.volume.driver.FakeAOEDriver - :members: - :undoc-members: - :show-inheritance: - -The :class:`service_unittest.FakeManager` Class ------------------------------------------------ - -.. autoclass:: nova.tests.service_unittest.FakeManager - :members: - :undoc-members: - :show-inheritance: - -The :mod:`api.openstack.fakes` Module ------------------------------------------------- - -.. automodule:: nova.tests.api.openstack.fakes - :members: - :undoc-members: - :show-inheritance: - diff --git a/doc/source/flags.rst b/doc/source/flags.rst deleted file mode 100644 index 4c950aa88..000000000 --- a/doc/source/flags.rst +++ /dev/null @@ -1,23 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Flags and Flagfiles -=================== - -* python-gflags -* flagfiles -* list of flags by component (see concepts list) diff --git a/doc/source/getting.started.rst b/doc/source/getting.started.rst deleted file mode 100644 index 79d2cf204..000000000 --- a/doc/source/getting.started.rst +++ /dev/null @@ -1,167 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Getting Started with Nova -========================= - -This code base is continually changing, so dependencies also change. If you -encounter any problems, see the :doc:`reaching.out` page. -The `contrib/nova.sh` script should be kept up to date, and may be a good -resource to review when debugging. - -The purpose of this document is to get a system installed that you can use to -test your setup assumptions. Working from this base installtion you can -tweak configurations and work with different flags to monitor interaction with -your hardware, network, and other factors that will allow you to determine -suitability for your deployment. After following this setup method, you should -be able to experiment with different managers, drivers, and flags to get the -best performance. - -Dependencies ------------- - -Related servers we rely on - -* **RabbitMQ**: messaging queue, used for all communication between components - -Optional servers - -* **OpenLDAP**: By default, the auth server uses the RDBMS-backed datastore by - setting FLAGS.auth_driver to `nova.auth.dbdriver.DbDriver`. But OpenLDAP - (or LDAP) could be configured by specifying `nova.auth.ldapdriver.LdapDriver`. - There is a script in the sources (`nova/auth/slap.sh`) to install a very basic - openldap server on ubuntu. -* **ReDIS**: There is a fake ldap auth driver - `nova.auth.ldapdriver.FakeLdapDriver` that backends to redis. This was - created for testing ldap implementation on systems that don't have an easy - means to install ldap. -* **MySQL**: Either MySQL or another database supported by sqlalchemy needs to - be avilable. Currently, only sqlite3 an mysql have been tested. - -Python libraries that we use (from pip-requires): - -.. literalinclude:: ../../tools/pip-requires - -Other libraries: - -* **XenAPI**: Needed only for Xen Cloud Platform or XenServer support. Available - from http://wiki.xensource.com/xenwiki/XCP_SDK or - http://community.citrix.com/cdn/xs/sdks. - -External unix tools that are required: - -* iptables -* ebtables -* gawk -* curl -* kvm -* libvirt -* dnsmasq -* vlan -* open-iscsi and iscsitarget (if you use iscsi volumes) -* aoetools and vblade-persist (if you use aoe-volumes) - -Nova uses cutting-edge versions of many packages. There are ubuntu packages in -the nova-core ppa. You can use add this ppa to your sources list on an ubuntu -machine with the following commands:: - - sudo apt-get install -y python-software-properties - sudo add-apt-repository ppa:nova-core/ppa - -Recommended ------------ - -* euca2ools: python implementation of aws ec2-tools and ami tools -* build tornado to use C module for evented section - - -Installation --------------- - -You can install from packages for your particular Linux distribution if they are -available. Otherwise you can install from source by checking out the source -files from the `Nova Source Code Repository `_ -and running:: - - python setup.py install - -Configuration ---------------- - -Configuring the host system -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -As you read through the Administration Guide you will notice configuration hints -inline with documentation on the subsystem you are configuring. Presented in -this "Getting Started with Nova" document, we only provide what you need to -get started as quickly as possible. For a more detailed description of system -configuration, start reading through :doc:`multi.node.install`. - -* Create a volume group (you can use an actual disk for the volume group as - well):: - - # This creates a 1GB file to create volumes out of - dd if=/dev/zero of=MY_FILE_PATH bs=100M count=10 - losetup --show -f MY_FILE_PATH - # replace /dev/loop0 below with whatever losetup returns - # nova-volumes is the default for the --volume_group flag - vgcreate nova-volumes /dev/loop0 - - -Configuring Nova -~~~~~~~~~~~~~~~~ - -Configuration of the entire system is performed through python-gflags. The -best way to track configuration is through the use of a flagfile. - -A flagfile is specified with the ``--flagfile=FILEPATH`` argument to the binary -when you launch it. Flagfiles for nova are typically stored in -``/etc/nova/nova.conf``, and flags specific to a certain program are stored in -``/etc/nova/nova-COMMAND.conf``. Each configuration file can include another -flagfile, so typically a file like ``nova-manage.conf`` would have as its first -line ``--flagfile=/etc/nova/nova.conf`` to load the common flags before -specifying overrides or additional options. - -A sample configuration to test the system follows:: - - --verbose - --nodaemon - --FAKE_subdomain=ec2 - --auth_driver=nova.auth.dbdriver.DbDriver - -Running ---------- - -There are many parts to the nova system, each with a specific function. They -are built to be highly-available, so there are may configurations they can be -run in (ie: on many machines, many listeners per machine, etc). This part -of the guide only gets you started quickly, to learn about HA options, see -:doc:`multi.node.install`. - -Launch supporting services - -* rabbitmq -* redis (optional) -* mysql (optional) - -Launch nova components, each should have ``--flagfile=/etc/nova/nova.conf`` - -* nova-api -* nova-compute -* nova-objectstore -* nova-volume -* nova-scheduler diff --git a/doc/source/glance.rst b/doc/source/glance.rst deleted file mode 100644 index 3d0f4ebec..000000000 --- a/doc/source/glance.rst +++ /dev/null @@ -1,27 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Glance Integration Programming Guide -==================================== - -The :mod:`image.service` Module -------------------------------- - -.. automodule:: nova.image.service - :members: - :undoc-members: - :show-inheritance: diff --git a/doc/source/index.rst b/doc/source/index.rst index 9a62570ed..0dcc1945e 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -43,8 +43,8 @@ Contents :maxdepth: 1 concepts.and.introduction - administration.guide - development.guide + adminguide/index + devguide/index reaching.out diff --git a/doc/source/modules.rst b/doc/source/modules.rst deleted file mode 100644 index 9b31664b6..000000000 --- a/doc/source/modules.rst +++ /dev/null @@ -1,38 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Nova Documentation -================== - -This page contains the Nova Modules documentation. - -Modules: --------- - -.. toctree:: - :maxdepth: 4 - - auth - compute - fakes - nova - volume - -Removed: --------- - -* endpoint diff --git a/doc/source/monitoring.rst b/doc/source/monitoring.rst deleted file mode 100644 index e7766a6e7..000000000 --- a/doc/source/monitoring.rst +++ /dev/null @@ -1,27 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Monitoring -========== - -* components -* throughput -* exceptions -* hardware - -* ganglia -* syslog diff --git a/doc/source/multi.node.install.rst b/doc/source/multi.node.install.rst deleted file mode 100644 index 1d484e779..000000000 --- a/doc/source/multi.node.install.rst +++ /dev/null @@ -1,57 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Multi-Node Nova -=============== - -When you move beyond evaluating the technology and into building an actual -production environemnt, you will need to know how to configure your datacenter -and how to deploy components across your clusters. This guide should help you -through that process. - -Bare-metal Provisioning ------------------------ - -To install the base operating system you can use PXE booting. - -Deployment Technologies ------------------------ - -Once you have machines with a base operating system installation, you can deploy -code and configuration with your favorite tools: - -* Puppet -* Chef - -Types of Hosts --------------- - -A single machine in your cluster can act as one or more of the following types -of host: - -Nova Services - -* Network -* Compute -* Volume -* API -* Objectstore - -Other supporting services - -* Datastore -* Message Queue diff --git a/doc/source/network.rst b/doc/source/network.rst deleted file mode 100644 index 9426ece82..000000000 --- a/doc/source/network.rst +++ /dev/null @@ -1,124 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Networking Programming Guide -============================ - -:: - - TODO(todd): * document hardware specific commands (maybe in admin guide?) - * document a map between flags and managers/backends - -The :mod:`Manager` module -------------------------- - -.. automodule:: nova.network.manager - :members: - :undoc-members: - :show-inheritance: - -The :mod:`LinuxNet` driver --------------------------- - -.. automodule:: nova.network.linux_net - :members: - :undoc-members: - :show-inheritance: - -Tests ------ - -The :mod:`network_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.network_unittest - :members: - :undoc-members: - :show-inheritance: - - -legacy docs ------------ - -The nova networking components manage private networks, public IP addressing, VPN connectivity, and firewall rules. - -Components ----------- -There are several key components: - -* NetworkController (Manages address and vlan allocation) -* RoutingNode (NATs public IPs to private IPs, and enforces firewall rules) -* AddressingNode (runs DHCP services for private networks) -* BridgingNode (a subclass of the basic nova ComputeNode) -* TunnelingNode (provides VPN connectivity) - -Component Diagram ------------------ - -Overview:: - - (PUBLIC INTERNET) - | \ - / \ / \ - [RoutingNode] ... [RN] [TunnelingNode] ... [TN] - | \ / | | - | < AMQP > | | - [AddressingNode]-- (VLAN) ... | (VLAN)... (VLAN) --- [AddressingNode] - \ | \ / - / \ / \ / \ / \ - [BridgingNode] ... [BridgingNode] - - - [NetworkController] ... [NetworkController] - \ / - < AMQP > - | - / \ - [CloudController]...[CloudController] - -While this diagram may not make this entirely clear, nodes and controllers communicate exclusively across the message bus (AMQP, currently). - -State Model ------------ -Network State consists of the following facts: - -* VLAN assignment (to a project) -* Private Subnet assignment (to a security group) in a VLAN -* Private IP assignments (to running instances) -* Public IP allocations (to a project) -* Public IP associations (to a private IP / running instance) - -While copies of this state exist in many places (expressed in IPTables rule chains, DHCP hosts files, etc), the controllers rely only on the distributed "fact engine" for state, queried over RPC (currently AMQP). The NetworkController inserts most records into this datastore (allocating addresses, etc) - however, individual nodes update state e.g. when running instances crash. - -The Public Traffic Path ------------------------ - -Public Traffic:: - - (PUBLIC INTERNET) - | - <-- [RoutingNode] - | - [AddressingNode] --> | - ( VLAN ) - | <-- [BridgingNode] - | - - -The RoutingNode is currently implemented using IPTables rules, which implement both NATing of public IP addresses, and the appropriate firewall chains. We are also looking at using Netomata / Clusto to manage NATting within a switch or router, and/or to manage firewall rules within a hardware firewall appliance. - -Similarly, the AddressingNode currently manages running DNSMasq instances for DHCP services. However, we could run an internal DHCP server (using Scapy ala Clusto), or even switch to static addressing by inserting the private address into the disk image the same way we insert the SSH keys. (See compute for more details). diff --git a/doc/source/nova.manage.rst b/doc/source/nova.manage.rst deleted file mode 100644 index eeb463f52..000000000 --- a/doc/source/nova.manage.rst +++ /dev/null @@ -1,200 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - - -nova-manage -=========== - -Introduction -~~~~~~~~~~~~ - -The nova-manage command is used to perform many essential functions for -administration and ongoing maintenance of nova, such as user creation, -vpn management, and much more. - -The standard pattern for executing a nova-manage command is: - -``nova-manage []`` - -For example, to obtain a list of all projects: - -``nova-manage project list`` - -You can run without arguments to see a list of available command categories: - -``nova-manage`` - -You can run with a category argument to see a list of all commands in that -category: - -``nova-manage user`` - -User Maintenance -~~~~~~~~~~~~~~~~ - -Users, including admins, are created through the ``user`` commands. - -* user admin: creates a new admin and prints exports - * arguments: name [access] [secret] -* user create: creates a new user and prints exports - * arguments: name [access] [secret] -* user delete: deletes an existing user - * arguments: name -* user exports: prints access and secrets for user in export format - * arguments: name -* user list: lists all users - * arguments: none -* user modify: update a users keys & admin flag - * arguments: accesskey secretkey admin - * leave any field blank to ignore it, admin should be 'T', 'F', or blank - -Project Maintenance -~~~~~~~~~~~~~~~~~~~ - -* project add: Adds user to project - * arguments: project user -* project create: Creates a new project - * arguments: name project_manager [description] -* project delete: Deletes an existing project - * arguments: project_id -* project environment: Exports environment variables to an sourcable file - * arguments: project_id user_id [filename='novarc] -* project list: lists all projects - * arguments: none -* project quota: Set or display quotas for project - * arguments: project_id [key] [value] -* project remove: Removes user from project - * arguments: project user -* project scrub: Deletes data associated with project - * arguments: project -* project zipfile: Exports credentials for project to a zip file - * arguments: project_id user_id [filename='nova.zip] - -User Role Management -~~~~~~~~~~~~~~~~~~~~ - -* role add: adds role to user - * if project is specified, adds project specific role - * arguments: user, role [project] -* role has: checks to see if user has role - * if project is specified, returns True if user has - the global role and the project role - * arguments: user, role [project] -* role remove: removes role from user - * if project is specified, removes project specific role - * arguments: user, role [project] - - -Nova Shell -~~~~~~~~~~ - -* shell bpython - * start a new bpython shell -* shell ipython - * start a new ipython shell -* shell python - * start a new python shell -* shell run - * ??? -* shell script: Runs the script from the specifed path with flags set properly. - * arguments: path - -VPN Management -~~~~~~~~~~~~~~ - -* vpn list: Print a listing of the VPNs for all projects. - * arguments: none -* vpn run: Start the VPN for a given project. - * arguments: project -* vpn spawn: Run all VPNs. - * arguments: none - - -Floating IP Management -~~~~~~~~~~~~~~~~~~~~~~ - -* floating create: Creates floating ips for host by range - * arguments: host ip_range -* floating delete: Deletes floating ips by range - * arguments: range -* floating list: Prints a listing of all floating ips - * arguments: none - -Network Management -~~~~~~~~~~~~~~~~~~ - -* network create: Creates fixed ips for host by range - * arguments: [fixed_range=FLAG], [num_networks=FLAG], - [network_size=FLAG], [vlan_start=FLAG], - [vpn_start=FLAG] - - -Concept: Flags --------------- - -python-gflags - - -Concept: Plugins ----------------- - -* Managers/Drivers: utils.import_object from string flag -* virt/connections: conditional loading from string flag -* db: LazyPluggable via string flag -* auth_manager: utils.import_class based on string flag -* Volumes: moving to pluggable driver instead of manager -* Network: pluggable managers -* Compute: same driver used, but pluggable at connection - - -Concept: IPC/RPC ----------------- - -Rabbit! - - -Concept: Fakes --------------- - -* auth -* ldap - - -Concept: Scheduler ------------------- - -* simple -* random - - -Concept: Security Groups ------------------------- - -Security groups - - -Concept: Certificate Authority ------------------------------- - -Nova does a small amount of certificate management. These certificates are used for :ref:`project vpns ` and decrypting bundled images. - - -Concept: Images ---------------- - -* launching -* bundling diff --git a/doc/source/nova.rst b/doc/source/nova.rst deleted file mode 100644 index 58125dc80..000000000 --- a/doc/source/nova.rst +++ /dev/null @@ -1,190 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Nova Libraries Programming Guide -================================ - -The :mod:`adminclient` Module ------------------------------ - -.. automodule:: nova.adminclient - :members: - :undoc-members: - :show-inheritance: - -The :mod:`context` Module -------------------------- - -.. automodule:: nova.context - :members: - :undoc-members: - :show-inheritance: - -The :mod:`exception` Module ---------------------------- - -.. automodule:: nova.exception - :members: - :undoc-members: - :show-inheritance: - -The :mod:`flags` Module ------------------------ - -.. automodule:: nova.flags - :members: - :undoc-members: - :show-inheritance: - -The :mod:`process` Module -------------------------- - -.. automodule:: nova.process - :members: - :undoc-members: - :show-inheritance: - -The :mod:`rpc` Module ---------------------- - -.. automodule:: nova.rpc - :members: - :undoc-members: - :show-inheritance: - -The :mod:`server` Module ------------------------- - -.. automodule:: nova.server - :members: - :undoc-members: - :show-inheritance: - -The :mod:`test` Module ----------------------- - -.. automodule:: nova.test - :members: - :undoc-members: - :show-inheritance: - -The :mod:`twistd` Module ------------------------- - -.. automodule:: nova.twistd - :members: - :undoc-members: - :show-inheritance: - -The :mod:`utils` Module ------------------------ - -.. automodule:: nova.utils - :members: - :undoc-members: - :show-inheritance: - -The :mod:`validate` Module --------------------------- - -.. automodule:: nova.validate - :members: - :undoc-members: - :show-inheritance: - -The :mod:`wsgi` Module ----------------------- - -.. automodule:: nova.wsgi - :members: - :undoc-members: - :show-inheritance: - -Tests ------ - -The :mod:`declare_flags` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.declare_flags - :members: - :undoc-members: - :show-inheritance: - -The :mod:`fake_flags` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.fake_flags - :members: - :undoc-members: - :show-inheritance: - -The :mod:`flags_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.flags_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`process_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.process_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`real_flags` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.real_flags - :members: - :undoc-members: - :show-inheritance: - -The :mod:`rpc_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.rpc_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`runtime_flags` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.runtime_flags - :members: - :undoc-members: - :show-inheritance: - -The :mod:`twistd_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.twistd_unittest - :members: - :undoc-members: - :show-inheritance: - -The :mod:`validator_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.validator_unittest - :members: - :undoc-members: - :show-inheritance: diff --git a/doc/source/objectstore.rst b/doc/source/objectstore.rst deleted file mode 100644 index 4087b5dd0..000000000 --- a/doc/source/objectstore.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Objectstore Programming Guide -============================= - -The :mod:`handler` Module -------------------------- - -.. automodule:: nova.objectstore.handler - :members: - :undoc-members: - :show-inheritance: - -The :mod:`bucket` Module ------------------------- - -.. automodule:: nova.objectstore.bucket - :members: - :undoc-members: - :show-inheritance: - -The :mod:`stored` Module ------------------------- - -.. automodule:: nova.objectstore.stored - :members: - :undoc-members: - :show-inheritance: - -The :mod:`image` Module ------------------------ - -.. automodule:: nova.objectstore.image - :members: - :undoc-members: - :show-inheritance: - -Tests ------ - -The :mod:`objectstore_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.objectstore_unittest - :members: - :undoc-members: - :show-inheritance: diff --git a/doc/source/quickstart.rst b/doc/source/quickstart.rst deleted file mode 100644 index 511ed2483..000000000 --- a/doc/source/quickstart.rst +++ /dev/null @@ -1,104 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Nova Quickstart -=============== - -:: - - TODO(todd): * Document the assumptions about pluggable interfaces - (sqlite3 instead of mysql, etc) - * Document env vars that can change things - (USE_MYSQL, HOST_IP) - - -The `contrib/novascript.sh` file in the source distribution is a script that -will quickly set up nova to run on a single machine. It is tested against -Ubuntu only, but other distributions are forthcoming. - -Usage ------ - -Unless you want to spend a lot of time fiddling with permissions and sudoers, -you should probably run nova as root. - -:: - - sudo -i - -If you are concerned about security, nova runs just fine inside a virtual -machine. - -Use the script to install and run the current trunk. You can also specify a -specific branch by putting `lp:~someone/nova/some-branch` after the branch -command - -:: - - ./nova.sh branch - ./nova.sh install - ./nova.sh run - -The run command will drop you into a screen session with all of the workers -running in different windows You can use eucatools to run commands against the -cloud. - -:: - - euca-add-keypair test > test.pem - euca-run-instances -k test -t m1.tiny ami-tiny - euca-describe-instances - -To see output from the various workers, switch screen windows - -:: - - " - -will give you a list of running windows. - -When the instance is running, you should be able to ssh to it. - -:: - - chmod 600 test.pem - ssh -i test.pem root@10.0.0.3 - -When you exit screen - -:: - - - -nova will terminate. It may take a while for nova to finish cleaning up. If -you exit the process before it is done because there were some problems in your -build, you may have to clean up the nova processes manually. If you had any -instances running, you can attempt to kill them through the api: - -:: - - ./nova.sh terminate - -Then you can destroy the screen: - -:: - - ./nova.sh clean - -If things get particularly messed up, you might need to do some more intense -cleanup. Be careful, the following command will manually destroy all runnning -virsh instances and attempt to delete all vlans and bridges. diff --git a/doc/source/scheduler.rst b/doc/source/scheduler.rst deleted file mode 100644 index df820c2f2..000000000 --- a/doc/source/scheduler.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Scheduler Programming Guide -=========================== - -The :mod:`manager` Module -------------------------- - -.. automodule:: nova.scheduler.manager - :members: - :undoc-members: - :show-inheritance: - -The :mod:`driver` Module ------------------------- - -.. automodule:: nova.scheduler.driver - :members: - :undoc-members: - :show-inheritance: - -The :mod:`chance` Module ------------------------- - -.. automodule:: nova.scheduler.chance - :members: - :undoc-members: - :show-inheritance: - -The :mod:`simple` Module ------------------------- - -.. automodule:: nova.scheduler.simple - :members: - :undoc-members: - :show-inheritance: - -Tests ------ - -The :mod:`scheduler_unittest` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.tests.scheduler_unittest - :members: - :undoc-members: - :show-inheritance: diff --git a/doc/source/services.rst b/doc/source/services.rst deleted file mode 100644 index 517a51901..000000000 --- a/doc/source/services.rst +++ /dev/null @@ -1,72 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _service_manager_driver: - -Services Managers and Drivers -============================= - -The responsibilities of Services, Managers, and Drivers, can be a bit confusing to people that are new to nova. This document attempts to outline the division of responsibilities to make understanding the system a little bit easier. - -Currently, Managers and Drivers are specified by flags and loaded using utils.load_object(). This method allows for them to be implemented as singletons, classes, modules or objects. As long as the path specified by the flag leads to an object (or a callable that returns an object) that responds to getattr, it should work as a manager or driver. - -Service -------- - -A service is a very thin wrapper around a Manager object. It exposes the manager's public methods to other components of the system via rpc. It will report state periodically to the database and is responsible for initiating any periodic tasts that need to be executed on a given host. - -The :mod:`service` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.service - :members: - :undoc-members: - :show-inheritance: - -Manager -------- - -Managers are responsible for a certain aspect of the sytem. It is a logical grouping of code relating to a portion of the system. In general other components should be using the manager to make changes to the components that it is responsible for. - -For example, other components that need to deal with volumes in some way, should do so by calling methods on the VolumeManager instead of directly changing fields in the database. This allows us to keep all of the code relating to volumes in the same place. - -We have adopted a basic strategy of Smart managers and dumb data, which means rather than attaching methods to data objects, components should call manager methods that act on the data. - -Methods on managers that can be executed locally should be called directly. If a particular method must execute on a remote host, this should be done via rpc to the service that wraps the manager - -Managers should be responsible for most of the db access, and non-implementation specific data. Anything implementation specific that can't be generalized should be done by the Driver. - -In general, we prefer to have one manager with multiple drivers for different implementations, but sometimes it makes sense to have multiple managers. You can think of it this way: Abstract different overall strategies at the manager level(FlatNetwork vs VlanNetwork), and different implementations at the driver level(LinuxNetDriver vs CiscoNetDriver). - -Managers will often provide methods for initial setup of a host or periodic tasksto a wrapping service. - -The :mod:`manager` Module -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: nova.manager - :members: - :undoc-members: - :show-inheritance: - -Driver ------- - -A manager will generally load a driver for some of its tasks. The driver is responsible for specific implementation details. Anything running shell commands on a host, or dealing with other non-python code should probably be happening in a driver. - -Drivers should minimize touching the database, although it is currently acceptable for implementation specific data. This may be reconsidered at some point. - -It usually makes sense to define an Abstract Base Class for the specific driver (i.e. VolumeDriver), to define the methods that a different driver would need to implement. diff --git a/doc/source/volume.rst b/doc/source/volume.rst deleted file mode 100644 index 39b33a500..000000000 --- a/doc/source/volume.rst +++ /dev/null @@ -1,62 +0,0 @@ -.. - Copyright 2010 United States Government as represented by the - Administrator of the National Aeronautics and Space Administration. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -Volume Programming Guide -========================= - -:: - - TODO(todd): rework after iSCSI merge (see 'Old Docs') - - -The :mod:`nova.volume.manager` Module -------------------------------------- - -.. automodule:: nova.volume.manager - :members: - :undoc-members: - :show-inheritance: - -The :mod:`nova.volume.driver` Module -------------------------------------- - -.. automodule:: nova.volume.driver - :members: - :undoc-members: - :show-inheritance: - :exclude-members: FakeAOEDriver - -Tests ------ - -.. automodule:: nova.tests.volume_unittest - :members: - :undoc-members: - :show-inheritance: - -Old Docs --------- - -Nova uses ata-over-ethernet (AoE) to export storage volumes from multiple storage nodes. These AoE exports are attached (using libvirt) directly to running instances. - -Nova volumes are exported over the primary system VLAN (usually VLAN 1), and not over individual VLANs. - -AoE exports are numbered according to a "shelf and blade" syntax. In order to avoid collisions, we currently perform an AoE-discover of existing exports, and then grab the next unused number. (This obviously has race condition problems, and should be replaced by allocating a shelf-id to each storage node.) - -The underlying volumes are LVM logical volumes, created on demand within a single large volume group. - - -- cgit From 990f4451e7779654188afa514baef270182f8697 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 5 Nov 2010 14:09:36 -0400 Subject: update types of services that may run on machines. --- doc/source/adminguide/getting.started.rst | 1 + doc/source/adminguide/multi.node.install.rst | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/adminguide/getting.started.rst b/doc/source/adminguide/getting.started.rst index 65ba2d57e..1a5e913ce 100644 --- a/doc/source/adminguide/getting.started.rst +++ b/doc/source/adminguide/getting.started.rst @@ -157,6 +157,7 @@ Launch supporting services * rabbitmq * redis (optional) * mysql (optional) +* openldap (optional) Launch nova components, each should have ``--flagfile=/etc/nova/nova.conf`` diff --git a/doc/source/adminguide/multi.node.install.rst b/doc/source/adminguide/multi.node.install.rst index 1d484e779..5ea0b6038 100644 --- a/doc/source/adminguide/multi.node.install.rst +++ b/doc/source/adminguide/multi.node.install.rst @@ -53,5 +53,6 @@ Nova Services Other supporting services -* Datastore * Message Queue +* Database (optional) +* Authentication database (optional) -- cgit From f9bd8f41807a27d62b538707b9aaff44746d8933 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 5 Nov 2010 14:30:46 -0400 Subject: Change order of secions so puppeting is last, add more initial setup tasks. --- doc/source/adminguide/multi.node.install.rst | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) (limited to 'doc/source') diff --git a/doc/source/adminguide/multi.node.install.rst b/doc/source/adminguide/multi.node.install.rst index 5ea0b6038..eaf8dd966 100644 --- a/doc/source/adminguide/multi.node.install.rst +++ b/doc/source/adminguide/multi.node.install.rst @@ -28,15 +28,6 @@ Bare-metal Provisioning To install the base operating system you can use PXE booting. -Deployment Technologies ------------------------ - -Once you have machines with a base operating system installation, you can deploy -code and configuration with your favorite tools: - -* Puppet -* Chef - Types of Hosts -------------- @@ -56,3 +47,19 @@ Other supporting services * Message Queue * Database (optional) * Authentication database (optional) + +Initial Setup +------------- + +* Networking +* Cloudadmin User Creation + +Deployment Technologies +----------------------- + +Once you have machines with a base operating system installation, you can deploy +code and configuration with your favorite tools to specify which machines in +your cluster have which roles: + +* Puppet +* Chef -- cgit From 0a4785be35bf1fc66e41b7565edf5e3936b70673 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 5 Nov 2010 11:56:12 -0700 Subject: additions to home page --- doc/source/index.rst | 110 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) (limited to 'doc/source') diff --git a/doc/source/index.rst b/doc/source/index.rst index 0dcc1945e..5dec404ef 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -27,6 +27,7 @@ Nova is written with the following design guidelines in mind: * **Fault-Tollerant**: Isloated processes avoid cascading failures * **Recoverable**: Failures should be easy to diagnose, debug, and rectify * **Open Standards**: Be a reference implementation for a community-driven api +* **API Compatibility**: Nova strives to provide API-compatible with popular systems like Amazon EC2 This documentation is generated by the Sphinx toolkit and lives in the source tree. Additional documentation on Nova and other components of OpenStack can @@ -47,6 +48,115 @@ Contents devguide/index reaching.out +Recommended System Configuration +================================ + +Although Nova can be run on a variety of system architectures, for most users the following will be simplest: + +* Ubuntu Lucid +* 10GB Hard Disk Space +* 512MB RAM + +For development, Nova can run from within a VM. + +Quickstart +========== + +To make getting started with Nova easier, we provide a setup script that makes it easy to download and run the most recent version of Nova on your local machine. + +* sudo -i # become root +* cd # go to home directory +* git clone git://github.com/vishvananda/novascript.git +* cd novascript + +Inside this directory, you will find nova.sh, which is a utility designed to assist with getting your development environment up and running. + +Use nova.sh to install and run the current trunk. You can also specify a specific branch by putting lp:~someone/nova/some-branch after the branch command + +* ./nova.sh branch +* ./nova.sh install +* ./nova.sh run + +The run command will drop you into a screen session with all of the workers running in different windows You can use eucatools to run commands against the cloud. + +* euca-add-keypair test > test.pem +* euca-run-instances -k test -t m1.tiny ami-tiny +* euca-describe-instances + +To see output from the various workers, switch screen windows + +* " + +will give you a list of running windows. + +When the instance is running, you should be able to ssh to it. + +* chmod 600 test.pem +* ssh -i test.pem root@10.0.0.3 + +When you exit screen + +* + +nova will terminate. It may take a while for nova to finish cleaning up. If you exit the process before it is done because there were some problems in your build, you may have to clean up the nova processes manually. If you had any instances running, you can attempt to kill them through the api: + +* ./nova.sh terminate + +Then you can destroy the screen: + +* ./nova.sh clean + +If things get particularly messed up, you might need to do some more intense cleanup. Be careful, the following command will manually destroy all runnning virsh instances and attempt to delete all vlans and bridges. + +* ./nova.sh scrub + +You can edit files in the install directory or do a bzr pull to pick up new versions. You only need to do + +* ./nova.sh run + +to run nova after the first install. The database should be cleaned up on each run. + +Notes +----- + +The script starts nova-volume in fake mode, so it will not create any actual volumes. + +if you want to USE_VENV because you have different versions of python packages on your system that you want to keep, you should run install before branch: + +* ./nova.sh install +* ./nova.sh branch +* ./nova.sh run + +A sample image should be downloaded by the script, but if necessary you can download it by hand: + +* wget http://c2477062.cdn.cloudfiles.rackspacecloud.com/images.tgz + +untar the file to create a usable images directory + +* tar -zxf /path/to/images.tgz + +If you want to be able to contact the metadata server and route to the outside world from instances, you will need to make sure $HOST_IP is set properly. The script attemps to grab it from ifconfig, but if you have multiple adapters set up, it may fail. Fix it with export HOST_IP="": + +Customization +------------- + +You can make nova use mysql instead of sqlite with USE_MYSQL, it will attempt to install mysql with the specified root password and create a database called nova. + +If you are running nova on bare metal that supports hardware virtualization, you should probably edit the libvirt line near the top + +* LIBVIRT_TYPE=kvm + +If you are running in a virtual machine and software emulation is too slow for you, you can use user mode linux. + +* LIBVIRT_TYPE=uml + +You will need a few bleeding edge packages to make it work, so you should make sure to use the PPA. + +* USE_PPA=1 + +If you have any issues, there is usually someone in #openstack on irc.freenode.net that can help you out. + + Indices and tables ================== -- cgit From c5e616f5908c0c5966c0a3612b9cc565387f50a9 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 5 Nov 2010 15:11:04 -0400 Subject: Format TODO items for sphinx todo extension. --- doc/source/adminguide/quickstart.rst | 9 ++++----- doc/source/devguide/api.rst | 4 +--- doc/source/devguide/development.environment.rst | 4 +--- doc/source/devguide/fakes.rst | 4 +--- doc/source/devguide/network.rst | 6 +++--- doc/source/devguide/volume.rst | 4 +--- doc/source/index.rst | 5 +++++ doc/source/reaching.out.rst | 5 +---- 8 files changed, 17 insertions(+), 24 deletions(-) (limited to 'doc/source') diff --git a/doc/source/adminguide/quickstart.rst b/doc/source/adminguide/quickstart.rst index 511ed2483..718ea7d14 100644 --- a/doc/source/adminguide/quickstart.rst +++ b/doc/source/adminguide/quickstart.rst @@ -18,12 +18,11 @@ Nova Quickstart =============== -:: +.. todo:: - TODO(todd): * Document the assumptions about pluggable interfaces - (sqlite3 instead of mysql, etc) - * Document env vars that can change things - (USE_MYSQL, HOST_IP) + * Document the assumptions about pluggable interfaces (sqlite3 instead of + mysql, etc) (todd) + * Document env vars that can change things (USE_MYSQL, HOST_IP) (todd) The `contrib/novascript.sh` file in the source distribution is a script that diff --git a/doc/source/devguide/api.rst b/doc/source/devguide/api.rst index 0908d9e57..d1d50d745 100644 --- a/doc/source/devguide/api.rst +++ b/doc/source/devguide/api.rst @@ -18,9 +18,7 @@ API Endpoint Programming Guide ============================== -:: - - TODO(todd): get actual docstrings from ec2/osapi_verions instead of @wsgify +.. todo:: get actual docstrings from ec2/osapi_verions instead of @wsgify Nova has a system for managing multiple APIs on different subdomains. Currently there is support for the OpenStack API, as well as the Amazon EC2 diff --git a/doc/source/devguide/development.environment.rst b/doc/source/devguide/development.environment.rst index ee19ba2bf..34104c964 100644 --- a/doc/source/devguide/development.environment.rst +++ b/doc/source/devguide/development.environment.rst @@ -18,6 +18,4 @@ Setting up a development environment ==================================== -:: - - TODO(anthony) +.. todo:: write this diff --git a/doc/source/devguide/fakes.rst b/doc/source/devguide/fakes.rst index 5988e2ab8..61622754c 100644 --- a/doc/source/devguide/fakes.rst +++ b/doc/source/devguide/fakes.rst @@ -18,9 +18,7 @@ Fakes Programming Guide ======================= -:: - - TODO(todd): * document general info about fakes +.. todo:: document general info about fakes The :mod:`virt.fake` Module diff --git a/doc/source/devguide/network.rst b/doc/source/devguide/network.rst index 9426ece82..318286383 100644 --- a/doc/source/devguide/network.rst +++ b/doc/source/devguide/network.rst @@ -18,10 +18,10 @@ Networking Programming Guide ============================ -:: +.. todo:: - TODO(todd): * document hardware specific commands (maybe in admin guide?) - * document a map between flags and managers/backends + * document hardware specific commands (maybe in admin guide?) (todd) + * document a map between flags and managers/backends (todd) The :mod:`Manager` module ------------------------- diff --git a/doc/source/devguide/volume.rst b/doc/source/devguide/volume.rst index 39b33a500..19b750870 100644 --- a/doc/source/devguide/volume.rst +++ b/doc/source/devguide/volume.rst @@ -18,9 +18,7 @@ Volume Programming Guide ========================= -:: - - TODO(todd): rework after iSCSI merge (see 'Old Docs') +.. todo:: rework after iSCSI merge (see 'Old Docs') (todd or vish) The :mod:`nova.volume.manager` Module diff --git a/doc/source/index.rst b/doc/source/index.rst index 0dcc1945e..58c3cad58 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -48,6 +48,11 @@ Contents reaching.out +Outstanding Documentation Tasks +=============================== + +.. todolist:: + Indices and tables ================== diff --git a/doc/source/reaching.out.rst b/doc/source/reaching.out.rst index 0d8f675e0..064d2d87a 100644 --- a/doc/source/reaching.out.rst +++ b/doc/source/reaching.out.rst @@ -22,7 +22,4 @@ The OpenStack community is a very friendly place. Feel free to ask questions. This document points you to some of the places you may want to communicate with people at. -:: - - TODO(todd): * Write This - * Don't end a sentence with a preposition +.. todo:: Write reaching.out.rst (todd) -- cgit From dd505245c27d2abd2d710a44a120b21cacb9089c Mon Sep 17 00:00:00 2001 From: root Date: Fri, 5 Nov 2010 12:49:42 -0700 Subject: add in custom todo, and custom css --- doc/source/_static/tweaks.css | 44 +++++++++++++++++++++++++++++++++++++++++++ doc/source/_theme/layout.html | 2 ++ doc/source/_theme/theme.conf | 5 +++++ doc/source/conf.py | 8 +++++--- 4 files changed, 56 insertions(+), 3 deletions(-) create mode 100644 doc/source/_static/tweaks.css create mode 100644 doc/source/_theme/layout.html create mode 100644 doc/source/_theme/theme.conf (limited to 'doc/source') diff --git a/doc/source/_static/tweaks.css b/doc/source/_static/tweaks.css new file mode 100644 index 000000000..d1cbed96d --- /dev/null +++ b/doc/source/_static/tweaks.css @@ -0,0 +1,44 @@ +ul.todo_list { + list-style-type: none; + margin: 0; + padding: 0; +} + +ul.todo_list li { + display: block; + margin: 0; + padding: 7px 0; + border-top: 1px solid #eee; +} + +ul.todo_list li p { + display: inline; +} + +ul.todo_list li p.link { + font-weight: bold; +} + +ul.todo_list li p.details { + font-style: italic; +} + +ul.todo_list li { +} + +div.admonition { + border: 1px solid #FF6666; +} + +div.admonition p.admonition-title { + background-color: #FF6666; + border-bottom: 1px solid #FF6666; +} + +em { + font-style: normal; +} + +table.docutils { + font-size: 11px; +} \ No newline at end of file diff --git a/doc/source/_theme/layout.html b/doc/source/_theme/layout.html new file mode 100644 index 000000000..ed1cab0a6 --- /dev/null +++ b/doc/source/_theme/layout.html @@ -0,0 +1,2 @@ +{% extends "sphinxdoc/layout.html" %} +{% set css_files = css_files + ['_static/tweaks.css'] %} diff --git a/doc/source/_theme/theme.conf b/doc/source/_theme/theme.conf new file mode 100644 index 000000000..e039fe01f --- /dev/null +++ b/doc/source/_theme/theme.conf @@ -0,0 +1,5 @@ +[theme] +inherit = sphinxdoc +stylesheet = sphinxdoc.css +pygments_style = friendly + diff --git a/doc/source/conf.py b/doc/source/conf.py index 3f6ca2313..177c6568e 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -17,13 +17,14 @@ import sys, os # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. sys.path.insert(0, os.path.abspath('../../')) - +sys.path.insert(0, '../') +sys.path.insert(0, './') # -- General configuration ----------------------------------------------------- # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.pngmath', 'sphinx.ext.ifconfig'] +extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'ext.nova_todo', 'sphinx.ext.coverage', 'sphinx.ext.pngmath', 'sphinx.ext.ifconfig'] todo_include_todos = True # Add any paths that contain templates here, relative to this directory. @@ -99,7 +100,8 @@ modindex_common_prefix = ['nova.'] # The theme to use for HTML and HTML Help pages. Major themes that come with # Sphinx are currently 'default' and 'sphinxdoc'. -html_theme = 'sphinxdoc' +html_theme_path = ["."] +html_theme = '_theme' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the -- cgit From 3ef09f689b7f151aa9225b69d85c96fd39fd2c9c Mon Sep 17 00:00:00 2001 From: root Date: Fri, 5 Nov 2010 13:25:40 -0700 Subject: use include to grab todd's quickstart --- doc/source/index.rst | 98 +--------------------------------------------------- 1 file changed, 1 insertion(+), 97 deletions(-) (limited to 'doc/source') diff --git a/doc/source/index.rst b/doc/source/index.rst index 5dec404ef..bfc2984fe 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -59,103 +59,7 @@ Although Nova can be run on a variety of system architectures, for most users th For development, Nova can run from within a VM. -Quickstart -========== - -To make getting started with Nova easier, we provide a setup script that makes it easy to download and run the most recent version of Nova on your local machine. - -* sudo -i # become root -* cd # go to home directory -* git clone git://github.com/vishvananda/novascript.git -* cd novascript - -Inside this directory, you will find nova.sh, which is a utility designed to assist with getting your development environment up and running. - -Use nova.sh to install and run the current trunk. You can also specify a specific branch by putting lp:~someone/nova/some-branch after the branch command - -* ./nova.sh branch -* ./nova.sh install -* ./nova.sh run - -The run command will drop you into a screen session with all of the workers running in different windows You can use eucatools to run commands against the cloud. - -* euca-add-keypair test > test.pem -* euca-run-instances -k test -t m1.tiny ami-tiny -* euca-describe-instances - -To see output from the various workers, switch screen windows - -* " - -will give you a list of running windows. - -When the instance is running, you should be able to ssh to it. - -* chmod 600 test.pem -* ssh -i test.pem root@10.0.0.3 - -When you exit screen - -* - -nova will terminate. It may take a while for nova to finish cleaning up. If you exit the process before it is done because there were some problems in your build, you may have to clean up the nova processes manually. If you had any instances running, you can attempt to kill them through the api: - -* ./nova.sh terminate - -Then you can destroy the screen: - -* ./nova.sh clean - -If things get particularly messed up, you might need to do some more intense cleanup. Be careful, the following command will manually destroy all runnning virsh instances and attempt to delete all vlans and bridges. - -* ./nova.sh scrub - -You can edit files in the install directory or do a bzr pull to pick up new versions. You only need to do - -* ./nova.sh run - -to run nova after the first install. The database should be cleaned up on each run. - -Notes ------ - -The script starts nova-volume in fake mode, so it will not create any actual volumes. - -if you want to USE_VENV because you have different versions of python packages on your system that you want to keep, you should run install before branch: - -* ./nova.sh install -* ./nova.sh branch -* ./nova.sh run - -A sample image should be downloaded by the script, but if necessary you can download it by hand: - -* wget http://c2477062.cdn.cloudfiles.rackspacecloud.com/images.tgz - -untar the file to create a usable images directory - -* tar -zxf /path/to/images.tgz - -If you want to be able to contact the metadata server and route to the outside world from instances, you will need to make sure $HOST_IP is set properly. The script attemps to grab it from ifconfig, but if you have multiple adapters set up, it may fail. Fix it with export HOST_IP="": - -Customization -------------- - -You can make nova use mysql instead of sqlite with USE_MYSQL, it will attempt to install mysql with the specified root password and create a database called nova. - -If you are running nova on bare metal that supports hardware virtualization, you should probably edit the libvirt line near the top - -* LIBVIRT_TYPE=kvm - -If you are running in a virtual machine and software emulation is too slow for you, you can use user mode linux. - -* LIBVIRT_TYPE=uml - -You will need a few bleeding edge packages to make it work, so you should make sure to use the PPA. - -* USE_PPA=1 - -If you have any issues, there is usually someone in #openstack on irc.freenode.net that can help you out. - +.. include:: adminguide/quickstart.rst Indices and tables -- cgit From 64ce9316b694ae286ce8caaeb9c486dc9af12c50 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 5 Nov 2010 13:44:22 -0700 Subject: small tweaks before context switch --- doc/source/index.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc/source') diff --git a/doc/source/index.rst b/doc/source/index.rst index bfc2984fe..4da06740c 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -18,7 +18,7 @@ Welcome to Nova's documentation! ================================ -Nova is a cloud computing fabric controller, the main part of an IaaS system. +Nova is a cloud computing fabric controller, the main part of an IaaS system. Individuals and organizations can use it to host and manage their own cloud computing systems. Nova originated as a project out of NASA Ames Research Laboratory. Nova is written with the following design guidelines in mind: @@ -32,7 +32,7 @@ Nova is written with the following design guidelines in mind: This documentation is generated by the Sphinx toolkit and lives in the source tree. Additional documentation on Nova and other components of OpenStack can be found on the `OpenStack wiki`_. Also see the :doc:`reaching.out` page for -other ways to interact witht the community. +other ways to interact with the community. .. _`OpenStack wiki`: http://wiki.openstack.org -- cgit From 07f5d3abe26b57ef8d82ddaab80141e0130d95ac Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 5 Nov 2010 17:33:08 -0400 Subject: Community contact info. --- doc/source/reaching.out.rst | 41 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/reaching.out.rst b/doc/source/reaching.out.rst index 064d2d87a..ee7035aa3 100644 --- a/doc/source/reaching.out.rst +++ b/doc/source/reaching.out.rst @@ -22,4 +22,43 @@ The OpenStack community is a very friendly place. Feel free to ask questions. This document points you to some of the places you may want to communicate with people at. -.. todo:: Write reaching.out.rst (todd) + +#openstack on Freenode IRC Network +---------------------------------- + +There is a very active chat channel at ``_. This +is usually the best place to ask questions and find your way around. + +OpenStack Wiki +-------------- + +The wiki is a living source of knowledge. It is edited by the community, and +has collections of links and other sources of information. + +`OpenStack Wiki `_ + +Nova on Launchpad +----------------- + +Launchapd is a code hosting service that hosts the Nova source code. From +launchpad you can report bugs, ask questions, and register blueprints (feature +requests). + +`Launchpad Nova Page `_ + +OpenStack Blog +-------------- + +The OpenStack blog includes a weekly newsletter that aggregates OpenStack news +from around the internet, as well as providing inside information on upcoming +events and posts from OpenStack contributors. + +`OpenStack Blog `_ + +See also: `Planet OpenStack `_, aggregating blogs +about OpenStack from around the internet into a single feed. + +Twitter +------- + +Because all the cool kids do it: `@openstack `_. -- cgit From 9675be4ffd54db83b7417c968943238958ace9db Mon Sep 17 00:00:00 2001 From: root Date: Fri, 5 Nov 2010 15:29:54 -0700 Subject: add twitter feed to the home page --- doc/source/_static/jquery.tweet.js | 154 +++++++++++++++++++++++++++++++++++++ doc/source/_static/tweaks.css | 10 ++- doc/source/_theme/layout.html | 83 ++++++++++++++++++++ 3 files changed, 246 insertions(+), 1 deletion(-) create mode 100644 doc/source/_static/jquery.tweet.js (limited to 'doc/source') diff --git a/doc/source/_static/jquery.tweet.js b/doc/source/_static/jquery.tweet.js new file mode 100644 index 000000000..c93fea876 --- /dev/null +++ b/doc/source/_static/jquery.tweet.js @@ -0,0 +1,154 @@ +(function($) { + + $.fn.tweet = function(o){ + var s = { + username: ["seaofclouds"], // [string] required, unless you want to display our tweets. :) it can be an array, just do ["username1","username2","etc"] + list: null, //[string] optional name of list belonging to username + avatar_size: null, // [integer] height and width of avatar if displayed (48px max) + count: 3, // [integer] how many tweets to display? + intro_text: null, // [string] do you want text BEFORE your your tweets? + outro_text: null, // [string] do you want text AFTER your tweets? + join_text: null, // [string] optional text in between date and tweet, try setting to "auto" + auto_join_text_default: "i said,", // [string] auto text for non verb: "i said" bullocks + auto_join_text_ed: "i", // [string] auto text for past tense: "i" surfed + auto_join_text_ing: "i am", // [string] auto tense for present tense: "i was" surfing + auto_join_text_reply: "i replied to", // [string] auto tense for replies: "i replied to" @someone "with" + auto_join_text_url: "i was looking at", // [string] auto tense for urls: "i was looking at" http:... + loading_text: null, // [string] optional loading text, displayed while tweets load + query: null // [string] optional search query + }; + + if(o) $.extend(s, o); + + $.fn.extend({ + linkUrl: function() { + var returning = []; + var regexp = /((ftp|http|https):\/\/(\w+:{0,1}\w*@)?(\S+)(:[0-9]+)?(\/|\/([\w#!:.?+=&%@!\-\/]))?)/gi; + this.each(function() { + returning.push(this.replace(regexp,"$1")); + }); + return $(returning); + }, + linkUser: function() { + var returning = []; + var regexp = /[\@]+([A-Za-z0-9-_]+)/gi; + this.each(function() { + returning.push(this.replace(regexp,"@$1")); + }); + return $(returning); + }, + linkHash: function() { + var returning = []; + var regexp = / [\#]+([A-Za-z0-9-_]+)/gi; + this.each(function() { + returning.push(this.replace(regexp, ' #$1')); + }); + return $(returning); + }, + capAwesome: function() { + var returning = []; + this.each(function() { + returning.push(this.replace(/\b(awesome)\b/gi, '$1')); + }); + return $(returning); + }, + capEpic: function() { + var returning = []; + this.each(function() { + returning.push(this.replace(/\b(epic)\b/gi, '$1')); + }); + return $(returning); + }, + makeHeart: function() { + var returning = []; + this.each(function() { + returning.push(this.replace(/(<)+[3]/gi, "♥")); + }); + return $(returning); + } + }); + + function relative_time(time_value) { + var parsed_date = Date.parse(time_value); + var relative_to = (arguments.length > 1) ? arguments[1] : new Date(); + var delta = parseInt((relative_to.getTime() - parsed_date) / 1000); + var pluralize = function (singular, n) { + return '' + n + ' ' + singular + (n == 1 ? '' : 's'); + }; + if(delta < 60) { + return 'less than a minute ago'; + } else if(delta < (45*60)) { + return 'about ' + pluralize("minute", parseInt(delta / 60)) + ' ago'; + } else if(delta < (24*60*60)) { + return 'about ' + pluralize("hour", parseInt(delta / 3600)) + ' ago'; + } else { + return 'about ' + pluralize("day", parseInt(delta / 86400)) + ' ago'; + } + } + + function build_url() { + var proto = ('https:' == document.location.protocol ? 'https:' : 'http:'); + if (s.list) { + return proto+"//api.twitter.com/1/"+s.username[0]+"/lists/"+s.list+"/statuses.json?per_page="+s.count+"&callback=?"; + } else if (s.query == null && s.username.length == 1) { + return proto+'//twitter.com/status/user_timeline/'+s.username[0]+'.json?count='+s.count+'&callback=?'; + } else { + var query = (s.query || 'from:'+s.username.join('%20OR%20from:')); + return proto+'//search.twitter.com/search.json?&q='+query+'&rpp='+s.count+'&callback=?'; + } + } + + return this.each(function(){ + var list = $('
    ').appendTo(this); + var intro = '

    '+s.intro_text+'

    '; + var outro = '

    '+s.outro_text+'

    '; + var loading = $('

    '+s.loading_text+'

    '); + + if(typeof(s.username) == "string"){ + s.username = [s.username]; + } + + if (s.loading_text) $(this).append(loading); + $.getJSON(build_url(), function(data){ + if (s.loading_text) loading.remove(); + if (s.intro_text) list.before(intro); + $.each((data.results || data), function(i,item){ + // auto join text based on verb tense and content + if (s.join_text == "auto") { + if (item.text.match(/^(@([A-Za-z0-9-_]+)) .*/i)) { + var join_text = s.auto_join_text_reply; + } else if (item.text.match(/(^\w+:\/\/[A-Za-z0-9-_]+\.[A-Za-z0-9-_:%&\?\/.=]+) .*/i)) { + var join_text = s.auto_join_text_url; + } else if (item.text.match(/^((\w+ed)|just) .*/im)) { + var join_text = s.auto_join_text_ed; + } else if (item.text.match(/^(\w*ing) .*/i)) { + var join_text = s.auto_join_text_ing; + } else { + var join_text = s.auto_join_text_default; + } + } else { + var join_text = s.join_text; + }; + + var from_user = item.from_user || item.user.screen_name; + var profile_image_url = item.profile_image_url || item.user.profile_image_url; + var join_template = ' '+join_text+' '; + var join = ((s.join_text) ? join_template : ' '); + var avatar_template = ''+from_user+'\'s avatar'; + var avatar = (s.avatar_size ? avatar_template : ''); + var date = ''+relative_time(item.created_at)+''; + var text = '' +$([item.text]).linkUrl().linkUser().linkHash().makeHeart().capAwesome().capEpic()[0]+ ''; + + // until we create a template option, arrange the items below to alter a tweet's display. + list.append('
  • ' + avatar + date + join + text + '
    • '); + + list.children('li:first').addClass('tweet_first'); + list.children('li:odd').addClass('tweet_even'); + list.children('li:even').addClass('tweet_odd'); + }); + if (s.outro_text) list.after(outro); + }); + + }); + }; +})(jQuery); \ No newline at end of file diff --git a/doc/source/_static/tweaks.css b/doc/source/_static/tweaks.css index d1cbed96d..630344555 100644 --- a/doc/source/_static/tweaks.css +++ b/doc/source/_static/tweaks.css @@ -41,4 +41,12 @@ em { table.docutils { font-size: 11px; -} \ No newline at end of file +} + +.tweet_list li { + margin-bottom: 10px; +} + +.tweet_list li .tweet_avatar { + float: left; +} diff --git a/doc/source/_theme/layout.html b/doc/source/_theme/layout.html index ed1cab0a6..d30607ed3 100644 --- a/doc/source/_theme/layout.html +++ b/doc/source/_theme/layout.html @@ -1,2 +1,85 @@ {% extends "sphinxdoc/layout.html" %} {% set css_files = css_files + ['_static/tweaks.css'] %} +{% set script_files = script_files + ['_static/jquery.tweet.js'] %} +{% block extrahead %} + +{% endblock %} + +{%- macro sidebar() %} + {%- if not embedded %}{% if not theme_nosidebar|tobool %} +
    +
    + {%- block sidebarlogo %} + {%- if logo %} + + {%- endif %} + {%- endblock %} + {%- block sidebartoc %} + {%- if display_toc %} +

    {{ _('Table Of Contents') }}

    + {{ toc }} + {%- endif %} + {%- endblock %} + {%- block sidebarrel %} + {%- if prev %} +

    {{ _('Previous topic') }}

    +

    {{ prev.title }}

    + {%- endif %} + {%- if next %} +

    {{ _('Next topic') }}

    +

    {{ next.title }}

    + {%- endif %} + {%- endblock %} + {%- block sidebarsourcelink %} + {%- if show_source and has_source and sourcename %} +

    {{ _('This Page') }}

    + + {%- endif %} + {%- endblock %} + {%- if customsidebar %} + {% include customsidebar %} + {%- endif %} + {%- block sidebarsearch %} + {%- if pagename != "search" %} + + + {%- endif %} + + {%- if pagename == "index" %} +

    {{ _('Twitter Feed') }}

    + + {%- endif %} + + + {%- endblock %} +
    +
    + {%- endif %}{% endif %} +{%- endmacro %} -- cgit From c3054a9a55371fe3acf37589df7d81c6381d7220 Mon Sep 17 00:00:00 2001 From: Todd Willey Date: Fri, 5 Nov 2010 18:31:26 -0400 Subject: Document nova.sh environment. --- doc/source/adminguide/quickstart.rst | 52 +++++++++++++++++++++++++++++++----- 1 file changed, 45 insertions(+), 7 deletions(-) (limited to 'doc/source') diff --git a/doc/source/adminguide/quickstart.rst b/doc/source/adminguide/quickstart.rst index 718ea7d14..7e746e3f2 100644 --- a/doc/source/adminguide/quickstart.rst +++ b/doc/source/adminguide/quickstart.rst @@ -18,16 +18,54 @@ Nova Quickstart =============== -.. todo:: +The `contrib/nova.sh` file in the source distribution is a script that +will quickly set up nova to run on a single machine. It is tested against +Ubuntu only, but other distributions are forthcoming. - * Document the assumptions about pluggable interfaces (sqlite3 instead of - mysql, etc) (todd) - * Document env vars that can change things (USE_MYSQL, HOST_IP) (todd) +Environment Variables +--------------------- +By tweaking the environment that nova.sh run in, you can build slightly +different configurations (though for more complex setups you should see +:doc:`/adminguide/getting.started` and :doc:`/adminguide/multi.node.install`). -The `contrib/novascript.sh` file in the source distribution is a script that -will quickly set up nova to run on a single machine. It is tested against -Ubuntu only, but other distributions are forthcoming. +HOST_IP +~~~~~~~ + +**Default**: address of first interface from the ifconfig command +**Values**: 127.0.0.1, or any other valid address + +TEST +~~~~ + +**Default**: 0 +**Values**: 1, run tests after checkout and initial setup + +USE_MYSQL +~~~~~~~~~ + +**Default**: 0, use sqlite3 +**Values**: 1, use mysql instead of sqlite3 + +MYSQL_PASS +~~~~~~~~~~ + +Only useful if $USE_MYSQL=1. + +**Default**: nova +**Values**: value of root password for mysql + +USE_LDAP +~~~~~~~~ + +**Default**: 0, use :mod:`nova.auth.dbdriver` +**Values**: 1, use :mod:`nova.auth.ldapdriver` + +LIBVIRT_TYPE +~~~~~~~~~~~~ + +**Default**: qemu +**Values**: uml, kvm Usage ----- -- cgit From 9c2af052e4ca34a14b84a9dafcb72830e3351a60 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 5 Nov 2010 15:36:19 -0700 Subject: shrink tweet text a bit --- doc/source/_static/tweaks.css | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'doc/source') diff --git a/doc/source/_static/tweaks.css b/doc/source/_static/tweaks.css index 630344555..ea2ed3cf5 100644 --- a/doc/source/_static/tweaks.css +++ b/doc/source/_static/tweaks.css @@ -44,7 +44,9 @@ table.docutils { } .tweet_list li { - margin-bottom: 10px; + font-size: 0.9em; + border-bottom: 1px solid #eee; + padding: 5px 0; } .tweet_list li .tweet_avatar { -- cgit From d24c7090448ebdb9774b69490df8a983e3d27660 Mon Sep 17 00:00:00 2001 From: root Date: Fri, 5 Nov 2010 15:52:59 -0700 Subject: fix default twitter username --- doc/source/_theme/layout.html | 1 + 1 file changed, 1 insertion(+) (limited to 'doc/source') diff --git a/doc/source/_theme/layout.html b/doc/source/_theme/layout.html index d30607ed3..e3eb54b71 100644 --- a/doc/source/_theme/layout.html +++ b/doc/source/_theme/layout.html @@ -5,6 +5,7 @@