Moving contributor docs into rst (bug #843056)

- Added "Setting up a development environment" doc
  - Removed related info from pip-requires & project README
- Added "Testing" doc
  - Removed related info from project README
- Also removed contributor doc build info from project README (bug #843056)
- Updated version string to '2012.1-dev'

Change-Id: I58c79acd91dc391e3fa85911d09f74ad54d9d444

9 files changed, 239 insertions, 247 deletions

diff --git a/README.md b/README.md
index ba06dd37..539615ac 100644
--- a/README.md
+++ b/README.md
@@ -2,14 +2,9 @@
 
 Keystone is a Python implementation of the [OpenStack](http://www.openstack.org) identity service API.
 
-This project aims to address the current use cases in Swift and Nova which are:
-
-* RESTful token auth for Swift
-* Many-to-many relationship between identity and tenant for Nova.
-
 # Documentation
 
-## For new users, deployers, and system administrators
+## For users and sysadmins
 
 Learn how to install, configure, manage, and interact with the OpenStack
 Identity Service API at the [OpenStack Documentation](http://docs.openstack.org/) site.
@@ -51,41 +46,6 @@ Having trouble? We'd like to help!
 * bin/keystone-service - Provides HTTP API for users
 * bin/keystone-manage - Provides command-line interface for managing all aspects of Keystone
 
-By default, configuration parameters are parsed from `etc/keystone.conf`.
-
-## Installing Dependencies
-
-Keystone maintains a list of PyPi dependencies, designed for use by [pip](http://pypi.python.org/pypi/pip).
-
-*However*, your system may need additional dependencies that `pip` (and by extension, PyPi) cannot satisfy.
-A list of such dependences is maintained in the `tools/pip-requires` file, and should be installed prior to using `pip`.
-
-You may also need to prefix `pip install` with `sudo`, depending on your environment.
-
-    # Describe dependencies (including non-PyPi dependencies)
-    $ cat tools/pip-requires
-
-    # Install all PyPi dependencies (for production, testing, and development)
-    $ pip install -r tools/pip-requires
-
-## Updating your PYTHONPATH
-
-There are a number of methods for getting Keystone into your PYTHON PATH, the easiest of which is:
-
-    # Fake-install the project by symlinking Keystone into your Python site-packages
-    $ python setup.py develop
-
-You should then be able to `import keystone` from your Python shell without issue:
-
-    >>> import keystone
-    >>>
-
-## Testing Keystone
-
-To run the entire test suite, with test progress shown in realtime, use:
-
-    $ ./run_tests.sh --with-progress
-
 ## Running Keystone
 
 Starting both Admin and Service API endpoints:
@@ -114,11 +74,6 @@ Keystone looks in the following location to find a configuration file:
 
 Additional configuration templates are maintained in `keystone/test/etc/` that may be useful as a reference.
 
-## Contributing Changes
-
-Refer to our [Gerrit-Jenkins-Github Workflow](http://wiki.openstack.org/GerritJenkinsGithub).
-## Writing Documentation
-
 ### Editing and Building the API Developer Guide
 
 Users of the Keystone API are often developers making ReSTful API calls to Keystone. The guide to provide them
@@ -135,10 +90,6 @@ To build the Developer Guide from source, you need [Maven](http://maven.apache.o
 
 The output will go into the `target` folder (the source is in `src`). Output generated is PDF and webhelp.
 
-### Editing and Building the Contributor Guide
-
-Refer to `doc/README.rst`.
-
 # Additional Information:
 
 ## Sample data
diff --git a/doc/source/community.rst b/doc/source/community.rst
index 419a5951..860d0c9a 100644
--- a/doc/source/community.rst
+++ b/doc/source/community.rst
@@ -14,6 +14,7 @@
       License for the specific language governing permissions and limitations
       under the License.
 
+================
 Getting Involved
 ================
 
@@ -21,14 +22,20 @@ The Keystone community is a very friendly group and there are places online to j
 community. Feel free to ask questions. This document points you to some of the places where you can
 communicate with people.
 
-How to Join the Keystone Community
-----------------------------------
+How to Join the Community
+=========================
 
 Our community welcomes all people interested in open source cloud computing, and there are no formal
 membership requirements. The best way to join the community is to talk with others online or at a meetup
-and offer contributions through Launchpad, the wiki, or blogs. We welcome all types of contributions, 
-from blueprint designs to documentation to testing to deployment scripts. 
+and offer contributions through Launchpad_, the wiki_, or blogs. We welcome all types of contributions,
+from blueprint designs to documentation to testing to deployment scripts.
+
+.. _Launchpad: https://launchpad.net/keystone
+.. _wiki: http://wiki.openstack.org/
+
+Contributing Changes
+====================
 
-Contributing Code
------------------
+To contribute tests, docs, code, etc, refer to our `Gerrit-Jenkins-Github Workflow`_.
 
+.. _`Gerrit-Jenkins-Github Workflow`: http://wiki.openstack.org/GerritJenkinsGithub
diff --git a/doc/source/configuring.rst b/doc/source/configuration.rst
index 4aa5b4fb..4aa5b4fb 100644
--- a/doc/source/configuring.rst
+++ b/doc/source/configuration.rst
diff --git a/doc/source/index.rst b/doc/source/index.rst
index b2474507..85d19565 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -14,72 +14,68 @@
       License for the specific language governing permissions and limitations
       under the License.
 
+=========================
 Keystone for Contributors
 =========================
 
-Keystone is the canonical implementation of `Openstack's Identity API
-<https://github.com/openstack/identity-api>`_, which provides authentication,
-an OpenStack service catalog, and (soon) authorization services.
+Keystone is a cloud identity service written in Python, which provides
+authentication, authorization, and an OpenStack service catalog. It
+implements `OpenStack's Identity API`_.
+
+This document describes Keystone for contributors of the project, and assumes
+that you are already familiar with Keystone from an `end-user perspective`_.
 
-This document describes keystone for contributors of the project.
+.. _`OpenStack's Identity API`: https://github.com/openstack/identity-api
+.. _`end-user perspective`: http://docs.openstack.org/
 
-Using Keystone
---------------
+Getting Started
+===============
 
 .. toctree::
     :maxdepth: 1
 
-    installing
-    configuring
-    usingkeystone
+    setup
+    testing
+    configuration
+    community
 
-Configuration File Examples
----------------------------
+API Use Case Examples
+=====================
 
 .. toctree::
     :maxdepth: 1
 
-    nova-api-paste
-    keystone.conf
+    adminAPI_curl_examples
+    serviceAPI_curl_examples
 
-Man Pages
----------
+Configuration File Examples
+===========================
 
 .. toctree::
     :maxdepth: 1
 
-    man/keystonemanage.rst
+    nova-api-paste
+    keystone.conf
 
-example API usage
------------------
+Man Pages
+=========
 
 .. toctree::
     :maxdepth: 1
 
-    adminAPI_curl_examples
-    serviceAPI_curl_examples
+    man/keystonemanage.rst
 
 Developer Docs
---------------
+==============
 
 .. toctree::
     :maxdepth: 1
 
     architecture
-    community
-
-.. toctree::
-    :maxdepth: 1
-
     sourcecode/autoindex
 
-Outstanding Documentation Tasks
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. todolist::
-
 Indices and tables
-------------------
+==================
 
 * :ref:`genindex`
 * :ref:`modindex`
diff --git a/doc/source/installing.rst b/doc/source/installing.rst
deleted file mode 100644
index 54991da4..00000000
--- a/doc/source/installing.rst
+++ /dev/null
@@ -1,155 +0,0 @@
-..
-      Copyright 2011 OpenStack, LLC
-      All Rights Reserved.
-
-      Licensed under the Apache License, Version 2.0 (the "License"); you may
-      not use this file except in compliance with the License. You may obtain
-      a copy of the License at
-
-          http://www.apache.org/licenses/LICENSE-2.0
-
-      Unless required by applicable law or agreed to in writing, software
-      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-      License for the specific language governing permissions and limitations
-      under the License.
-
-Installing Keystone
-===================
-
-Installing from packages
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-To install the latest version of Keystone from the Github repositories,
-following the following instructions.
-
-Debian/Ubuntu
-#############
-
-1. Add the Keystone PPA to your sources.lst::
-
-   $> sudo add-apt-repository ppa:keystone-core/trunk
-   $> sudo apt-get update
-
-2. Install Keystone::
-
-   $> sudo apt-get install keystone
-
-
-RedHat/Fedora
-#############
-
-On some OSes, specifically Fedora 15, the current versions of
-greenlet/eventlet segfault when running keystone. To fix this, install
-the development versions of greenlet and eventlet::
-
-    $ pip uninstall greenlet eventlet
-    $ cd <appropriate working directory>
-    $ hg clone https://bitbucket.org/ambroff/greenlet
-    $ cd greenlet
-    $ sudo python setup.py install
-
-    $ cd <appropriate working directory>
-    $ hg clone https://bitbucket.org/which_linden/eventlet
-    $ cd greenlet
-    $ sudo python setup.py install
-
-Installing from source tarballs
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To install the latest version of Keystone from the Launchpad Bazaar repositories,
-following the following instructions.
-
-#. Grab the source tarball from `Github <https://github.com/openstack/keystone>`_
-
-#. Untar the source tarball::
-
-   $> tar -xzf <FILE>
-
-#. Install dependencies::
-
-   $> sudo apt-get install -y git python-pip gcc python-lxml libxml2 python-greenlet-dbg python-dev libsqlite3-dev libldap2-dev libssl-dev libxml2-dev libxslt1-dev libsasl2-dev
-
-#. Change into the package directory and build/install::
-
-   $> cd keystone-<RELEASE>
-   $> sudo python setup.py install
-
-Installing from a Github Branch
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To install the latest version of Keystone from the Github repositories,
-see the following instructions.
-
-Debian/Ubuntu
-#############
-
-.. note::
-   If you want to build the Keystone documentation locally, you will also want
-   to install the python-sphinx package in the first step.
-
-#. Install Git and build dependencies::
-
-   $> sudo apt-get install git python-eventlet python-routes python-greenlet swift
-   $> sudo apt-get install python-argparse python-sqlalchemy python-wsgiref python-pastedeploy
-
-#. Branch Keystone's trunk branch (see http://wiki.openstack.org/GerritWorkflow to get the project initially setup for development)::
-
-   $> git checkout master
-   $> git pull origin master
-
-#. Install Keystone::
-
-   $> sudo python setup.py install
-
-RedHat/Fedora
-#############
-
-#. Install git::
-    $> yum install -y git-core python-setuptools
-
-#. Branch Keystone's trunk branch (see http://wiki.openstack.org/GerritWorkflow to get the project initially setup)::
-
-   $> git checkout master
-   $> git pull origin master
-
-#. Set up the virtual environment to get the additional dependencies
-
-   $> python tools/install_venv.py
-
-   If you don't want to use a virtual environment, install the dependencies
-   directly using:
-
-   $> sudo pip install -r tools/pip-requires
-
-#. Activate the virtual environment
-
-   $> source .keystone-venv/bin/activate
-
-
-Mac OSX
-#######
-
-#. Install git - on your Mac this is most easily done by installing Xcode.
-
-#. Branch Keystone's trunk branch (see http://wiki.openstack.org/GerritWorkflow to get the project initially setup)::
-
-   $> git checkout master
-   $> git pull origin master
-
-#. Set up the virtual environment to get the additional dependencies
-
-   $> python tools/install_venv.py
-
-   If you don't want to use a virtual environment, install the dependencies
-   directly using:
-
-   $> sudo pip install -r tools/pip-requires
-
-#. Activate the virtual environment
-
-   $> source .keystone-venv/bin/activate
-
-#. Install keystone:
-
-   $> python setup.py develop
diff --git a/doc/source/setup.rst b/doc/source/setup.rst
new file mode 100644
index 00000000..c8ae6e21
--- /dev/null
+++ b/doc/source/setup.rst
@@ -0,0 +1,132 @@
+..
+      Copyright 2011 OpenStack, LLC
+      All Rights Reserved.
+
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+===============================================
+Setting up a Keystone a development environment
+===============================================
+
+This document describes setting up keystone directly from GitHub_
+for development purposes.
+
+To install keystone from packaging, refer instead to Keystone's `User Documentation`_.
+
+.. _GitHub: http://github.com/openstack/keystone
+.. _`User Documentation`: http://docs.openstack.org/
+
+Prerequisites
+=============
+
+This document assumes you are using:
+
+- Ubuntu 11.10, Fedora 15, or Mac OS X Lion
+- `Python 2.7`_
+
+.. _`Python 2.7`: http://www.python.org/
+
+And that you have the following tools available on your system:
+
+- git_
+- setuptools_
+- pip_
+
+**Reminder**: If you're successfully using a different platform, or a
+different version of the above, please document your configuration here!
+
+.. _git: http://git-scm.com/
+.. _setuptools: http://pypi.python.org/pypi/setuptools
+
+Installing dependencies
+=======================
+
+Keystone maintains a list of PyPi_ dependencies, designed for use by
+pip_.
+
+.. _PyPi: http://pypi.python.org/
+.. _pip: http://pypi.python.org/pypi/pip
+
+However, your system *may* need additional dependencies that `pip` (and by
+extension, PyPi) cannot satisfy. These dependencies should be installed
+prior to using `pip`, and the installation method may vary depending on
+your platform.
+
+Ubuntu 11.10::
+
+    $ sudo apt-get install python-dev libxml2-dev libxslt1-dev libsasl2-dev libsqlite3-dev libssl-dev libldap2-dev
+
+Fedora 15::
+
+    $ sudo yum install python-sqlite2 python-lxml python-greenlet-devel python-ldap
+
+Mac OS X Lion (requires MacPorts_)::
+
+    $ sudo port install py-ldap
+
+.. _MacPorts: http://www.macports.org/
+
+PyPi Packages
+-------------
+
+Assuming you have any necessary binary packages & header files available
+on your system, you can then install PyPi dependencies.
+
+You may also need to prefix `pip install` with `sudo`, depending on your
+environment::
+
+    # Describe dependencies (including non-PyPi dependencies)
+    $ cat tools/pip-requires
+
+    # Install all PyPi dependencies (for production, testing, and development)
+    $ pip install -r tools/pip-requires
+
+Updating your PYTHONPATH
+========================
+
+There are a number of methods for getting Keystone into your PYTHON PATH,
+the easiest of which is::
+
+    # Fake-install the project by symlinking Keystone into your Python site-packages
+    $ python setup.py develop
+
+You should then be able to `import keystone` from your Python shell
+without issue::
+
+    >>> import keystone
+    >>>
+
+If you can import keystone successfully, you should be ready to move on to :doc:`testing`.
+
+Troubleshooting
+===============
+
+Eventlet segfaults on RedHat / Fedora
+-------------------------------------
+
+[*If this is no longer an issue, please remove this section, thanks!*]
+
+On some OSes, specifically Fedora 15, the current versions of
+greenlet/eventlet segfault when running keystone. To fix this, install
+the development versions of greenlet and eventlet::
+
+    $ pip uninstall greenlet eventlet
+    $ cd <appropriate working directory>
+    $ hg clone https://bitbucket.org/ambroff/greenlet
+    $ cd greenlet
+    $ sudo python setup.py install
+
+    $ cd <appropriate working directory>
+    $ hg clone https://bitbucket.org/which_linden/eventlet
+    $ cd greenlet
+    $ sudo python setup.py install
diff --git a/doc/source/testing.rst b/doc/source/testing.rst
new file mode 100644
index 00000000..35eb16f8
--- /dev/null
+++ b/doc/source/testing.rst
@@ -0,0 +1,62 @@
+================
+Testing Keystone
+================
+
+Keystone uses a number of testing methodologies to ensure correctness.
+
+Running Built-In Tests
+======================
+
+To run the full suites of tests maintained within Keystone, run::
+
+    $ ./run_tests.sh --with-progress
+
+This shows realtime feedback during test execution, and iterates over
+multiple configuration variations.
+
+This differs from how tests are executed from the continuous integration
+environment. Specifically, Jenkins doesn't care about realtime progress,
+and aborts after the first test failure (a fail-fast behavior)::
+
+    $ ./run_tests.sh
+
+Writing Tests
+=============
+
+Tests are maintained in the ``keystone.test`` module. Unit tests are
+isolated from functional tests.
+
+Functional Tests
+----------------
+
+The ``keystone.test.functional.common`` module provides a ``unittest``-based
+``httplib`` client which you can extend and use for your own tests.
+Generally, functional tests should serve to illustrate intended use cases
+and API behaviors. To help make your tests easier to read, the test client:
+
+- Authenticates with a known user name and password combination
+- Asserts 2xx HTTP status codes (unless told otherwise)
+- Abstracts keystone REST verbs & resources into single function calls
+
+Testing Multiple Configurations
+-------------------------------
+
+Several variations of the default configuration are iterated over to
+ensure test coverage of mutually exclusive featuresets, such as the
+various backend options.
+
+These configuration templates are maintained in ``keystone/test/etc`` and
+are iterated over by ``run_tests.py``.
+
+Further Testing
+===============
+
+devstack_ is the *best* way to quickly deploy keystone with the rest of the
+OpenStack universe and should be critical step in your development workflow!
+
+You may also be interested in either the `OpenStack Continuous Integration Project`_
+or the `OpenStack Integration Testing Project`_.
+
+.. _devstack: http://devstack.org/
+.. _OpenStack Continuous Integration Project: https://github.com/openstack/openstack-ci
+.. _OpenStack Integration Testing Project: https://github.com/openstack/openstack-integration-tests
diff --git a/keystone/__init__.py b/keystone/__init__.py
index fff87863..68bc1057 100644
--- a/keystone/__init__.py
+++ b/keystone/__init__.py
@@ -19,7 +19,7 @@ API_VERSION = "2.0"
 API_VERSION_STATUS = "beta"
 API_VERSION_DATE = "2011-11-19T00:00:00Z"
 
-RELEASE_VERSION = "essex"
+RELEASE_VERSION = "2012.1"
 RELEASE_VERSION_FINAL = False  # becomes true at Release Candidate time
 
 
diff --git a/tools/pip-requires b/tools/pip-requires
index fbace42b..1400093d 100644
--- a/tools/pip-requires
+++ b/tools/pip-requires
@@ -1,6 +1,5 @@
-# You may need to install development files before using 'pip install'
-# For example:
-#   sudo apt-get install python-dev libxml2-dev libxslt1-dev libsasl2-dev libldap2-dev libsqlite3-dev libssl-dev
+# NOTE: You may need to install additional binary packages prior to using 'pip install'
+#       See the Contributor Documentation for more information
 
 # Production
 httplib2 # handles additional HTTP features such as SSL, HEAD/PUT/DELETE, etc