Added separate bare-metal MySQL DB.

Part 2 of 6: blueprint general-bare-metal-provisioning-framework

In baremetal provisioning, one nova-compute manages multiple bare-metal
machines. A bare-metal machine does not run openstack at all.
Previously, bare-metal provisioning used text files to store information
of bare-metal machines. In this patch, a MySQL database is used to store
the information. We target only MySQL database. The DB is designed to
support PXE/non-PXE booting methods, heterogeneous hypervisor types, and
architectures. Using a MySQL database makes maintenance and upgrades
easier than using text files. The DB for bare-metal machines is
implemented as a separate DB from the main Nova DB. The DB can be on any
machines/places. The location of the DB and its server needs to be
specified as a flag in the nova.conf file (as in the case of glance).
There are a couple of reasons for this approach. First, the information
needed for bare-metal machines is different from that for non-bare-metal
machines. With a separate database for bare-metal machines, the database
can be customized without affecting the main Nova DB. Second, fault
tolerance can be embedded in nova-compute. Since one nova-compute
manages multiple bare-metal machines, fault tolerance of a nova-compute
node is very important. With a separate DB for bare-metal machines,
fault-tolerance can be achieved independently from the main Nova DB.
Replication of the bare-metal DB and implementation of fault-tolerance
are not part of this patch. The implementation models nova and its DB as
much as possible. The bare-metal driver must be upgraded to use this DB.

Change-Id: I7b7ba1903a672a50c567f95fc6554d119463b0c5
Co-authored-by: Mikyung Kang <mkkang@isi.edu>
Co-authored-by: David Kang <dkang@isi.edu>
Co-authored-by: Ken Igarashi <igarashik@nttdocomo.co.jp>
Co-authored-by: Arata Notsu <notsu@virtualtech.jp>

13 files changed, 1088 insertions, 0 deletions

diff --git a/nova/virt/baremetal/db/__init__.py b/nova/virt/baremetal/db/__init__.py
new file mode 100644
index 000000000..ad883f505
--- /dev/null
+++ b/nova/virt/baremetal/db/__init__.py
@@ -0,0 +1,16 @@
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from nova.virt.baremetal.db.api import *
diff --git a/nova/virt/baremetal/db/api.py b/nova/virt/baremetal/db/api.py
new file mode 100644
index 000000000..a9b6b3fe2
--- /dev/null
+++ b/nova/virt/baremetal/db/api.py
@@ -0,0 +1,175 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# Copyright (c) 2011 X.commerce, a business unit of eBay Inc.
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""Defines interface for DB access.
+
+The underlying driver is loaded as a :class:`LazyPluggable`.
+
+Functions in this module are imported into the nova.virt.baremetal.db
+namespace. Call these functions from nova.virt.baremetal.db namespace, not
+the nova.virt.baremetal.db.api namespace.
+
+All functions in this module return objects that implement a dictionary-like
+interface. Currently, many of these objects are sqlalchemy objects that
+implement a dictionary interface. However, a future goal is to have all of
+these objects be simple dictionaries.
+
+
+**Related Flags**
+
+:baremetal_db_backend:  string to lookup in the list of LazyPluggable backends.
+                        `sqlalchemy` is the only supported backend right now.
+
+:baremetal_sql_connection:  string specifying the sqlalchemy connection to use,
+                            like: `sqlite:///var/lib/nova/nova.sqlite`.
+
+"""
+
+from nova import config
+from nova.openstack.common import cfg
+from nova import utils
+
+
+db_opts = [
+    cfg.StrOpt('baremetal_db_backend',
+               default='sqlalchemy',
+               help='The backend to use for db'),
+    ]
+
+CONF = config.CONF
+CONF.register_opts(db_opts)
+
+IMPL = utils.LazyPluggable(
+        'baremetal_db_backend',
+        sqlalchemy='nova.virt.baremetal.db.sqlalchemy.api')
+
+
+def bm_node_get_all(context, service_host=None):
+    return IMPL.bm_node_get_all(context,
+                                service_host=service_host)
+
+
+def bm_node_find_free(context, service_host=None,
+                      memory_mb=None, cpus=None, local_gb=None):
+    return IMPL.bm_node_find_free(context,
+                                  service_host=service_host,
+                                  memory_mb=memory_mb,
+                                  cpus=cpus,
+                                  local_gb=local_gb)
+
+
+def bm_node_get(context, bm_node_id):
+    return IMPL.bm_node_get(context, bm_node_id)
+
+
+def bm_node_get_by_instance_uuid(context, instance_uuid):
+    return IMPL.bm_node_get_by_instance_uuid(context,
+                                             instance_uuid)
+
+
+def bm_node_create(context, values):
+    return IMPL.bm_node_create(context, values)
+
+
+def bm_node_destroy(context, bm_node_id):
+    return IMPL.bm_node_destroy(context, bm_node_id)
+
+
+def bm_node_update(context, bm_node_id, values):
+    return IMPL.bm_node_update(context, bm_node_id, values)
+
+
+def bm_pxe_ip_create(context, address, server_address):
+    return IMPL.bm_pxe_ip_create(context, address, server_address)
+
+
+def bm_pxe_ip_create_direct(context, bm_pxe_ip):
+    return IMPL.bm_pxe_ip_create_direct(context, bm_pxe_ip)
+
+
+def bm_pxe_ip_destroy(context, ip_id):
+    return IMPL.bm_pxe_ip_destroy(context, ip_id)
+
+
+def bm_pxe_ip_destroy_by_address(context, address):
+    return IMPL.bm_pxe_ip_destroy_by_address(context, address)
+
+
+def bm_pxe_ip_get_all(context):
+    return IMPL.bm_pxe_ip_get_all(context)
+
+
+def bm_pxe_ip_get(context, ip_id):
+    return IMPL.bm_pxe_ip_get(context, ip_id)
+
+
+def bm_pxe_ip_get_by_bm_node_id(context, bm_node_id):
+    return IMPL.bm_pxe_ip_get_by_bm_node_id(context, bm_node_id)
+
+
+def bm_pxe_ip_associate(context, bm_node_id):
+    return IMPL.bm_pxe_ip_associate(context, bm_node_id)
+
+
+def bm_pxe_ip_disassociate(context, bm_node_id):
+    return IMPL.bm_pxe_ip_disassociate(context, bm_node_id)
+
+
+def bm_interface_get(context, if_id):
+    return IMPL.bm_interface_get(context, if_id)
+
+
+def bm_interface_get_all(context):
+    return IMPL.bm_interface_get_all(context)
+
+
+def bm_interface_destroy(context, if_id):
+    return IMPL.bm_interface_destroy(context, if_id)
+
+
+def bm_interface_create(context, bm_node_id, address, datapath_id, port_no):
+    return IMPL.bm_interface_create(context, bm_node_id, address,
+                                    datapath_id, port_no)
+
+
+def bm_interface_set_vif_uuid(context, if_id, vif_uuid):
+    return IMPL.bm_interface_set_vif_uuid(context, if_id, vif_uuid)
+
+
+def bm_interface_get_by_vif_uuid(context, vif_uuid):
+    return IMPL.bm_interface_get_by_vif_uuid(context, vif_uuid)
+
+
+def bm_interface_get_all_by_bm_node_id(context, bm_node_id):
+    return IMPL.bm_interface_get_all_by_bm_node_id(context, bm_node_id)
+
+
+def bm_deployment_create(context, key, image_path, pxe_config_path, root_mb,
+                         swap_mb):
+    return IMPL.bm_deployment_create(context, key, image_path,
+                                     pxe_config_path, root_mb, swap_mb)
+
+
+def bm_deployment_get(context, dep_id):
+    return IMPL.bm_deployment_get(context, dep_id)
+
+
+def bm_deployment_destroy(context, dep_id):
+    return IMPL.bm_deployment_destroy(context, dep_id)
diff --git a/nova/virt/baremetal/db/migration.py b/nova/virt/baremetal/db/migration.py
new file mode 100644
index 000000000..40631bf45
--- /dev/null
+++ b/nova/virt/baremetal/db/migration.py
@@ -0,0 +1,38 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""Database setup and migration commands."""
+
+from nova import utils
+
+
+IMPL = utils.LazyPluggable(
+        'baremetal_db_backend',
+        sqlalchemy='nova.virt.baremetal.db.sqlalchemy.migration')
+
+INIT_VERSION = 0
+
+
+def db_sync(version=None):
+    """Migrate the database to `version` or the most recent version."""
+    return IMPL.db_sync(version=version)
+
+
+def db_version():
+    """Display the current database version."""
+    return IMPL.db_version()
diff --git a/nova/virt/baremetal/db/sqlalchemy/__init__.py b/nova/virt/baremetal/db/sqlalchemy/__init__.py
new file mode 100644
index 000000000..19071662c
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/__init__.py
@@ -0,0 +1,14 @@
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
diff --git a/nova/virt/baremetal/db/sqlalchemy/api.py b/nova/virt/baremetal/db/sqlalchemy/api.py
new file mode 100644
index 000000000..1127d77e8
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/api.py
@@ -0,0 +1,351 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# Copyright (c) 2011 X.commerce, a business unit of eBay Inc.
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""Implementation of SQLAlchemy backend."""
+
+from sqlalchemy import and_
+from sqlalchemy.exc import IntegrityError
+from sqlalchemy import or_
+from sqlalchemy.orm import joinedload
+from sqlalchemy.orm import joinedload_all
+from sqlalchemy.sql.expression import asc
+from sqlalchemy.sql.expression import desc
+from sqlalchemy.sql.expression import literal_column
+from sqlalchemy.sql import func
+
+from nova.db.sqlalchemy.api import is_user_context
+from nova.db.sqlalchemy.api import require_admin_context
+from nova import exception
+from nova.openstack.common import log as logging
+from nova.openstack.common import timeutils
+from nova.virt.baremetal.db.sqlalchemy import models
+from nova.virt.baremetal.db.sqlalchemy.session import get_session
+
+LOG = logging.getLogger(__name__)
+
+
+def model_query(context, *args, **kwargs):
+    """Query helper that accounts for context's `read_deleted` field.
+
+    :param context: context to query under
+    :param session: if present, the session to use
+    :param read_deleted: if present, overrides context's read_deleted field.
+    :param project_only: if present and context is user-type, then restrict
+            query to match the context's project_id.
+    """
+    session = kwargs.get('session') or get_session()
+    read_deleted = kwargs.get('read_deleted') or context.read_deleted
+    project_only = kwargs.get('project_only')
+
+    query = session.query(*args)
+
+    if read_deleted == 'no':
+        query = query.filter_by(deleted=False)
+    elif read_deleted == 'yes':
+        pass  # omit the filter to include deleted and active
+    elif read_deleted == 'only':
+        query = query.filter_by(deleted=True)
+    else:
+        raise Exception(
+                _("Unrecognized read_deleted value '%s'") % read_deleted)
+
+    if project_only and is_user_context(context):
+        query = query.filter_by(project_id=context.project_id)
+
+    return query
+
+
+def _save(ref, session=None):
+    if not session:
+        session = get_session()
+    # We must not call ref.save() with session=None, otherwise NovaBase
+    # uses nova-db's session, which cannot access bm-db.
+    ref.save(session=session)
+
+
+def _build_node_order_by(query):
+    query = query.order_by(asc(models.BareMetalNode.memory_mb))
+    query = query.order_by(asc(models.BareMetalNode.cpus))
+    query = query.order_by(asc(models.BareMetalNode.local_gb))
+    return query
+
+
+@require_admin_context
+def bm_node_get_all(context, service_host=None):
+    query = model_query(context, models.BareMetalNode, read_deleted="no")
+    if service_host:
+        query = query.filter_by(service_host=service_host)
+    return query.all()
+
+
+@require_admin_context
+def bm_node_find_free(context, service_host=None,
+                      cpus=None, memory_mb=None, local_gb=None):
+    query = model_query(context, models.BareMetalNode, read_deleted="no")
+    query = query.filter(models.BareMetalNode.instance_uuid == None)
+    if service_host:
+        query = query.filter_by(service_host=service_host)
+    if cpus is not None:
+        query = query.filter(models.BareMetalNode.cpus >= cpus)
+    if memory_mb is not None:
+        query = query.filter(models.BareMetalNode.memory_mb >= memory_mb)
+    if local_gb is not None:
+        query = query.filter(models.BareMetalNode.local_gb >= local_gb)
+    query = _build_node_order_by(query)
+    return query.first()
+
+
+@require_admin_context
+def bm_node_get(context, bm_node_id):
+    result = model_query(context, models.BareMetalNode, read_deleted="no").\
+                     filter_by(id=bm_node_id).\
+                     first()
+    return result
+
+
+@require_admin_context
+def bm_node_get_by_instance_uuid(context, instance_uuid):
+    result = model_query(context, models.BareMetalNode, read_deleted="no").\
+                     filter_by(instance_uuid=instance_uuid).\
+                     first()
+    return result
+
+
+@require_admin_context
+def bm_node_create(context, values):
+    bm_node_ref = models.BareMetalNode()
+    bm_node_ref.update(values)
+    _save(bm_node_ref)
+    return bm_node_ref
+
+
+@require_admin_context
+def bm_node_update(context, bm_node_id, values, ):
+    model_query(context, models.BareMetalNode, read_deleted="no").\
+            filter_by(id=bm_node_id).\
+            update(values)
+
+
+@require_admin_context
+def bm_node_destroy(context, bm_node_id):
+    model_query(context, models.BareMetalNode).\
+            filter_by(id=bm_node_id).\
+            update({'deleted': True,
+                    'deleted_at': timeutils.utcnow(),
+                    'updated_at': literal_column('updated_at')})
+
+
+@require_admin_context
+def bm_pxe_ip_get_all(context, session=None):
+    query = model_query(context, models.BareMetalPxeIp, read_deleted="no")
+    return query.all()
+
+
+@require_admin_context
+def bm_pxe_ip_create(context, address, server_address):
+    ref = models.BareMetalPxeIp()
+    ref.address = address
+    ref.server_address = server_address
+    _save(ref)
+    return ref
+
+
+@require_admin_context
+def bm_pxe_ip_create_direct(context, bm_pxe_ip):
+    ref = bm_pxe_ip_create(context,
+                           address=bm_pxe_ip['address'],
+                           server_address=bm_pxe_ip['server_address'])
+    return ref
+
+
+@require_admin_context
+def bm_pxe_ip_destroy(context, ip_id):
+    # Delete physically since it has unique columns
+    model_query(context, models.BareMetalPxeIp, read_deleted="no").\
+            filter_by(id=ip_id).\
+            delete()
+
+
+@require_admin_context
+def bm_pxe_ip_destroy_by_address(context, address):
+    # Delete physically since it has unique columns
+    model_query(context, models.BareMetalPxeIp, read_deleted="no").\
+            filter_by(address=address).\
+            delete()
+
+
+@require_admin_context
+def bm_pxe_ip_get(context, ip_id):
+    ref = model_query(context, models.BareMetalPxeIp, read_deleted="no").\
+            filter_by(id=ip_id).\
+            first()
+    return ref
+
+
+@require_admin_context
+def bm_pxe_ip_get_by_bm_node_id(context, bm_node_id):
+    ref = model_query(context, models.BareMetalPxeIp, read_deleted="no").\
+            filter_by(bm_node_id=bm_node_id).\
+            first()
+    return ref
+
+
+@require_admin_context
+def bm_pxe_ip_associate(context, bm_node_id):
+    session = get_session()
+    with session.begin():
+        # Check if the node really exists
+        node_ref = model_query(context, models.BareMetalNode,
+                               read_deleted="no", session=session).\
+                     filter_by(id=bm_node_id).\
+                     first()
+        if not node_ref:
+            raise exception.NovaException("bm_node %s not found" % bm_node_id)
+        # Check if the node already has a pxe_ip
+        ip_ref = model_query(context, models.BareMetalPxeIp,
+                             read_deleted="no", session=session).\
+                         filter_by(bm_node_id=bm_node_id).\
+                         first()
+        if ip_ref:
+            return ip_ref.id
+        # with_lockmode('update') and filter_by(bm_node_id=None) will lock all
+        # records. It may cause a performance problem in high-concurrency
+        # environment.
+        ip_ref = model_query(context, models.BareMetalPxeIp,
+                             read_deleted="no", session=session).\
+                         filter_by(bm_node_id=None).\
+                         with_lockmode('update').\
+                         first()
+        if not ip_ref:
+            raise exception.NovaException("free bm_pxe_ip not found")
+        ip_ref.bm_node_id = bm_node_id
+        session.add(ip_ref)
+        return ip_ref.id
+
+
+@require_admin_context
+def bm_pxe_ip_disassociate(context, bm_node_id):
+    model_query(context, models.BareMetalPxeIp, read_deleted="no").\
+            filter_by(bm_node_id=bm_node_id).\
+            update({'bm_node_id': None})
+
+
+@require_admin_context
+def bm_interface_get(context, if_id):
+    result = model_query(context, models.BareMetalInterface,
+                         read_deleted="no").\
+                     filter_by(id=if_id).\
+                     first()
+    return result
+
+
+def bm_interface_get_all(context):
+    query = model_query(context, models.BareMetalInterface,
+                        read_deleted="no")
+    return query.all()
+
+
+@require_admin_context
+def bm_interface_destroy(context, if_id):
+    # Delete physically since it has unique columns
+    model_query(context, models.BareMetalInterface, read_deleted="no").\
+            filter_by(id=if_id).\
+            delete()
+
+
+@require_admin_context
+def bm_interface_create(context, bm_node_id, address, datapath_id, port_no):
+    ref = models.BareMetalInterface()
+    ref.bm_node_id = bm_node_id
+    ref.address = address
+    ref.datapath_id = datapath_id
+    ref.port_no = port_no
+    _save(ref)
+    return ref.id
+
+
+@require_admin_context
+def bm_interface_set_vif_uuid(context, if_id, vif_uuid):
+    session = get_session()
+    with session.begin():
+        ref = model_query(context, models.BareMetalInterface,
+                          read_deleted="no", session=session).\
+                         filter_by(id=if_id).\
+                         with_lockmode('update').\
+                         first()
+        if not ref:
+            raise exception.NovaException('interface id=%s is not found' %
+                                          if_id)
+        ref.vif_uuid = vif_uuid
+        try:
+            session.add(ref)
+            session.flush()
+        except IntegrityError:
+            raise exception.NovaException('vif_uuid %s is already assigned' %
+                                          vif_uuid)
+
+
+@require_admin_context
+def bm_interface_get_by_vif_uuid(context, vif_uuid):
+    result = model_query(context, models.BareMetalInterface,
+                         read_deleted="no").\
+                filter_by(vif_uuid=vif_uuid).\
+                first()
+    return result
+
+
+@require_admin_context
+def bm_interface_get_all_by_bm_node_id(context, bm_node_id):
+    result = model_query(context, models.BareMetalInterface,
+                         read_deleted="no").\
+                 filter_by(bm_node_id=bm_node_id).\
+                 all()
+    return result
+
+
+@require_admin_context
+def bm_deployment_create(context, key, image_path, pxe_config_path, root_mb,
+                         swap_mb):
+    ref = models.BareMetalDeployment()
+    ref.key = key
+    ref.image_path = image_path
+    ref.pxe_config_path = pxe_config_path
+    ref.root_mb = root_mb
+    ref.swap_mb = swap_mb
+    _save(ref)
+    return ref.id
+
+
+@require_admin_context
+def bm_deployment_get(context, dep_id):
+    result = model_query(context, models.BareMetalDeployment,
+                         read_deleted="no").\
+                     filter_by(id=dep_id).\
+                     first()
+    return result
+
+
+@require_admin_context
+def bm_deployment_destroy(context, dep_id):
+    model_query(context, models.BareMetalDeployment).\
+                filter_by(id=dep_id).\
+                update({'deleted': True,
+                        'deleted_at': timeutils.utcnow(),
+                        'updated_at': literal_column('updated_at')})
diff --git a/nova/virt/baremetal/db/sqlalchemy/migrate_repo/__init__.py b/nova/virt/baremetal/db/sqlalchemy/migrate_repo/__init__.py
new file mode 100644
index 000000000..19071662c
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/migrate_repo/__init__.py
@@ -0,0 +1,14 @@
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
diff --git a/nova/virt/baremetal/db/sqlalchemy/migrate_repo/migrate.cfg b/nova/virt/baremetal/db/sqlalchemy/migrate_repo/migrate.cfg
new file mode 100644
index 000000000..368e93a52
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/migrate_repo/migrate.cfg
@@ -0,0 +1,20 @@
+[db_settings]
+# Used to identify which repository this database is versioned under.
+# You can use the name of your project.
+repository_id=nova_bm
+
+# The name of the database table used to track the schema version.
+# This name shouldn't already be used by your project.
+# If this is changed once a database is under version control, you'll need to
+# change the table name in each database too.
+version_table=migrate_version
+
+# When committing a change script, Migrate will attempt to generate the
+# sql for all supported databases; normally, if one of them fails - probably
+# because you don't have that database installed - it is ignored and the
+# commit continues, perhaps ending successfully.
+# Databases in this list MUST compile successfully during a commit, or the
+# entire commit will fail. List the databases your application will actually
+# be using to ensure your updates to that database work properly.
+# This must be a list; example: ['postgres','sqlite']
+required_dbs=[]
diff --git a/nova/virt/baremetal/db/sqlalchemy/migrate_repo/versions/001_init.py b/nova/virt/baremetal/db/sqlalchemy/migrate_repo/versions/001_init.py
new file mode 100644
index 000000000..d945755fc
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/migrate_repo/versions/001_init.py
@@ -0,0 +1,124 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from migrate import ForeignKeyConstraint
+from sqlalchemy import Boolean, BigInteger, Column, DateTime, Float, ForeignKey
+from sqlalchemy import Index, Integer, MetaData, String, Table, Text
+
+from nova.openstack.common import log as logging
+
+LOG = logging.getLogger(__name__)
+
+
+def upgrade(migrate_engine):
+    meta = MetaData()
+    meta.bind = migrate_engine
+
+    bm_nodes = Table('bm_nodes', meta,
+        Column('created_at', DateTime),
+        Column('updated_at', DateTime),
+        Column('deleted_at', DateTime),
+        Column('deleted', Boolean),
+        Column('id', Integer, primary_key=True, nullable=False),
+        Column('cpus', Integer),
+        Column('memory_mb', Integer),
+        Column('local_gb', Integer),
+        Column('pm_address', String(length=255)),
+        Column('pm_user', String(length=255)),
+        Column('pm_password', String(length=255)),
+        Column('service_host', String(length=255)),
+        Column('prov_mac_address', String(length=255)),
+        Column('instance_uuid', String(length=36)),
+        Column('registration_status', String(length=16)),
+        Column('task_state', String(length=255)),
+        Column('prov_vlan_id', Integer),
+        Column('terminal_port', Integer),
+        mysql_engine='InnoDB',
+        #mysql_charset='utf8'
+    )
+
+    bm_interfaces = Table('bm_interfaces', meta,
+        Column('created_at', DateTime),
+        Column('updated_at', DateTime),
+        Column('deleted_at', DateTime),
+        Column('deleted', Boolean),
+        Column('id', Integer, primary_key=True, nullable=False),
+        Column('bm_node_id', Integer),
+        Column('address', String(length=255), unique=True),
+        Column('datapath_id', String(length=255)),
+        Column('port_no', Integer),
+        Column('vif_uuid', String(length=36), unique=True),
+        mysql_engine='InnoDB',
+        #mysql_charset='utf8'
+    )
+
+    bm_pxe_ips = Table('bm_pxe_ips', meta,
+        Column('created_at', DateTime),
+        Column('updated_at', DateTime),
+        Column('deleted_at', DateTime),
+        Column('deleted', Boolean),
+        Column('id', Integer, primary_key=True, nullable=False),
+        Column('address', String(length=255), unique=True),
+        Column('bm_node_id', Integer),
+        Column('server_address', String(length=255), unique=True),
+        mysql_engine='InnoDB',
+        #mysql_charset='utf8'
+    )
+
+    bm_deployments = Table('bm_deployments', meta,
+        Column('created_at', DateTime),
+        Column('updated_at', DateTime),
+        Column('deleted_at', DateTime),
+        Column('deleted', Boolean),
+        Column('id', Integer, primary_key=True, nullable=False),
+        Column('bm_node_id', Integer),
+        Column('key', String(length=255)),
+        Column('image_path', String(length=255)),
+        Column('pxe_config_path', String(length=255)),
+        Column('root_mb', Integer),
+        Column('swap_mb', Integer),
+        mysql_engine='InnoDB',
+        #mysql_charset='utf8'
+    )
+
+    bm_nodes.create()
+    bm_interfaces.create()
+    bm_pxe_ips.create()
+    bm_deployments.create()
+
+    Index('idx_bm_nodes_service_host_deleted',
+          bm_nodes.c.service_host, bm_nodes.c.deleted)\
+          .create(migrate_engine)
+    Index('idx_bm_nodes_instance_uuid_deleted',
+          bm_nodes.c.instance_uuid, bm_nodes.c.deleted)\
+          .create(migrate_engine)
+    Index('idx_bm_nodes_hmcld',
+          bm_nodes.c.service_host, bm_nodes.c.memory_mb, bm_nodes.c.cpus,
+          bm_nodes.c.local_gb, bm_nodes.c.deleted)\
+          .create(migrate_engine)
+
+    Index('idx_bm_interfaces_bm_node_id_deleted',
+          bm_interfaces.c.bm_node_id, bm_interfaces.c.deleted)\
+          .create(migrate_engine)
+
+    Index('idx_bm_pxe_ips_bm_node_id_deleted',
+          bm_pxe_ips.c.bm_node_id, bm_pxe_ips.c.deleted)\
+          .create(migrate_engine)
+
+
+def downgrade(migrate_engine):
+    pass
diff --git a/nova/virt/baremetal/db/sqlalchemy/migrate_repo/versions/__init__.py b/nova/virt/baremetal/db/sqlalchemy/migrate_repo/versions/__init__.py
new file mode 100644
index 000000000..19071662c
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/migrate_repo/versions/__init__.py
@@ -0,0 +1,14 @@
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
diff --git a/nova/virt/baremetal/db/sqlalchemy/migration.py b/nova/virt/baremetal/db/sqlalchemy/migration.py
new file mode 100644
index 000000000..473f6e6f6
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/migration.py
@@ -0,0 +1,115 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+import distutils.version as dist_version
+import migrate
+from migrate.versioning import util as migrate_util
+import os
+import sqlalchemy
+
+from nova import exception
+from nova import flags
+from nova.openstack.common import log as logging
+from nova.virt.baremetal.db import migration
+from nova.virt.baremetal.db.sqlalchemy.session import get_engine
+
+LOG = logging.getLogger(__name__)
+
+
+@migrate_util.decorator
+def patched_with_engine(f, *a, **kw):
+    url = a[0]
+    engine = migrate_util.construct_engine(url, **kw)
+
+    try:
+        kw['engine'] = engine
+        return f(*a, **kw)
+    finally:
+        if isinstance(engine, migrate_util.Engine) and engine is not url:
+            migrate_util.log.debug('Disposing SQLAlchemy engine %s', engine)
+            engine.dispose()
+
+
+# TODO(jkoelker) When migrate 0.7.3 is released and nova depends
+#                on that version or higher, this can be removed
+MIN_PKG_VERSION = dist_version.StrictVersion('0.7.3')
+if (not hasattr(migrate, '__version__') or
+    dist_version.StrictVersion(migrate.__version__) < MIN_PKG_VERSION):
+    migrate_util.with_engine = patched_with_engine
+
+
+# NOTE(jkoelker) Delay importing migrate until we are patched
+from migrate import exceptions as versioning_exceptions
+from migrate.versioning import api as versioning_api
+from migrate.versioning.repository import Repository
+
+FLAGS = flags.FLAGS
+
+_REPOSITORY = None
+
+
+def db_sync(version=None):
+    if version is not None:
+        try:
+            version = int(version)
+        except ValueError:
+            raise exception.NovaException(_("version should be an integer"))
+
+    current_version = db_version()
+    repository = _find_migrate_repo()
+    if version is None or version > current_version:
+        return versioning_api.upgrade(get_engine(), repository, version)
+    else:
+        return versioning_api.downgrade(get_engine(), repository,
+                                        version)
+
+
+def db_version():
+    repository = _find_migrate_repo()
+    try:
+        return versioning_api.db_version(get_engine(), repository)
+    except versioning_exceptions.DatabaseNotControlledError:
+        meta = sqlalchemy.MetaData()
+        engine = get_engine()
+        meta.reflect(bind=engine)
+        tables = meta.tables
+        if len(tables) == 0:
+            db_version_control(migration.INIT_VERSION)
+            return versioning_api.db_version(get_engine(), repository)
+        else:
+            # Some pre-Essex DB's may not be version controlled.
+            # Require them to upgrade using Essex first.
+            raise exception.NovaException(
+                _("Upgrade DB using Essex release first."))
+
+
+def db_version_control(version=None):
+    repository = _find_migrate_repo()
+    versioning_api.version_control(get_engine(), repository, version)
+    return version
+
+
+def _find_migrate_repo():
+    """Get the path for the migrate repository."""
+    global _REPOSITORY
+    path = os.path.join(os.path.abspath(os.path.dirname(__file__)),
+                        'migrate_repo')
+    assert os.path.exists(path)
+    if _REPOSITORY is None:
+        _REPOSITORY = Repository(path)
+    return _REPOSITORY
diff --git a/nova/virt/baremetal/db/sqlalchemy/models.py b/nova/virt/baremetal/db/sqlalchemy/models.py
new file mode 100644
index 000000000..c1ab191d0
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/models.py
@@ -0,0 +1,80 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""
+SQLAlchemy models for baremetal data.
+"""
+
+from sqlalchemy.orm import relationship, backref, object_mapper
+from sqlalchemy import Column, Integer, BigInteger, String, schema
+from sqlalchemy import ForeignKey, DateTime, Boolean, Text, Float, Index
+from sqlalchemy.exc import IntegrityError
+from sqlalchemy.ext.declarative import declarative_base
+from sqlalchemy.schema import ForeignKeyConstraint
+
+from nova.db.sqlalchemy import models
+
+
+BASE = declarative_base()
+
+
+class BareMetalNode(BASE, models.NovaBase):
+    """Represents a bare metal node."""
+
+    __tablename__ = 'bm_nodes'
+    id = Column(Integer, primary_key=True)
+    service_host = Column(String(255))
+    instance_uuid = Column(String(36), nullable=True)
+    cpus = Column(Integer)
+    memory_mb = Column(Integer)
+    local_gb = Column(Integer)
+    pm_address = Column(Text)
+    pm_user = Column(Text)
+    pm_password = Column(Text)
+    prov_mac_address = Column(Text)
+    registration_status = Column(String(16))
+    task_state = Column(String(255))
+    prov_vlan_id = Column(Integer)
+    terminal_port = Column(Integer)
+
+
+class BareMetalPxeIp(BASE, models.NovaBase):
+    __tablename__ = 'bm_pxe_ips'
+    id = Column(Integer, primary_key=True)
+    address = Column(String(255), unique=True)
+    server_address = Column(String(255), unique=True)
+    bm_node_id = Column(Integer, ForeignKey('bm_nodes.id'), nullable=True)
+
+
+class BareMetalInterface(BASE, models.NovaBase):
+    __tablename__ = 'bm_interfaces'
+    id = Column(Integer, primary_key=True)
+    bm_node_id = Column(Integer, ForeignKey('bm_nodes.id'), nullable=True)
+    address = Column(String(255), unique=True)
+    datapath_id = Column(String(255))
+    port_no = Column(Integer)
+    vif_uuid = Column(String(36), unique=True)
+
+
+class BareMetalDeployment(BASE, models.NovaBase):
+    __tablename__ = 'bm_deployments'
+    id = Column(Integer, primary_key=True)
+    key = Column(String(255))
+    image_path = Column(String(255))
+    pxe_config_path = Column(String(255))
+    root_mb = Column(Integer)
+    swap_mb = Column(Integer)
diff --git a/nova/virt/baremetal/db/sqlalchemy/session.py b/nova/virt/baremetal/db/sqlalchemy/session.py
new file mode 100644
index 000000000..2cae17f18
--- /dev/null
+++ b/nova/virt/baremetal/db/sqlalchemy/session.py
@@ -0,0 +1,58 @@
+# vim: tabstop=4 shiftwidth=4 softtabstop=4
+
+# Copyright (c) 2012 NTT DOCOMO, INC.
+# Copyright 2010 United States Government as represented by the
+# Administrator of the National Aeronautics and Space Administration.
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""Session Handling for SQLAlchemy backend."""
+
+from nova import config
+from nova.db.sqlalchemy import session as nova_session
+from nova.openstack.common import cfg
+
+opts = [
+    cfg.StrOpt('baremetal_sql_connection',
+               default='sqlite:///$state_path/baremetal_$sqlite_db',
+               help='The SQLAlchemy connection string used to connect to the '
+                    'bare-metal database'),
+    ]
+
+CONF = config.CONF
+CONF.register_opts(opts)
+
+_ENGINE = None
+_MAKER = None
+
+
+def get_session(autocommit=True, expire_on_commit=False):
+    """Return a SQLAlchemy session."""
+    global _MAKER
+
+    if _MAKER is None:
+        engine = get_engine()
+        _MAKER = nova_session.get_maker(engine, autocommit, expire_on_commit)
+
+    session = _MAKER()
+    session = nova_session.wrap_session(session)
+    return session
+
+
+def get_engine():
+    """Return a SQLAlchemy engine."""
+    global _ENGINE
+    if _ENGINE is None:
+        _ENGINE = nova_session.create_engine(CONF.baremetal_sql_connection)
+    return _ENGINE
diff --git a/nova/virt/baremetal/doc/README.rst b/nova/virt/baremetal/doc/README.rst
new file mode 100644
index 000000000..6d5cfd466
--- /dev/null
+++ b/nova/virt/baremetal/doc/README.rst
@@ -0,0 +1,69 @@
+General Bare-metal Provisioning README
+======================================
+
+:Authors:
+  [USC/ISI] Mikyung Kang <mkkang@isi.edu>, David Kang <dkang@isi.edu>
+
+  [NTT DOCOMO] Ken Igarashi <igarashik@nttdocomo.co.jp>
+
+  [VirtualTech Japan Inc.] Arata Notsu <notsu@virtualtech.jp>
+:Date:   2012-08-02
+:Version: 2012.8
+:Wiki: http://wiki.openstack.org/GeneralBareMetalProvisioningFramework
+
+Code changes
+------------
+
+::
+
+  nova/nova/virt/baremetal/*
+  nova/nova/virt/driver.py
+  nova/nova/tests/baremetal/*
+  nova/nova/tests/compute/test_compute.py
+  nova/nova/compute/manager.py
+  nova/nova/compute/resource_tracker.py
+  nova/nova/manager.py
+  nova/nova/scheduler/driver.py
+  nova/nova/scheduler/filter_scheduler.py
+  nova/nova/scheduler/host_manager.py
+  nova/nova/scheduler/baremetal_host_manager.py
+  nova/bin/bm_deploy_server
+  nova/bin/nova-bm-manage
+
+Additional setting for bare-metal provisioning [nova.conf]
+----------------------------------------------------------
+
+::
+
+  # baremetal database connection
+  baremetal_sql_connection = mysql://$ID:$Password@$IP/nova_bm
+
+  # baremetal compute driver
+  compute_driver = nova.virt.baremetal.driver.BareMetalDriver
+  baremetal_driver = {nova.virt.baremetal.tilera.TILERA | nova.virt.baremetal.pxe.PXE}
+  power_manager = {nova.virt.baremetal.tilera_pdu.Pdu | nova.virt.baremetal.ipmi.Ipmi}
+
+  # instance_type_extra_specs this baremetal compute
+  instanse_type_extra_specs = cpu_arch:{tilepro64 | x86_64 | arm}
+
+  # TFTP root
+  baremetal_tftp_root = /tftpboot
+
+  # baremetal scheduler host manager
+  scheduler_host_manager = nova.scheduler.baremetal_host_manager.BaremetalHostManager
+
+
+Non-PXE (Tilera) Bare-metal Provisioning
+----------------------------------------
+
+1. tilera-bm-instance-creation.rst
+
+2. tilera-bm-installation.rst
+
+PXE Bare-metal Provisioning
+---------------------------
+
+1. pxe-bm-instance-creation.rst
+
+2. pxe-bm-installation.rst
+