Open source technologies used with &PRODUCT;KoanBackground
Koan is a tool coming out of Red Hat Emerging
Technologies that is used to provision machines from
Cobbler servers. Following the unix philosophy it's
very simple to use and the man page will tell you
everything you need to know. For more information check
out the cobbler
documentation.
Note
Most provisioning with the &PRODUCT; tools
can be done without having to work with Koan
directly. However, a good understanding of its
basic operation is useful for advanced usage of
the &PRODUCT; tooling.
Installation RPMs exist for both Fedora and RHEL (through EPEL).
If your repositories are configured correctly you should simply
be able to yum install koan. Koan doesn't have many
dependencies so if you don't feel like adding the EPEL repo to
your RHEL machine you can simply install the RPM.
Once installed you should test your installation against a
cobbler server.
koan -s genome-repo.usersys.redhat.com --list=profiles
koan -s genome-repo.usersys.redhat.com --list=systems
Guest ProvisioningNotegenome-bootstrap
now wraps Koan for provisioning virtual machines.
This is only included for advanced use cases.
koan -s genome-repo.usersys.redhat.com --virt --virt-type=xenpv --virt-path=HostVolGroup00 --system=[your hostname]
Here the most important part is obviously the
--virt flag. If you pass in a
Volume Group name for --virt-path
koan will automatically create (or reuse) a logical
volume in the format of
[name]-disk0. With cobbler much of the
configuration lies on the server side (the memory, size
of the logical volume, etc). If you have different
requirements you can either create a new profile for
cobbler or you can use the tooling that
makes up &PRODUCT; achieve the desired results.
Tip
One trick to creating a quest with a larger logical
volume than a cobbler profile specifies is to simply
create it by hand and specify the size you desire. Koan
will simply reuse that logical volume.
Watching the VM
During the kickstart provisioning process you can
connect to the virtual framebuffer which is accessible
through VNC. It's only available locally so don't try
and connect from another machine. From the Xen host you
should be able to use:
ssh -X root@YourXenHost.usersys.redhat.com
vncviewer localhost:5900
The port may vary according to how many guests you have
running. To find out which ports are being used:
# If you are using RHEL5 less than U2
netstat -nlp | grep vnc
#otherwise
netstat -nlp | grep qemu-dm
Cleaning Up
If you would like to remove work performed by koan:
Remove the Xen configuration
for the guest under
/etc/xen
Remove the file or logical
volume that backs your guest.
Known Issues
Provisioning will fail if a config file under
/etc/xen has the same name as the
machine you are trying to create. The error message is
fairly cryptic and says something like "machine already
exists". The fix is to simply remove the config file.
LVM
LVM is used to back our virtualized guests. It is an
extremely flexible and pervasive storage technology for
Linux. One of the most useful features is the ability
to create copy-on-write snapshots.
RHEL 5 Documentation about Virtualization
Make sure you are comfortable with what
LVM is and how to create / remove some
simple logical volumes.
Self Test
Xen Virtualization
Virtualization is a key component of the new
architecture. Managing development, build and
deployment environments on a variety of hardware and
operating systems has always been extremely costly. We
have a fairly low tolerance for inconsistencies in both
environments, yet customization is critical to most
developers and avoided at all costs in production.
Virtualization is the technology that gives &PRODUCT;
isolation in both worlds. The development, build, and
deployment environments can all be isolated and managed
on virtual machines to enable different configurations
and optimizations while still residing on the same
machine. It also gives us the flexibility to modify our
virtualization option as time progress (e.g. from Xen
to KVM) but keep the core strategy consistent for the
foreseable future.
RHEL 5 Documentation about Virtualization
Make sure you are comfortable with xen
and how to create virtual machines,
start them, stop them, and customize
them.
Self Test
JBoss
JBoss is going to be a cornerstone of our new
infrastructure. We will be using a slightly newer
version of the JBoss EAP stack with components from the
JBoss SOA team to incorporate the JBoss ESB.
JBoss Getting Started Guide
Make sure you are comfortable with
starting and stopping JBoss as well as
the server configurations, deployment
mechanisms, jmx console, and general
filesystem layout to find logs.
JBoss ESB Documentation
These documents aren't incredibly
thorough at the moment, but it should
give you a good initial understanding
of the JBoss ESB technologies.
JBoss Seam Documentation
The reference documentation for Seam
2.0 CR3 will probably be the best
document to read through.
Source Code Management (Git)
In addition to refining our infrastructure, we also
have needed to refine our development and deployment
practices for quite a while. We need to be able to run
multiple development efforts in parallel, collaborate
between them, and maintain a sane state of a deployable
branch (e.g. trunk). Subversion has worked in some
regards but has fallen short in our ability to utilize
it for multiple development streams. Complicated
merges end up very error prone and have almost always
resulted in production defects. Also, given the errors
around branching and merging, it has been very
difficult to get the development community to maintain
a clean revision history and state of our production
branch. Git's distributed nature will allow development
to proceed in an offline fashion and result with a
small number of clean patches being applied to our
production branches. So in essence, be warned, stream
of consciousness coding and commits will no longer be
accepted.
General GIT Tutorial
Understand how to create repositories,
add files, commit, and view logs
Everyday GIT with 20 commands
Read through the entire document but be
very comfortable with the Individual
Developer and Integrator sections
An introduction to git-svn for
Subversion/SVK users and
deserters
Self Test
Configuration Management (Puppet)
Puppet is a configuration management technology that
will help us eliminate many of the manual steps
required during releases. Today, the configuration and
release process is extremely manual and becoming
increasingly difficult to scale. Moving the
configuration management aspects down to development
allows developers to drive more automation into the
release process by providing container and system
configurations using a mechanism that can be deployed
without modification into production. This also allows
groups like Release Engineering to operate in more of a
review role and reduce the manual steps they are
required to deploy projects.
Puppet Documentation
Since your virtual environment will be
running a puppet master to configure
all of your virtual machines, make sure
you have an understanding of what the
puppet master does as well as the
templating process used to generate
files. This knowledge will be key in
enabling developers to make system
configuration changes, testing and
submitting patches instead of making
manual requests for various changes to
be applied.
General
Two places that you should always look for
documentation are:
Red Hat Enterprise Linux Documentation
JBoss Documentation