path: root/genome-docs/genome-docs-1.0.0/en-US/Cookbook.xml

<?xml version='1.0'?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
]>

<chapter id="genome-Cookbook">
	<title>Cookbook</title>
	<para>
		This chapter contains many useful &PRODUCT; recipes.  While a solid understanding of the 
		toolset is always preferred this chapter is meant to be the user's reference manual.
	</para>

	<section id="genome-hypervisor-setup">
		<title>Setting up an environment to host virtual machines</title>
		<para>
			Virtualization is by no means a requirement to make use of
			the &PRODUCT; tooling, though it is the more common than
			"bare metal" provisioning.
		</para>

		<para>
			The machine used to host virtual machines is called the
			Cloud Appliance.  As the name suggests, there can be
			one-to-many physical machines.  The first
			goal of this machine is to provide and environment to
			host virtual machines and for that reason are
			<emphasis>always</emphasis> provisioned on "bare
			metal".  The second is to provide an effective way to
			manage resources amongst underutilized commodity
			hardware. 
		</para>

		<para>
			Since virtualization plays such a key role in the
			&PRODUCT; environment these machines amongst the first
			that users of the &PRODUCT; tooling desire to get up
			and running quickly.
		</para>

		<note>
			<title>Note</title>
			<para>
				One of the main goals of the current
				Cloud machine tooling is to
				<emphasis>Do the simplest thing that
					could possibly work</emphasis>.
				This functionality, implemented through
				<ulink
					url="https://fedorahosted.org/func">Func</ulink>
				modules, will most likely be entirely
				replaced with <ulink
					url="http://ovirt.org/">ovirt</ulink>.
			</para>
		</note>

		<section id="cloud-appliance-setup">
			<title>Using genome-replace-self</title>
			<para>
				The first step is to install the genome-replace-self RPM.  You can
				do this by running one of the following commands:
				<screen>
rpm -Uvh --force http://ftp.redhat.com/pub/redhat/genome/yum/Fedora-9-genome-noarch/genome-replace-self-1.0.0-2.fc9.noarch.rpm
				</screen>

				If you want to install the RPM from your local Genome server,
				the format would be:
				<screen>
rpm -Uvh --force http://$GENOME_MACHINE/cobbler/repo_mirror/Fedora-9-genome-noarch/genome-replace-self-1.0.0-2.fc9.noarch.rpm
				</screen>
			</para>
			<para>
				Cloud Appliances use <link
					linkend="genome-replace-self">genome-replace-self</link>
				to get up and running quickly.  The key to
				using
				<application>genome-replace-self</application>
				to provision a Cloud Appliance is to:
				<itemizedlist>
					<listitem>
						<para>
							Use a <emphasis>Cloud</emphasis> profile.
						</para>
					</listitem>
					<listitem>
						<para>
							Set the
							<emphasis>-m</emphasis>
							(metadata) flag
							appropriately.
						</para>

						<para>
							The value for this flag varies depending if it is
							the <emphasis>certmaster</emphasis> or 
							a <emphasis>minion</emphasis> in
							<application>func</application> parlance.  
						</para>

						<para>
							For the master set <emphasis>-m</emphasis> to
							<emphasis>certmaster=localhost</emphasis>
						</para>

						<para>
							For the minion set
							<emphasis>-m</emphasis>
							to
							<emphasis>certmaster=[Any
								previously
								created Cloud
								master]</emphasis>
						</para>
					</listitem>
				</itemizedlist>
			</para>
		</section>
	</section>

	<section id="genome-RepoMachineBootstrapping">
		<title>Creating your own &PRODUCT; Repo Appliance</title>
		<para>
			The Repo Appliance provisioning story is fairly straight forward:
			<itemizedlist>
				<listitem>
					<para>install the &REPORPM; (either via kickstart or manually)</para>
				</listitem>
				<listitem>
					<para>Replicate
						<application>cobbler</application>
						to pull down the bits and
						kickstarts</para>
				</listitem>
				<listitem>
					<para>Syncronize the
						<application>git</application>
						repositories from
						another Repo Appliance.</para>
				</listitem>
				<listitem>
					<para>
						Define access controls
					</para>
				</listitem>
			</itemizedlist>
		</para>

	        <section>
			<title>Installing the &REPORPM;</title>
			<para>
				There are several options on how to go about
				this step.  If another Repo Appliance is already
				available
				<application>genome-replace-self</application>
				or <application>koan</application> based
				options are generally preferred since they require
				fewer manual steps.
			</para>

			<section>
				<title>Creating a virtual Repo Appliance with koan</title>

				<screen>
# See what profiles are available
koan -s [remote Repo Appliance] --list=profiles

# If choosing a virtual machine
koan -s [remote Repo Appliance] --virt --virt-name=[Any name]  --profile=[Profile from first step] --virt-path=[Usually a Volume Group or path to a file]
				</screen>
			</section>
				
			<section>
				<title>Creating a baremetal Repo Appliance with genome-replace-self</title>

				<para>
					The only step needed for this is to use <link
						linkend="genome-replace-self">genome-replace-self</link>
					and pass in a Repo profile.
				</para>
			</section>
				
			<section>
				<title>Creating a Repo Appliance out of a machine that already has an Operating System</title>
				<para>
					If installing this on a
					machine that already has an operating system (OS)
					installed simply <userinput>yum install genome-repo</userinput>.
				</para>

				<para>
					In case the &REPORPM; is not available in the
					stock <application>yum</application>
					repositories for your OS you can grab the RPMs
					from any other &PRODUCT; Repo Appliance at
					http://[hostname]/cobbler/repo_mirror/[distro]-genome-noarch.
				</para>

				<para>
					Whenever installing the &REPORPM;
					manually there is one other step that
					needs to happen to properly configure a
					Repo Appliance. 
					<screen>
/sbin/service genome-repo-bootstrap start
					</screen>
				</para>
				<note>
					<title>Note</title>
					<para>
						This step is designed to work
						"offline" since all the
						external dependencies are
						pulled down via the &REPORPM;.
						That being said there are
						external factors than can cause
						it to fail.  If you suspect a
						Repo Appliance is not configured
						correctly it is always safe to
						run this command multiple times
						as
						<application>puppet</application>
						will only perform outstanding
						tasks.
					</para>
				</note>
			</section>
	        </section>

	        <section>
			<title>Replicate cobbler data</title>
			<para>
				The simplest way to do this is to use
				<application>cobbler</application>'s replicate
				functionality.  
				<screen>
cobbler replicate --master=[remote Genome Repo machine] --full-data-sync
				</screen>
				For more advanced usage see the
				<application>cobbler</application> manpage.

				<important>
					<title>Important</title>
					<para>
						Currently the use of
						<userinput>cobbler
							replicate</userinput>
						requires the user to know the
						root password on the master
						cobbler server (the remote Repo
						machine).  This is less than
						desirable but for now the
						default password is
						<emphasis>&GENOMEPASSWORD;</emphasis>.
					</para>
				</important>

				<note>
					<title>Note</title>
					<para>
						Technically it is also possible
						to configure
						<application>cobbler</application>
						by hand.  This obviously
						requires a more in depth
						understanding of
						<application>cobbler</application>
						and is outside the scope of
						this document.
					</para>
				</note>
			</para>
	        </section>

	        <section>
			<title>Syncronize the git repositories</title>
			<para>
				 Since this is a newly provisioned Repo Appliance
				 the fastest way to get up and running is to
				 run:
				
				 <screen>
# Switch to the genome user
su - genome

# "reset --hard" to all the git repositories on a remote Repo
genome-sync start quick --repo=[remote Repo Appliance]

# Push all the repos where they need to go on the new Repo Appliance
genome-sync save

# Restart the puppetmaster (see below) and re-run puppet
su -
service puppetmaster restart
puppetd --test
				 </screen>
			</para>

			<important>
				<title>Important</title>
				<para>
					If any <emphasis>new</emphasis> puppet
					modules are saved the
					<application>puppetmaster</application>
					will have to be restarted for them to be
					available. 
				</para>
			</important>
	        </section>

	        <section>
			<title>Define access controls</title>
			<para>
				All machines in the &PRODUCT; environment have
				a default root password set as detailed in
				<link linkend="genome-LoggingIn">the
					appendix</link>.  By default Repo
				Appliances have a local user named &GENOMEUSER;
				who owns all content under
				<filename>/pub/git</filename>.  Out of the box
				there is no password set and users of Repo
				Appliances should feel free to use whatever
				method of authentication they choose.
			</para>

			<para>
				Typical methods of authentication are:

				<itemizedlist>
					<listitem>
						<para>
							Setting a password
						</para>
					</listitem>
					<listitem>
						<para>
							Using SSH public key authentication
						</para>
					</listitem>
				</itemizedlist>
			</para>

			<note>
				<title>Note</title>
				<para>
					There is a good chance &PRODUCT; will
					make use of <ulink
						url="http://freeipa.org">FreeIPA</ulink>
					someday.
				</para>
			</note>
	        </section>
	</section>

	<section> 
		<title>Cleaning up SSL certificates</title>
		<para>
			Due to the volatile nature of &PRODUCT; machines there
			occasionally comes a need to clean up SSL certificates.
			To clean up all Puppet certs you can simply stop the
			<application>puppetmaster</application> (in the case of
			a Repo Application) and <application>puppetd</application>
			services and then remove
			<filename>/var/lib/puppet/ssl</filename>.  When you
			start the services back up the certificates will be
			created anew.
		</para>

		<para>
			Sometimes the Puppetmaster will have a cert that
			corresponds to a machine previously provisioned with
			the same hostname.  Our bootstrap process cleans this
			up automatically but it's not hard to get into a state
			where it will need to be cleaned manually on the
			Puppetmaster side.  Luckily this is easy to do.  The
			error from Puppet even hints at how to do it.  Login to
			your Repo Appliance as the local user (usually
			<emphasis>genome</emphasis>) and run <userinput>sudo
				/usr/sbin/puppetca --clean [your
				hostname]</userinput>
		</para>

		<important>
			<title>Important</title>
			<para>
				<application>sudo</application> access to
				<application>puppetca</application> has been
				given to the local &PRODUCT; user.
			</para>
		</important>
	</section>

	<section id="genome-NoKoanBootstrap">
		<title>Bootstrapping a machine that already has an OS</title>
		<important>
			<title>Important</title>
			<para>
				This particular recipe should be
				considered advanced.  There are plenty of ways to make a custom system
				incompatible with the &PRODUCT; tooling.  Be sure you are comfortable with how
				&FIRSTBOOTFILE; works.
			</para>
		</important>
		<para>
			In some cases you might not want to reinstall
			the OS on a system in the &PRODUCT;
			environment.  This happens mostly for laptops
			where a developer simply wants a build
			environment so they can work "off the grid".  
		</para>

		<para>
			For this recipe the user simply needs to pass
			the <emphasis>--config-only</emphasis> option
			to <application>genome-bootstrap</application>
			to create the needed server-side configuration.
			Once that is done it's just a matter of
			installing the
			<application>genome-firstboot</application>
			RPM and editing
			<filename>/etc/sysconfig/genome-firstboot</filename>.
			Here's an example config file:
			<screen>
				RUN_BOOTSTRAP=YES
				export GENOME_REPO=genome-staging-repo.usersys.redhat.com
				export FQDN=arch-repo.usersys.redhat.com
			</screen>
		</para>
		<note>
			<title>Note</title>
			<para>
				It's always best to grab the RPMs from
				the &PRODUCT; Repo you are going to
				use.  That way you can be certain you
				are using a compatible version.
			</para>
		</note>
	</section>


	<section id="genome-AddMachineType">
		<title>Adding a new machine type</title>
		<para>
			There are several tools and config files that need to
			know what types of machines are available for
			provisioning on a particular <link
				linkend="genome-appliance">Genome Appliance
				machine</link>.  To simplify this process
			&PRODUCT; includes a DSL (Domain Specific Language) for
			describing the machines available.
		</para>

		<para>
			Let's start with an example for creating a new developer
			workstation machine type.  It's important to get all the
			wiring setup to make puppet and your machine aware of the
			new module before adding too much function.
		</para>

		<para>
			The first step is usually to create a stubbed out module
			on the &PRODUCT; Appliance and use <link linkend="genome-sync">genome-sync</link>
			to setup the right information.  To do this, run these steps (on
			the &PRODUCT; Appliance):
			<screen>
# Switch to the genome user
su - genome

# Create a working directory (the 'puppet' directory is important)
mkdir -p /tmp/working/puppet/dev_workstation/manifests
cd /tmp/working/puppet/dev_workstation

# Create a simple puppet configuration
# The name of the class must match the directory name
echo """
class dev_workstation {
  package { 'vim-enhanced':
    ensure => latest;
  }
}
""" > manifests/init.pp

# Make the directory a git repository
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
git init
git add .
git commit -m "Priming the dev_workstation module"

# Run genome-sync to publish this module
# This sets up the 'public' git repositories
# and the puppet working directory
genome-sync save --workingdir=/tmp/working

# This is probably a bug in puppet, but you
# need to restart the puppetmaster when you introduce
# a new module
su -
service puppetmaster restart
exit
			</screen>
		</para>
		<para>
			At this point, you should see the git repositories listed
			on your &PRODUCT; Appliance at http://$GENOME/git/gitweb.cgi
		</para>
		<para>
			The next step is to update the &PRODUCT; DSL file to make
			it aware of a new 'machine' that contains this new puppet
			class.  Machines can consist of multiple puppet classes,
			but in this case, we are starting simple.  To do this, run 
			these steps (on the &PRODUCT; Appliance):
		</para>
		<screen>
# Make sure the permissions are okay on the machine_types.rb file
su -
chown genome:genome /etc/genome/machine_types.rb
chown -R genome:genome /etc/puppet/modules/main
exit

# Switch to the genome user to update the machine_types.rb file
su - genome

# Clone the repo_extensions repository
git clone /pub/git/puppet/repo_extensions /tmp/repo_extensions
cd /tmp/repo_extensions

# Add the new machine consisting of your new puppet module
echo """
newmachine('developer-workstation') do
   set_classes 'dev_workstation'
end
""" >> files/machine_types.rb

# Commit and push the change
git add .
git commit -m "Adding the new developer-workstation machine"
git push
		</screen>
		<para>
			After you have pushed your changes, they will be
			applied the next time puppet runs on the repo, which
			is usually in 5 to 10 minutes.  However, for the sake
			of this example, we will manually kick off a puppet
			run:
		</para>
		<screen>
# Force a puppet run
su -
puppetd --test
		</screen>
		<para>
			At this point, you should see your new machine type listed
                        on your &PRODUCT; Appliance at http://$GENOME/genome/machine_types.html
		</para>
		<para>
			<link linkend="genome-sync">genome-sync</link> is a
			great way to handle moving custom machine types from
			one &PRODUCT; Appliance to another.
		</para>
	</section>

	<section id="genome-ChangingAMachineParameter">
		<title>Changing a machine's parameters</title>
		<para>
			It's convenient to be able to change bootstrapped
			parameters. One typical use case is if you would like to
			point your machine to a different puppet master. 
		</para>

		<para>
			To do this simplify access the
			<application>genomed</application> running on the
			&PRODUCT; Repo that your machine is subscribed to and
			update the yaml file.  Once the yaml has been updated
			you probably want to manually run Puppet
			(<userinput>puppetd --test</userinput>) to update the
			machine's configuration.
		</para>
	</section>

	<section id="genome-PuppetRebasing">
		<title>Change the puppet master of any given machine type</title>
		<para>
			<orderedlist numeration="arabic">
				<listitem>
					<para>
						Browse to
						http://[repo]/genome/nodes/[hostname].html
						and update the yaml file.
					</para>
				</listitem>
				<listitem>
					<para>
						Edit the &PUPPETCONFFILE; and
						change any instances of the
						current puppet master hostname
						to the hostname of the new
						puppet master
					</para>
				</listitem>
				<listitem>
					<para>
Make sure there exists a machine configuration for this host on the other repo
machine.  If not simply create a new one using
<application>genomed</application> or by using <userinput>genome-bootstrap
--config-only</userinput>.
					</para>
				</listitem>
				<listitem>
					<para>
						Remove the &PUPPET_SSL_DIR;
						directory, this will be
						recreated with a valid set of
						certificates and keys
					</para>
				</listitem>
				<listitem>
					<para>
						Run <command>puppetd --test</command> to make sure
						everything worked. If all goes well,
						start the puppet client and you're
						done.
					</para>
				</listitem>
			</orderedlist>
		</para>
	</section>

	<section id="genome-GitRecipes">
		<title>Git Recipes</title>
		<para>
			This recipe involves a situation where you have 4
			commits, with 1,2, and 4 being related and commit 3
			just in the middle. Ideally, before you submit this,
			you would like to combine 1,2, and 4 into a single
			commit and then have commit 3 and a second, isolated
			commit.
		</para>
		<para>
			Run the following to prime you repository with this
			scenario:
		</para>
		<screen>
cd ~
mkdir git-test
cd git-test
git init
touch a.txt
git add a.txt
git-commit -m "Adding one file"
touch b.txt
git add b.txt
git-commit -m "Adding another file"
touch different.txt
git add different.txt
git-commit -m "Commiting something entirely different"
touch c.txt
git add c.txt
git-commit -m "Adding yet another file"
		</screen>

		<para>
			Correcting the problem:
			<orderedlist numeration="arabic">
				<listitem>
					<para>
						Look at the log history and get
						the id of the initial commit
						revision:
						<userinput>git log</userinput>
					</para>
				</listitem>
				<listitem>
					<para>
						Start a working branch and check it out
						to serve as the cleanup branch,
						starting at the initial commit:
						<userinput>
							git checkout -b cleanup <varname>[REVISION]</varname>
						</userinput>
					</para>
				</listitem>
				<listitem>
					<para>
						Confirm you are now working on the new branch:
						<userinput>git status</userinput>
					</para>
				</listitem>
				<listitem>
					<para>
						Confirm the log history only contains the initial commit:
						<userinput>git log</userinput>
					</para>
				</listitem>
				<listitem>
					<para>
						Now, since we want to add two
						more things to this commit and
						re-commit it. First, you need
						to get the commit revisions
						from the master branch:	
						<userinput>git log master</userinput>
					</para>
				</listitem>
				<listitem>
					<para>
						Now, you need to cherry pick
						commits 2 and 4 without
						commiting:
						<screen>
git-cherry-pick -n <varname>[REVISION 2]</varname>
git-cherry-pick -n <varname>[REVISION 4]</varname>
						</screen>
					</para>
				</listitem>
				<listitem>
					<para>
						Now you need to revise the current
						commit comment to include what was done
						in revisions 2 and 4
						<userinput>git-commit --revise</userinput>
					</para>
				</listitem>
				<listitem>
					<para>
						At this point, you have combined 1,2,
						and 4 into a single commit. The last
						step is to cherry pick and commit
						revision 3:
						<userinput>git-cherry-pick <varname>[REVISION 3]</varname></userinput>
					</para>
				</listitem>
				<listitem>
					<para>
						Now you can push the changes from this
						branch to the desired public
						repository.
					</para>
				</listitem>
			</orderedlist>
		</para>
	</section>
    </chapter>