path: root/files/SevoneOSPprereqs_MOPV_1.9.txt

===========================
Sevone Manual Configuration
===========================

These are the post installation steps to configure an OpenStack deployment for Sevone.  This includes creating the Sevone linux accounts, readonly role and policy, ssh keys, sudoers, SNMP, and logging configurations.  All steps will be run as the stack user on the OSP director box with the overcloudrc sourced.

=============
User creation
=============

In this section you will create the Sevone linux user on the controller nodes and the OSP director.
Create the Sevone linux user on the controller nodes


[stack@ospdirector ~]$ source /home/stack/overcloudrc
[stack@ospdirector ~]$ for i in $(nova host-list | awk '/consoleauth/ {split($2,a,"."); print a[1]}'); do \
ssh heat-admin@$i sudo groupadd --gid 6005 sevone; done
[stack@ospdirector ~]$ for i in $(nova host-list | awk '/consoleauth/ {split($2,a,"."); print a[1]}'); do \
ssh heat-admin@$i sudo useradd --uid 6005 --gid sevone sevone; done
[stack@ospdirector ~]$ for i in $(nova host-list | awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; do \
echo -n "$i : "; ssh heat-admin@$i id sevone; done

Create the Sevone linux user on the OSP director


[stack@ospdirector ~]$ sudo groupadd --gid 6005 sevone
[stack@ospdirector ~]$ sudo useradd --uid 6005 --gid sevone sevone
[stack@ospdirector ~]$ id sevone

Openstack role creation

In this section you will create the readonly role for the overcloud

[stack@ospdirector ~]$ openstack role create readonly

=============
Policy Upload
=============

In this section you will create the policy directory structure under the stack user, create the policy.json 
files for the OpenStack services, and upload them to the controller nodes.  You should have been supplied a 
policydir.tgz file with this documentation.  This file should be placed in the stack’s home directory.

	I. Untar policydir_osp10.tgz

[stack@ospdirector ~]$ tar -xzvf /home/stack/policydir_osp10.tgz

	II. Upload the policy.json files to the controllers

[stack@ospdirector ~]$ for x in $(find ./policydir -name "*.json"); do echo $x ; cat $x | json_verify ; done
[stack@ospdirector ~]$ ./policydir/files/push_sevone_policies_to_overcloud.sh 

	III. Restart services; ensure all cluster managed services are up before continuing


[stack@ospdirector ~]$ ssh heat-admin@$(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}' | \
head -n 1) sudo pcs resource restart haproxy-clone

	Be patient as the prompt will not return until all services have successfully restarted.  If you 
want to monitor the process - run the below in a separate console.

[stack@ospdirector ~]$ watch "ssh heat-admin@$(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}' | \
head -n 1) sudo pcs status"


************************ RUN ONLY IF PROBLEMS OCCUR **********************************

If controller services do not fully restart, or there is a suspicion that something may be wrong with the policy files, revert policy files to the previous state.

[stack@ospdirector ~]$ for x in $(find ./policydir -name "*.json"); do echo $x ; cat $x | json_verify ; done
[stack@ospdirector ~]$ ./policydir/files/restore_default_OSP_policies_on_overcloud.sh

[stack@ospdirector ~]$ ssh heat-admin@$(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}' | \
head -n 1) sudo pcs resource restart haproxy-clone

**************************************************************************************

==============
Setup ssh keys
==============

In this section you will distribute the ssh public keys to the Sevone accounts on the controllers and OSP director.  If an ssh key is not provided by your team, see instructions in Appendix B for generating one.

	I. Create the public key file.  The utility ssh-keygen can be used to generate a new key or you can provide your own.

[stack@ospdirector ~]$ cat << EOF > ~/id_rsa.pub
ssh-rsa
<Insert the content of your generated key here> root@SevOne
EOF

	II. Distribute the ssh keys to the controllers

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; ssh heat-admin@$i sudo mkdir /home/sevone/.ssh; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; ssh heat-admin@$i sudo chown sevone:sevone /home/sevone/.ssh; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; ssh heat-admin@$i sudo chmod 700 /home/sevone/.ssh; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; cat id_rsa.pub | \
ssh heat-admin@$i "sudo sh -c 'cat >> /home/sevone/.ssh/authorized_keys'" ; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; \
ssh heat-admin@$i sudo chown sevone:sevone /home/sevone/.ssh/authorized_keys ; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; \
ssh heat-admin@$i sudo chmod 600 /home/sevone/.ssh/authorized_keys ; done


	III. Distribute the ssh keys to the OSP director

[stack@ospdirector ~]$ sudo mkdir /home/sevone/.ssh
[stack@ospdirector ~]$ sudo chown sevone:sevone /home/sevone/.ssh
[stack@ospdirector ~]$ sudo chmod 700 /home/sevone/.ssh
[stack@ospdirector ~]$ sudo cp id_rsa.pub /home/sevone/.ssh/authorized_keys
[stack@ospdirector ~]$ sudo chown sevone:sevone /home/sevone/.ssh/authorized_keys
[stack@ospdirector ~]$ sudo chmod 600 /home/sevone/.ssh/authorized_keys

==================
Setup sudoers file
==================

In this section you will distribute the Sevone sudoers file to the controllers and the OSP director and set 
up the Sevone nova script.

	I. Distribute the sevone file to the controllers

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; cat ~/policydir/files/sevone.sudoers | \
ssh heat-admin@$i "sudo sh -c 'cat > /etc/sudoers.d/sevone'" ; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; ssh heat-admin@${i} sudo -l -U sevone ; echo -e "\n\n" ; done

	II. Ensure the nova script for Sevone is in place with the correct permissions.

[stack@ospdirector ~]$ sudo mkdir /opt/sevone 
[stack@ospdirector ~]$ sudo cp ~/policydir/files/nova_sevone.sh /opt/sevone/
[stack@ospdirector ~]$ sudo chown stack:stack /opt/sevone/nova_sevone.sh
[stack@ospdirector ~]$ sudo chmod 700 /opt/sevone/nova_sevone.sh

	III. Add the additional line to the sevone sudoers file so that the nova script can be called on 
the OSP director

[stack@ospdirector ~]$ sudo echo "sevone ALL=(stack) NOPASSWD:/opt/sevone/nova_sevone.sh" \
>> ~/policydir/files/sevone.sudoers

	IV. Distribute the sevone file to the OSP director

[stack@ospdirector ~]$ sudo cp ~/policydir/files/sevone.sudoers /etc/sudoers.d/sevone
[stack@ospdirector ~]$ sudo chmod 600 /etc/sudoers.d/sevone
[stack@ospdirector ~]$ sudo -l -U sevone

	V. Test that sevone user can perform the required commands. If sevone has no password, this step 
may first require logging in as root

[stack@ospdirector ~]$ sudo su -
[root@ospdirector ~]# su - sevone
[sevone@ospdirector ~]$ sudo -u stack /opt/sevone/nova_sevone.sh
[sevone@ospdirector ~]$ ls /tmp/sevone/
[sevone@ospdirector ~]$ exit
[root@ospdirector ~]# exit
[stack@ospdirector ~]$

	VI. Allow the sevone user to query mysql status

[root@slmsc2ctl0 ~]# mysql -e “create user 'sevone'@'localhost';

If during this step, you exit the current shell session instead of escalating to root, you will need to 
re-source overcloudrc before continuing.

==================
SNMP configuration
==================

In this section you will create the SNMP user for Sevone and distribute the SNMP configuration to 
OpenStack.  You will need to supply the IPV4 addresses for the trap target destinations and a password.  
The same password should be used throughout the OSP environment including the OSP director, all 
controllers, and all computes.  If a password is not provided by your team, see instructions in Appendix A 
for generating one.

	I. Create the SNMP user on the controllers


[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; ssh heat-admin@$i sudo systemctl stop snmpd ; \
ssh heat-admin@$i sudo net-snmp-create-v3-user -ro -A <ProvideSnmpPasswordHere> -a SHA \
-X <ProvideSnmpPasswordHere> -x AES sev1snmpuser ; \
echo "" ; done

	II. Create the SNMP user on the compute nodes


[stack@ospdirector ~]$ for i in $( nova hypervisor-list | \
awk '/localdomain/ {print $4}' | sed s/.localdomain//) ; \
do echo $i ; ssh heat-admin@$i sudo systemctl stop snmpd ; \
ssh heat-admin@$i sudo net-snmp-create-v3-user -ro -A <ProvideSnmpPasswordHere> -a SHA \
-X <ProvideSnmpPasswordHere> -x AES sev1snmpuser ; \
echo "" ; done

	III. Create the SNMP user on the OSP director


[stack@ospdirector ~]$ sudo systemctl stop snmpd
[stack@ospdirector ~]$ sudo net-snmp-create-v3-user -ro -A <ProvideSnmpPasswordHere> -a SHA \
-X <ProvideSnmpPasswordHere> -x AES sev1snmpuser

	IV. Prepare the SNMP configuration file

[stack@ospdirector ~]$ TRAPDEST1=<Enter First Destination IP>
[stack@ospdirector ~]$ TRAPDEST2=<Enter Second Destination IP>
[stack@ospdirector ~]$ echo -e "TRAPDEST1 = ${TRAPDEST1}\nTRAPDEST2 = ${TRAPDEST2}"
[stack@ospdirector ~]$ sed -i s/TRAPTARGET1/$TRAPDEST1/ ~/policydir/files/snmpd.conf
[stack@ospdirector ~]$ sed -i s/TRAPTARGET2/$TRAPDEST2/ ~/policydir/files/snmpd.conf

	V. Distribute the SNMP configuration file to the controllers, and restart the snmpd service.  Ensure the service is running before proceeding to the next step.


[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; cat ~/policydir/files/snmpd.conf | \
ssh heat-admin@$i "sudo sh -c 'cat > /etc/snmp/snmpd.conf'" ; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i ; ssh heat-admin@$i \
'sudo sed -i s/REPLACEENGINEID/$(hostname -s)/ /etc/snmp/snmpd.conf'; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do echo $i; ssh heat-admin@${i} sudo systemctl start snmpd ; done

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}') ; \
do  echo -n "$i : " ; ssh heat-admin@${i} sudo systemctl status snmpd | grep Active: ; done

	VI. Distribute the SNMP configuration file to the compute nodes

[stack@ospdirector ~]$ for i in $( nova hypervisor-list | \
awk '/localdomain/ {print $4}' | sed s/.localdomain//) ; \
do echo $i  ; cat ~/policydir/files/snmpd.conf | \
ssh heat-admin@$i "sudo sh -c 'cat > /etc/snmp/snmpd.conf'" ; done

[stack@ospdirector ~]$ for i in $( nova hypervisor-list | \
awk '/localdomain/ {print $4}' | sed s/.localdomain//) ; \
do echo $i ; ssh heat-admin@$i \
'sudo sed -i s/REPLACEENGINEID/$(hostname -s)/ /etc/snmp/snmpd.conf'; done

[stack@ospdirector ~]$ for i in $( nova hypervisor-list | \
awk '/localdomain/ {print $4}' | sed s/.localdomain//) ; \
do echo $i ; ssh heat-admin@$i sudo systemctl start snmpd ; done

	VII. It will take a few moments to restart snmpd on all compute nodes.  Ensure the snmpd Active status shows active (running) on 
each node before proceeding.

[stack@ospdirector ~]$ for i in $( nova hypervisor-list | \
awk '/localdomain/ {print $4}' | sed s/.localdomain//) ; \
do echo -n "$i : " ; ssh heat-admin@$i sudo systemctl status snmpd | grep Active: ; done

	VIII. Distribute the SNMP configuration file to the OSP director

[stack@ospdirector ~]$ sudo systemctl stop snmpd
[stack@ospdirector ~]$ sudo cp ~/policydir/files/snmpd.conf /etc/snmp/snmpd.conf
[stack@ospdirector ~]$ sudo sed -i s/REPLACEENGINEID/$(hostname -s)/ /etc/snmp/snmpd.conf
[stack@ospdirector ~]$ sudo chown root:root /etc/snmp/snmpd.conf
[stack@ospdirector ~]$ sudo chmod 644 /etc/snmp/snmpd.conf
[stack@ospdirector ~]$ sudo systemctl start snmpd
[stack@ospdirector ~]$ sudo systemctl status snmpd | grep 'Active:'

=================
Log configuration
=================

In this section you will configure logging in OpenStack.  You will need to provide the IPV4 addresses of the rsyslog target servers.

	I. Create the client configuration file

[stack@ospdirector ~]$ cat << EOF > ~/client.conf
*.* @SYSLOGTARGET1:PORTNUM
*.* @SYSLOGTARGET2:PORTNUM
EOF

[stack@ospdirector ~]$ SYSLOGDEST1=<Enter First Destination IP>
[stack@ospdirector ~]$ SYSLOGDEST2=<Enter Second Destination IP>
[stack@ospdirector ~]$ echo -e "SYSLOGDEST1 = ${SYSLOGDEST1}\nSYSLOGDEST2 = ${SYSLOGDEST2}"
[stack@ospdirector ~]$ sed -i s/SYSLOGTARGET1/$SYSLOGDEST1/ ~/client.conf
[stack@ospdirector ~]$ sed -i s/SYSLOGTARGET2/$SYSLOGDEST2/ ~/client.conf

	II. Configure logging for the overcloud and director

[stack@ospdirector ~]$ chmod +x ~/policydir/files/logging.sh
[stack@ospdirector ~]$ ~/policydir/files/logging.sh

========
Appendix
========

A.    Password Generation

	I. Creating a password (The length of the password can be adjusted via the variable in the ‘fold’ command):

	[stack@ospdirector ~]$ PASSWORD=`cat /dev/urandom | tr -dc ‘a-zA-Z0-9’ | fold -w 12 | head -n 1` 

	II. Assigning a password to a user

	[stack@ospdirector ~]$ sudo "echo $PASSWORD | passwd --stdin $USER"


B.    SSH Key Generation

	I. Creating an ssh key
		A. Become the sevone user

		[stack@ospdirector ~]$ sudo su - sevone

		B. Run ssh-keygen accepting the default file to save the key and empty passphrase

		[sevone@ospdirector ~]$ ssh-keygen
		Generating public/private rsa key pair.
		Enter file in which to save the key (/home/sevone/.ssh/id_rsa):
		Created directory '/home/sevone2/.ssh'.
		Enter passphrase (empty for no passphrase):
		Enter same passphrase again:
		Your identification has been saved in /home/sevone/.ssh/id_rsa.
		Your public key has been saved in /home/sevone/.ssh/id_rsa.pub.
		The key fingerprint is:
		bd:1c:24:3f:43:66:e8:ce:68:a7:06:52:8a:3f:ff:8a sevone2@slmsc2ospd.msc2.solk.lab.vzwnfv.com
		The key's randomart image is:
		+--[ RSA 2048]----+
		|                 |
		|         .       |
		|        o =      |
		|    .  . O       |
		| . o    S *      |
		|. o .  + . =     |
		| . . .o + o      |
		|  o. ..o         |
		|  Eoo+o          |
		+-----------------+


C.	 Monitoring MongoDB Usage

At times, it may be useful to inspect the database being used by Ceilometer when the service is having issues. Perform the steps below 
to query various database statistics

View database usage by running the du command
	
This command is useful if you only want to see the amount of space your Ceilometer database is taking up, and you're not concerned about 
any other statistics.

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {print $2}' | sed s/.localdomain//); \
do echo $i; ssh heat-admin@$i 'du -hs /var/lib/mongodb'; done 


D.	 Reducing Log Message Quantity

	In their current configuration, many environments are producing more log messages than are necessary to appropriately debug the 
system. One can change this configuration relatively easily by changing some settings in rsyslog.d.

	Edit the /etc/rsyslog.d/client.conf file on the director node

[stack@ospdirector ~]$ sudo sed -i 's/\*\.\*/*.info/g' /etc/rsyslog.d/client.conf \
&& cat /etc/rsyslog.d/client.conf && sudo systemctl restart rsyslog \
&& sudo systemctl status -l rsyslog

	Edit the /etc/rsyslog.d/client.conf file on the controller nodes

[stack@ospdirector ~]$ for i in $( nova hypervisor-list | \
awk '/localdomain/ {print $4}' | sed s/.localdomain// | \
sort -V) ; do echo $i >> rsyslog-level-change.txt; \
ssh heat-admin@$i "sudo sed –i 's/^\*\.\*/\*\.info/' \
/etc/rsyslog.d/client.conf && cat /etc/rsyslog.d/client.conf \
&& sudo systemctl restart rsyslog && sudo systemctl status -l rsyslog" \
>> rsyslog-level-change.txt; done

	Edit the /etc/rsyslog.d/client.conf file on the compute nodes

[stack@ospdirector ~]$ for i in $(nova host-list | \
awk '/consoleauth/ {split($2,a,"."); print a[1]}' | \
sort -V) ; do echo $i >> rsyslog-level-change.txt; \
ssh heat-admin@$i "sudo sed -i 's/^\*\.\*/\*\.info/' /etc/rsyslog.d/client.conf \
&& cat /etc/rsyslog.d/client.conf && sudo systemctl restart rsyslog \
&& sudo systemctl status -l rsyslog" >> rsyslog-level-change.txt; done