Setting up and running the tests for Dogtag Certificate System ================================================================ ** Note- All the paths mentioned in this document start from the root of the cloned source tree. 1. Running the tests on local machine. 1.1.Running the tests standalone in Eclipse (only the Java tests (JUnit v>=4.11 )) 1.1.1. Setting up the test environment. A new DS instance, to be used by the CA instance for which the tests are run, can be created by running the following command: setup-ds.pl --silent --\ General.FullMachineName=$HOSTNAME\ General.SuiteSpotUserID=dirsrv\ General.SuiteSpotGroup=dirsrv\ slapd.ServerPort=389\ slapd.ServerIdentifier=pki-tomcat\ slapd.Suffix=dc=example,dc=com\ slapd.RootDN="cn=Directory Manager"\ slapd.RootDNPwd=Secret123 Then a CA instance can be created using the deploment configuration file tests/dogtag/conf/deploy.cfg with the command: pkispawn -s CA -f deploy.cfg The ca_admin_cert.p12 file has to be imported into an NSS DB to be used for authentication in the tests. Since pkispawn can only be run as root user, the ca_admin_cert.p12 file is created in /root/.dogtag/pki-tomcat/. The root user can run the following commands to make it available to a non-root user. cp ~/.dogtag/pki-tomcat/ca_admin_cert.p12 /tmp chmod 777 /tmp/ca_admin_cert.p12 Now the following commands can be executed to import the cert PKCS12 file into a local NSS DB. mkdir /tmp/nssdb certutil -N -d /tmp/nssdb ** When prompted for a password, enter Secret123 pk12util -i /tmp/ca_admin_cert.p12 -d /tmp/nssdb ** Enter "Secret123" as password for both the NSSDB and the p12 file. Any the changes during the initial setup have to be updated in the tests/dogtag/conf/test.cfg. (Used by the tests) With all the setting above mentioned the tests will run successfully. 1.1.2. Running the tests The tests can be either run individually or as as part of a suite in Eclipse. To run the tests as a suite, Open BeakerTestSuite and run as JUnit test. A suite is a test runner which executes all the tests added to it using the @SuiteClasses tag. A customized suite, PKITestSuite, is used to provide additional functionality when the tests are run on a beaker test machine. To run the tests individually(a class like CATestJunit), just run the class as a JUnitTest. 1.2.Running all the tests on a beaker test machine from command line. 1.2.1 Setting up a beaker client on a local machine. The following steps help setup a beaker client for creating the task rpm and submitting the beaker job to the beaker server. -- Copy the file tests/dogtag/conf/beaker-client.repo to /etc/yum.repos.d and replace with the version of fedora. (run this command for fedora version: cat /etc/fedora-release) -- Do: yum install expect beakerlib beaker-client rhts-devel -- Create a folder /etc/beaker and copy the file tests/dogtag/conf/client.conf to that folder. Update the client.conf file with the beaker server's authentication details. For an Username/Password authentication enter values for HUB_URL, USERNAME, PASSWORD. The sample file has been configured for using this setting as default. For a Kerberos authentication, comment the AUTH_METHOD="password", USERNAME and PASSWORD fields uncomment and enter the values for HUB_URL and KRB_REALM and uncomment the line AUTH_METHOD="krbv". This sets up the machine as a beaker client, on which new task rpms can be built and new jobs submitted to the beaker server. 1.2.2 Building the beaker task rpm and submitting a beaker job. 1.2.2.1 Creating a repository of the built rpms. Use --createrepo option in the compose scripts to create a repository of the rpms built, like: ./compose_pki_core_packages --createrepo=conf/repository.cfg hybrid_rpms This command builds the rpms, copies them to the REPOSITORY_LOCATION specified in the file and creates it as a repository. If the location specified is hosted on a httpd server, this repository URL can be used in the job xml as a repo URL. If a remote host name is provided then the repos are scp'ed at the location provided on the remote host using the credentials provided. (If no login credentials are provided, then a prompt appears asking for them.) 1.2.2.2 Building the task rpm and submitting the job to a beaker server is done by executing the compose script in pki/scripts: ./compose_pki_test_package unique_identifier [--submit-tests] The builds are done in /../package.tests/ folder. -- The unique_identifier is for personalizing the rpm. This path at which the test source is extracted from the rpm on the beaker server is specified by the TEST variable in the Makefile (pki/tests/dogtag/Makefile). So another rpm with the same to-be-extracted path overwrites the existing test code. In order to prevent it, a unique id is asked to personalize the rpm to provide a unique to-be-extracted path for every test rpm of dogtag tests (especially at a user level). -- The optional --submit-tests option. This option submits the test rpm to the beaker server. ./create_beaker_job unique_identifier beaker_job_config [--submit-job] The unique_identifier has the same functionality as the one used in the compose_pki_test_rpm command. The "beaker_job_config" (the absolute path of the file from /) is for configuring the recipe of a job. A sample configuration file is at tests/conf/beaker-job.cfg. The repos parameter should point to the URL's to access the dogtag rpms. The hostname means the hostname of a specific beaker test machine. The distro_* are the details of the distribution on the beaker server, that has to be installed on the test machine. The optional --submit-job command submits the job to the beaker server. **Note- A beaker job is an XML file. The template used to create a job is in tests/dogtag/beakerjob.dogtag.xml.template. The task /distribution/reservesys is commented. On uncommenting this task in the template, the beaker test machine is reserved for a certain period of time after the tests are run. This option can be used to debug any tests that are failing. **Note- In the case of creating a repository on a remote machine, the remote machine must be a known_host to the local machine. 2. Setting up a Jenkins server for continuous integration testing Note- Follow the steps in section 1.2.1 to setup a beaker-client on the system. 2.1 Setting up a Jenkins server. 2.1.1 Installing the Jenkins (and plugins) The instructions for installing a jenkins server can be found at: https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins. (It is better to configure the Jenkins server to listen on a port other than 8080 to not raise port conflicts. The jenkins configuraion file is at /etc/sysconfig/jenkins. A server restart is necessary for any changes to this file to be effective. ) The Jenkins UI can be accessed by the URL: http://:/, where the hostname is the hostname of the machine where Jenkins is installed and port is the port on which the server is listening. The CI test framework requires a few additional plugins to be installed for the Jenkins server. -- Goto Jenkins -> Manage Jenkins -> Manage Plugins page on the UI. Click on the "Available" tab. Search and install the following plugins: * GitHub plugin * Jenkins Beaker builder plugin * Jenkins Email Extension Plugin 2.1.2 Configuring Jenkins setup The following configurations are required for the setup to be complete: Goto Jenkins -> Manage Jenkins -> Configure System. Update the "System Admin e-mail address" in "Jenkins Location" section. Enter the details of the beaker server in the "Beaker Builder" section. The server's certificate needs to be stored in a Java keystore if the connection uses https protocol. The details for fixing this issue can be found in the plugin documentation at: https://wiki.jenkins-ci.org/display/JENKINS/Beaker+Builder+Plugin. The "Email Notification" section needs to be completed with the details of the SMTP server and its authentication details (Click on the Advanced button). 2.2 Create a New Project - ex: Dogtag-pki 2.2.1 Configuring the project (Click on "Configure" in the left panel): Setting up the workspace: Set the path to the custom workspace in the Advanced Project Options section to "/pki" Click on Advanced -> Select Use custom workspace -> Enter the path. The build proces of Dogtag requires the top source directory to be named as pki. Hence it is required to setup a the workspace path. To use the default Jenkins workspace, set the path as /var/lib/jenkins/workspace/pki. i. For continuous integration, the project can be configured to clone the source from the repository. (git://git.fedorahosted.org/git/pki.git). The Git option can be found in the SCM section. ii. For using the working project directory as the source to build the rpms - Select None in the SCM section and clone the source in the above mentioned location. 2.2.2 Building the rpms and running the tests. In the "Build" section, select "Add build step" -> "Execute Shell" Paste the following code in the Command text box. cd $WORKSPACE/scripts; ./compose_pki_core_packages [-- createrepo=] hybrid_rpms echo "Compose the test package and submit the beaker task and job" cd $WORKSPACE/scripts; ./compose_pki_test_package dogtagCI --submit-tests This shell script builds the pki core rpms, copies them to a repository, creates the test rpms using the unique_identifier dogtagCI and submits it to the beaker server. Then, select "Add build step" -> "Execute Beaker Task". Select the option "Specify Job XML". Copy the text in the pki/tests/dogtag/beakerjob.dogtag.xml.template file into the test area. Replace the string PKI_TEST_USER_ID in the task name to dogtagCI. Enter values for the distro required in the distroRequires element of the XML. Add the repository URLs in the repos element as follows: Again, select "Add build step" -> "Execute shell" Paste the following code in the text box: echo "Check if there are any regressions in the current beaker test run." cd $WORKSPACE/tests/dogtag/util ./examine_results.sh ${BEAKER_JOB_ID} This shell script examines the results of the curent and the previous run and send an email as per the triggers, only if any regressions are found. 2.2.3 Setting up the mail client For setting up the post build email notification, select "Add post-build action" -> "Email Notification". This can be used to specify the email recipients and configuring when and to whom the email has to be sent. Configuring only this action will send emails for any result of the build process to all the recipients. (Even if the build passes successfully always.) To get an email only when there are any regressions, add another post build action, "Add post-build action" -> "Edit Email Notification". This action provides additional options to configure the email itself. Click on "Advanced" button and copy the following code into the Pre-send script text box : def fileName = new FilePath(build.getWorkspace(), "/beaker/regressions.txt"); if (fileName.exists) { def file = new File(fileName.toString()) msg.setText(file.getText()); } else { cancel=true; } This Groovy script looks for a file regressions.txt, created by the examine_results script mentioned shown in the previous section. The file contains the names of the tests that passed previously but failed now. A mail will also be generated if there are tests that passed now and were failing before. This helps reduce spamming of emails when running the continuous integration testng framework. The "Add Trigger" selection box will be useful to add a trigger to configure for which result of the build an email has to be sent to the recipients. Additional Information: -- The Build Triggers section provides options for specifying the time at which a new build can be triggered - periodically or on a git commit. -- If no Build triggers are configured in the project, the build process can be initiated by clicking on "Build Now" link in the left panel. The console output can be viewed by clicking the "Console Output" link.