Differential D357

Dockerize OpenQA
ClosedPublic
Actions

Authored by garretraziel on Apr 30 2015, 10:53 AM.

Details

Reviewers
adamwill
jskladan
Summary

We've been dockerizing OpenQA last few weeks with jskladan.
This enables us to run OpenQA on Fedora in our infrastructure, to
freeze specific version of OpenQA on production machines, it makes
deployment and installation easy. Read README.md to get information
about its architecture. This was already merged into upstream
here: https://github.com/os-autoinst/openQA/pull/340.

Test Plan

Read README.md for information how to build and run
OpenQA in Docker.

Diff Detail

Repository
rOPENQA fedora_openqa
Branch
feature/docker
Lint
No Linters Available
Unit
No Unit Test Coverage
garretraziel retitled this revision from to Dockerize OpenQA.Apr 30 2015, 10:53 AM
garretraziel updated this object.
garretraziel edited the test plan for this revision. (Show Details)
garretraziel added reviewers: jskladan, adamwill.
This revision is now accepted and ready to land.Apr 30 2015, 11:11 AM
adamwill added a comment.Apr 30 2015, 3:13 PM

Are all those empty .gitignore files intentionally a part of the commit?

jskladan added a comment.May 4 2015, 8:18 AM
In D357#6486, @adamwill wrote:

...

Yes, it is the only way to add "empty" dirs to git, sadly...

garretraziel closed this revision.Jul 2 2015, 10:58 AM

Revision Contents

Diff 958

View Options

docker/.gitignore

  • This file was added.
1​data/
View Options

docker/README.md

  • This file was added.
1​# Build docker images
2​
3​ docker build -t fedoraqa/openqa_webui ./webui
4​ docker build -t fedoraqa/openqa_worker ./worker
5​ docker build -t fedoraqa/openqa_data ./openqa_data
6​
7​# Running OpenQA
8​
9​Our intent was to create universal `webui` and `worker` containers and move all data storage and configurations to third container,
10​called `openqa_data`. `openqa_data` is so called [Data Volume Container](http://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container)
11​and is used as database, results and configuration storage. During development and in production, you could update `webui` and `worker` images
12​but as long as `openqa_data` is intact, you don't lose any data.
13​
14​To make development easier and to reduce final size of `openqa_data` container, this guide describes how to override `tests` and `factory` directories
15​with directories from your host system. It's not necessary, but it's recommended and this guide is written with this setup in mind.
16​
17​It's also possible to use `tests` and `factory` from within the `openqa_data` container (so you don't have any dependency on your host system) or
18​to ditch `openqa_data` container altogether (so you have only `webui` and `worker` containers and data is loaded and saved completely into your host
19​system). If this is what you want, refer to [Keeping all data in the Data Volume Container] and [Keeping all data on the host system] sections respectively.
20​
21​## Create directory structure
22​
23​ mkdir -p data/factory/{iso,hdd} data/tests
24​
25​It is also necessary to either run all containers in privileged mode, or set selinux properly:
26​
27​ chcon -Rt svirt_sandbox_file_t data
28​
29​## Run the Data & WebUI containers
30​
31​ docker run -d -h openqa_data --name openqa_data -v `pwd`/data/factory:/data/factory -v `pwd`/data/tests:/data/tests fedoraqa/openqa_data
32​ docker run -d -h openqa_webui --volumes-from openqa_data --name openqa_webui -p 8080:443 fedoraqa/openqa_webui
33​
34​It is now necessary to create and store the client keys for OpenQA. In the next two steps, you will set an OpenID provider (if necessary),
35​create the API keys in the OpenQA's web interface, and store the configuration in the Data Container.
36​
37​### Change the OpenID provider
38​
39​https://id.fedoraproject.org/ is set as a default OpenID provider. To change it, run:
40​
41​ docker exec -ti openqa_data /scripts/set_openid
42​
43​and enter the Provider's URL.
44​
45​### Set API keys
46​
47​Go to https://localhost:8080/api_keys, generate key and secret. Then run:
48​
49​ docker exec -ti openqa_data /scripts/set_keys
50​
51​and enter the Key and Secret.
52​
53​## Run the Worker container
54​
55​ docker run -h openqa_worker_1 --name openqa_worker_1 -d --link openqa_webui:openqa_webui --volumes-from openqa_data --volumes-from openqa_webui --privileged fedoraqa/openqa_worker 1
56​
57​Check whether the worker connected in the WebUI's administration interface.
58​
59​To add more workers, increase number that is used in hostname, container name and at the end of command, so to add worker 2 use:
60​
61​ docker run -h openqa_worker_2 --name openqa_worker_2 -d --link openqa_webui:openqa_webui --volumes-from openqa_data --volumes-from openqa_webui --privileged fedoraqa/openqa_worker 2
62​
63​## Get tests and ISOs
64​
65​You have to put your tests under `data/tests` directory and ISOs under `data/factory/iso` directory. For example, for testing Fedora, run:
66​
67​ git clone https://bitbucket.org/rajcze/openqa_fedora data/tests/fedora
68​ wget https://dl.fedoraproject.org/pub/alt/stage/22_Beta_RC3/Server/x86_64/iso/Fedora-Server-netinst-x86_64-22_Beta.iso -O data/factory/iso/Fedora-Server-netinst-x86_64-22_Beta_RC3.iso
69​
70​And set permissions, so any user can read/write the data:
71​
72​ chmod -R 777 data
73​
74​This step is unfortunately necessary because Docker [can't mount volume with specific user ownership](https://github.com/docker/docker/issues/7198) in container, so ownership of mounted folders (uid and gid) is the same as on your host system (presumably 1000:1000 which maps into nonexistent user in all of the containers).
75​
76​If you wish to keep the tests (for example) separate from the shared directory, for any reason (we do, in our development scenario) refer to the [Developing tests with Container setup] section at the end of this document.
77​
78​Populate the OpenQA's database:
79​
80​ docker exec openqa_webui /var/lib/openqa/tests/fedora/templates
81​
82​
83​# Running jobs
84​
85​After performing the "setup" tasks above - do not forget about tests and ISOs - you can schedule a test like this:
86​
87​ docker exec openqa_webui /var/lib/openqa/script/client isos post ISO=Fedora-Server-netinst-x86_64-22_Beta_RC3.iso DISTRI=fedora VERSION=rawhide FLAVOR=generic_boot ARCH=x86_64 BUILD=22_Beta_RC3
88​
89​# Other specific cases
90​
91​## Keeping all data in the Data Volume Container
92​
93​If you decided to keep all the data in the Volume Container (`openqa_data`) then instead of [Create directory structure] run:
94​
95​ docker exec openqa_data mkdir -p data/factory/{iso,hdd} data/tests
96​ docker exec openqa_data chmod -R 777 data/factory/{iso,hdd} data/tests
97​
98​In the [Run the Data & WebUI containers] section run the `openqa_data` container like this instead:
99​
100​ docker run -d -h openqa_data --name openqa_data fedoraqa/openqa_data
101​
102​And finally, download the tests and ISOs directly into the container:
103​
104​ docker exec openqa_data git clone https://bitbucket.org/rajcze/openqa_fedora /data/tests/fedora
105​ docker exec openqa_data wget https://dl.fedoraproject.org/pub/alt/stage/22_Beta_RC3/Server/x86_64/iso/Fedora-Server-netinst-x86_64-22_Beta.iso -O /data/factory/iso/Fedora-Server-netinst-x86_64-22_Beta_RC3
106​
107​The rest of the steps should be the same.
108​
109​## Keeping all data on the host system
110​
111​If you want to keep all the data in the host system and you don't want to use Volume Container, then instead of [Create directory structure] run:
112​
113​ cp -a openqa_data/data.template data
114​ chcon -Rt svirt_sandbox_file_t data
115​
116​In the [Run the Data & WebUI containers] section don't run `openqa_data` container and run the `webui` container like this instead:
117​
118​ docker run -d -h openqa_webui -v `pwd`/data:/data --name openqa_webui -p 8080:443 fedoraqa/openqa_webui
119​
120​Change OpenID provider in `data/conf/openqa.ini` under `provider` in `[openid]` section and then put Key and Secret under
121​both sections in `data/conf/client.conf`.
122​
123​In the [Run the Worker container] section, run worker as:
124​
125​ docker run -h openqa_worker_1 --name openqa_worker_1 -d --link openqa_webui:openqa_webui -v `pwd`/data:/data --volumes-from openqa_webui --privileged fedoraqa/openqa_worker 1
126​
127​Then continue with tests and ISOs downloading as before.
128​
129​## Developing tests with Container setup
130​
131​With this setup, the needles created from the WebUI will almost certainly have different owner and group than your user account.
132​As we have the tests in GIT, and still want to retain the original owner and permission, even as we update/crate needles from OpenQA.
133​To accomplish this, we are using BindFS. An example entry in `/etc/fstab`:
134​
135​ bindfs#/home/jskladan/src/openQA/openqa_fedora /home/jskladan/src/openQA/openqa_fedora_tools/docker/data/tests/fedora fuse create-for-user=jskladan,create-for-group=jskladan,create-with-perms=664:a+X,perms=777 0 0
136​
137​Mounts the `openqa_fedora` directory to the `.../tests/fedora directory`. All files in the `tests/fedora` directory seem to have 777 permissions set, but new files are created (in the underlying `openqa_fedora` directory) with `jskladan:jskladan` user and group, and 664:a+X permissions.
View Options

docker/openqa_data/Dockerfile

  • This file was added.
1​FROM opensuse:13.2
2​MAINTAINER Jan Sedlak <jsedlak@redhat.com>, Josef Skladanka <jskladan@redhat.com>
3​
4​RUN zypper --non-interactive --gpg-auto-import-keys in ca-certificates-mozilla git wget
5​
6​ADD data.template /data
7​ADD scripts /scripts
8​RUN chmod -R 777 /data /scripts
9​VOLUME ["/data"]
10​
11​CMD /usr/bin/tail -f /dev/null
View Options

docker/openqa_data/data.template/conf/client.conf

  • This file was added.
1​[openqa_webui]
2​key =
3​secret =
4​
5​[localhost]
6​key =
7​secret =
View Options

docker/openqa_data/data.template/conf/openqa.ini

  • This file was added.
1​[global]
2​## type of branding - [ openSUSE, plain ]
3​#branding = plain
4​
5​## type and location of needle repo
6​#scm = git
7​
8​## space seaparated list of IPs or regular expressions that match IPs
9​#allowed_hosts = 127.0.0.1 ::1
10​
11​## set if you have a local repo mirror
12​#suse_mirror = http://FIXME
13​
14​## base url [default: $self->req->url->base]
15​#base_url = http://FIXME
16​
17​# days for Strict-Transport-Security, 0 to not add this header
18​# http://en.wikipedia.org/wiki/Strict-Transport-Security
19​# hsts = 365
20​
21​#[scm git]
22​#do_push = no
23​
24​## Authentication method to use for user management
25​[auth]
26​method = OpenID
27​
28​[logging]
29​#if you use another file, remember the apparmor profile!
30​file = /var/log/openqa
31​#level = debug|info|warn|error|fatal
32​
33​[openid]
34​## base url for openid provider
35​provider = https://id.fedoraproject.org/
36​## enforce redirect back to https
37​#httpsonly = 1
View Options

docker/openqa_data/data.template/conf/workers.ini

  • This file was added.
1​[global]
2​BACKEND = qemu
3​HOST = http://openqa_webui
View Options

docker/openqa_data/data.template/db/db.sqlite

  • This binary file was added.
PropertyOld ValueNew Value
MIME Typenullapplication/octet-stream
File Size0 B86 KB
View Options

docker/openqa_data/data.template/factory/.gitignore

  • This file was added.
This is an empty file.
View Options

docker/openqa_data/data.template/factory/hdd/.gitignore

  • This file was added.
This is an empty file.
View Options

docker/openqa_data/data.template/factory/iso/.gitignore

  • This file was added.
This is an empty file.
View Options

docker/openqa_data/data.template/testresults/.gitignore

  • This file was added.
This is an empty file.
View Options

docker/openqa_data/data.template/tests/.gitignore

  • This file was added.
This is an empty file.
View Options

docker/openqa_data/scripts/set_keys

  • This file was added.
1​#!/bin/bash
2​
3​echo Key:
4​read KEY
5​echo Secret:
6​read SECRET
7​
8​sed "s/key =.*/key = $KEY/" -i /data/conf/client.conf
9​sed "s/secret =.*/secret = $SECRET/" -i /data/conf/client.conf
10​
11​echo "The file now contains:"
12​cat /data/conf/client.conf
View Options

docker/openqa_data/scripts/set_openid

  • This file was added.
1​#!/bin/bash
2​
3​echo OpenID provider:
4​read OPENID
5​
6​sed "s/provider =.*/provider = $OPENID/" -i /data/conf/openqa.ini
7​
8​echo "The file now contains:"
9​cat /data/conf/openqa.ini
View Options

docker/webui/Dockerfile

  • This file was added.
1​FROM opensuse:13.2
2​MAINTAINER Jan Sedlak <jsedlak@redhat.com>, Josef Skladanka <jskladan@redhat.com>
3​
4​RUN useradd -m geekotest && \
5​ zypper --non-interactive --gpg-auto-import-keys in ca-certificates-mozilla curl && \
6​ zypper ar -f http://download.opensuse.org/repositories/devel:openQA:stable/openSUSE_13.2/devel:openQA:stable.repo && \
7​ zypper ar -f obs://devel:openQA/openSUSE_13.2 openQA && \
8​ zypper ar -f obs://devel:openQA:13.2/openSUSE_13.2 openQA-perl-modules && \
9​ curl https://build.opensuse.org/source/devel:openQA/_pubkey > /tmp/openQA-pubkey && rpm --import /tmp/openQA-pubkey && \
10​ zypper --non-interactive in --from devel_openQA_stable openQA
11​
12​# setup apache
13​RUN gensslcert && \
14​ a2enmod headers && \
15​ a2enmod proxy && \
16​ a2enmod proxy_http && \
17​ a2enmod proxy_wstunnel && \
18​ a2enmod ssl && \
19​ a2enflag SSL
20​ADD openqa-ssl.conf /etc/apache2/vhosts.d/openqa-ssl.conf
21​ADD openqa.conf /etc/apache2/vhosts.d/openqa.conf
22​
23​# set-up shared data and configuration
24​RUN rm -rf /etc/openqa/openqa.ini /etc/openqa/client.conf \
25​ /var/lib/openqa/share/factory /var/lib/openqa/share/tests \
26​ /var/lib/openqa/db/db.sqlite /var/lib/openqa/testresults && \
27​ mkdir -p /var/lib/openqa/pool && \
28​ ln -s /data/conf/openqa.ini /etc/openqa/openqa.ini && \
29​ ln -s /data/conf/client.conf /etc/openqa/client.conf && \
30​ ln -s /data/factory /var/lib/openqa/share/factory && \
31​ ln -s /data/tests /var/lib/openqa/share/tests && \
32​ ln -s /data/testresults /var/lib/openqa/testresults && \
33​ ln -s /data/db/db.sqlite /var/lib/openqa/db/db.sqlite && \
34​ chmod -R 777 /var/lib/openqa/pool
35​
36​VOLUME ["/var/lib/openqa/pool"]
37​
38​EXPOSE 80 443
39​CMD rcapache2 start && /usr/share/openqa/script/openqa daemon -m production --proxy
View Options

docker/webui/openqa-ssl.conf

  • This file was added.
1​# sample apache config file for the ssl enabled openqa vhost.
2​# At the very minimum you need to set the ServerName.
3​
4​<IfDefine SSL>
5​<IfDefine !NOSSL>
6​
7​<VirtualHost *:443>
8​
9​ #ServerName your.server.name
10​
11​ SSLEngine on
12​
13​ ## SSL settings in case you want to override your server's
14​ ## default:
15​
16​ SSLProtocol all -SSLv2 -SSLv3
17​ SSLCipherSuite HIGH:MEDIUM:!aNULL:!MD5
18​ SSLCertificateFile /etc/apache2/ssl.crt/server.crt
19​ SSLCertificateKeyFile /etc/apache2/ssl.key/server.key
20​ SSLCertificateChainFile /etc/apache2/ssl.crt/ca.crt
21​
22​ Include /etc/apache2/vhosts.d/openqa-common.inc
23​
24​</VirtualHost>
25​
26​</IfDefine>
27​</IfDefine>
28​
View Options

docker/webui/openqa.conf

  • This file was added.
1​# sample apache config file for the openqa vhost. At the very
2​# minimum you need to set the ServerName.
3​#
4​# If this is your first vhost you must create a default vhost, e.g.
5​# in 01-default.conf with the following content:
6​#
7​#NameVirtualHost *:80
8​#<VirtualHost *:80>
9​# ServerName www.example.com
10​# DocumentRoot "/srv/www/htdocs"
11​#</VirtualHost>
12​#
13​# Also, the following apache modules must be enabled:
14​#
15​# a2enmod headers
16​# a2enmod proxy
17​# a2enmod proxy_http
18​# a2enmod proxy_wstunnel
19​
20​<VirtualHost *:80>
21​
22​ #ServerName your.server.name
23​
24​ Include /etc/apache2/vhosts.d/openqa-common.inc
25​
26​</VirtualHost>
27​
28​# vim: syntax=apache
View Options

docker/worker/Dockerfile

  • This file was added.
1​FROM opensuse:13.2
2​MAINTAINER Jan Sedlak <jsedlak@redhat.com>, Josef Skladanka <jskladan@redhat.com>
3​
4​RUN useradd -m geekotest && \
5​ zypper --non-interactive --gpg-auto-import-keys in ca-certificates-mozilla curl && \
6​ zypper ar -f http://download.opensuse.org/repositories/devel:openQA:stable/openSUSE_13.2/devel:openQA:stable.repo && \
7​ zypper ar -f obs://devel:openQA/openSUSE_13.2 openQA && \
8​ zypper ar -f obs://devel:openQA:13.2/openSUSE_13.2 openQA-perl-modules && \
9​ curl https://build.opensuse.org/source/devel:openQA/_pubkey > /tmp/pubkey && rpm --import /tmp/pubkey && \
10​ zypper --non-interactive in --from devel_openQA_stable openQA-worker qemu-kvm
11​
12​# set-up qemu
13​RUN mkdir -p /root/qemu
14​ADD kvm-mknod.sh /root/qemu/kvm-mknod.sh
15​RUN chmod +x /root/qemu/*.sh && /root/qemu/kvm-mknod.sh && \
16​ # set-up shared data and configuration
17​ rm -rf /etc/openqa/client.conf /etc/openqa/workers.ini \
18​ /var/lib/openqa/pool && \
19​ mkdir -p /var/lib/openqa/share && \
20​ ln -s /data/conf/client.conf /etc/openqa/client.conf && \
21​ ln -s /data/conf/workers.ini /etc/openqa/workers.ini && \
22​ ln -s /data/factory /var/lib/openqa/share/factory && \
23​ ln -s /data/tests /var/lib/openqa/share/tests
24​
25​ENTRYPOINT ["/usr/share/openqa/script/worker", "--instance"]
26​CMD ["1"]
View Options

docker/worker/kvm-mknod.sh

  • This file was added.
1​#!/bin/bash
2​# If possible, create the /dev/kvm device node.
3​
4​set -e
5​
6​lsmod | grep '\<kvm\>' > /dev/null || {
7​ echo >&2 "KVM module not loaded; software emulation will be used"
8​ exit 1
9​}
10​
11​mknod /dev/kvm c 10 $(grep '\<kvm\>' /proc/misc | cut -f 1 -d' ') || {
12​ echo >&2 "Unable to make /dev/kvm node; software emulation will be used"
13​ echo >&2 "(This can happen if the container is run without -privileged)"
14​ exit 1
15​}
View Options

tools/openqa_trigger/report_job_results.py

Show First 20 Lines • Show All 44 Lines • ▼ Show 20 Line(s)41 for testcase in conf_test_suites.TESTSUITES[testsuite]:
45 # than one for a testcase) and environment for which the result should be filed. 45​ # than one for a testcase) and environment for which the result should be filed.
46 uniqueres = conf_test_suites.TESTCASES[testcase] 46​ uniqueres = conf_test_suites.TESTCASES[testcase]
47 testname = '' 47​ testname = ''
48 if 'name_cb' in uniqueres: 48​ if 'name_cb' in uniqueres:
49 testname = uniqueres['name_cb'](flavor) 49​ testname = uniqueres['name_cb'](flavor)
50 env = arch if uniqueres['env'] == '$RUNARCH$' else uniqueres['env'] 50​ env = arch if uniqueres['env'] == '$RUNARCH$' else uniqueres['env']
51 result = ResTuple( 51​ result = ResTuple(
52 testtype=uniqueres['type'], release=release, milestone=milestone, compose=compose, 52​ testtype=uniqueres['type'], release=release, milestone=milestone, compose=compose,
53 testcase=testcase, section=uniqueres['section'], testname=testname, env=env, status='pass') 53​ testcase=testcase, section=uniqueres['section'], testname=testname, env=env, status='pass',
54​ bot=True)
54 passed_testcases.add(result) 55​ passed_testcases.add(result)
55 56​
56 return sorted(list(passed_testcases), key=attrgetter('testcase')) 57​ return sorted(list(passed_testcases), key=attrgetter('testcase'))
57 58​
58def report_results(job_ids, printcases=False, report=True): 59​def report_results(job_ids, printcases=False, report=True):
59 passed_testcases = get_passed_testcases(job_ids) 60​ passed_testcases = get_passed_testcases(job_ids)
60 if printcases: 61​ if printcases:
61 for restup in passed_testcases: 62​ for restup in passed_testcases:
Show All 25 Lines