update openQA in Docker to latest version and fix all related problems
ClosedPublic

Authored by garretraziel on Oct 2 2015, 1:39 PM.

Details

Summary

For known reasons, we need to update Dockerized openQA
(see for example T623). I updated Dockerfiles, removed unnecessary
args and commands, updated db.sqlite to be compatible with newest
version (there was change in db schema) and bumped config a little bit.

Before testing this thought, we probably need to wait until
https://github.com/os-autoinst/openQA/issues/438 gets fixed.

Test Plan

Try to build openqa_data, openqa_worker and openqa_webui,
schedule some tests with them.

Diff Detail

Repository
rOPENQA fedora_openqa
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
garretraziel retitled this revision from to update openQA in Docker to latest version and fix all related problems.Oct 2 2015, 1:39 PM
garretraziel updated this object.
garretraziel edited the test plan for this revision. (Show Details)
garretraziel added reviewers: adamwill, jskladan.
garretraziel updated this object.Oct 2 2015, 1:40 PM

Mostly looks fine, but I know nothing about Docker so I feel unqualified to sign off :)

docker/openqa_data/data.template/conf/openqa.ini
6

We can set this to 'plain' to lose a bit of the SUSE branding (the default is 'opensuse').

35

I always wonder if maybe we want to change this to info? Meh.

  • change openQA settings a bit, run scheduler and websocket processess

The code still isn't working because of https://github.com/os-autoinst/openQA/issues/438.

The problem is that I can't really get DBus to work in Docker and I got to the point where I don't know what to do. I was even trying to run systemd inside Docker (according to dwalsh guide), but that isn't something we should really do (and it didn't work anyway).

I'll wait to see how the bug is solved before acking/nacking. My gut tells me that there will be changes anyway...

I figured out the dbus problem and sent a fix to the feature/docker_webui branch in git. Now you can view the workers. There's another thing that needs doing, though: the database schema changed again upstream, so the db.sqlite file in openqa_data/data.template/db needs updating again. I didn't know exactly how you're handling that bit so I'm gonna leave it to you (suggestion: document how that stuff is produced).

(admission: OK, OK, it was me who changed the database schema upstream. I'm sorry. :>)

garretraziel updated this revision to Diff 1594.Oct 7 2015, 8:14 AM
  • change openQA settings a bit, run scheduler and websocket processess
  • docker: run services as geekotest not root
  • docker: run gru service too
  • update database

Current version should be OK, I tried to schedule some tests and it works.

Sure, I will document how to update db, but current state is not ideal - I basically remove all rm db.sqlite stuff from webui, build it and use docker cp. There is init-db script installed alongside openQA, but problem is that we need database to be present in openqa_data, so there is no immediate solution.

So the current code still doesn't really work - turns out if you try to restart webui, running dbus fails, because /run/dbus/pid file exists. Fix should be straightforward.

garretraziel updated this revision to Diff 1596.Oct 7 2015, 1:28 PM
  • remove leftover pid file

The other thing I wondered is if we should run the database update (and anything else the RPM package does on upgrade) at the top of the webui Dockerfile, and recommend starting the data container before the webui one. That way we can handle future updates by updating the webui and worker containers, I guess? Still not very familiar with Docker so let me know if I'm missing something there.

recommend starting the data container before the webui one

That is the current state - you have to run openqa_data first, because all config files are located there.

That way we can handle future updates by updating the webui and worker containers, I guess?

Bingo! That's the very reason we have separate openqa_data - you keep all the tests, results, config files (with key+secret) and database there, so you can do whatever you want with webui and worker and still have the same environment.

Unfortunately, we haven't considered db migration up until now.

This revision is now accepted and ready to land.Oct 8 2015, 7:00 AM

on the update stuff - yeah, that's what I figured. i'd look at all the RPM scriptlets and make sure we replicate that stuff somehow.

garretraziel updated this revision to Diff 1597.Oct 8 2015, 9:44 AM
  • upgrade openQA db before running

I went through specfile and found out that only necessary command is db upgrading. There were other things (mainly moving from geekotest to _openqa_worker user, moving existing directories when upgrading from ancient versions...) but I don't think they are necessary for us.

jskladan accepted this revision.Oct 8 2015, 10:06 AM

apart of the nitpick, great job! I envy you the level of insanity, you must have to deal with stuff like this :)

docker/webui/run_openqa.sh
6

either document what : does, or replace by true, thx!

garretraziel added inline comments.Oct 8 2015, 11:38 AM
docker/webui/run_openqa.sh
6

I've just realized that there is no reason why it should matter whether upgradedb returns non-zero value, so I'll delete it altogether.

This revision was automatically updated to reflect the committed changes.

I went through specfile and found out that only necessary command is db upgrading. There were other things (mainly moving from geekotest to _openqa_worker user, moving existing directories when upgrading from ancient versions...) but I don't think they are necessary for us.

Yeah, those are migrations, though they're not as ancient as you think: I think both will apply to us when we switch our production instance, the /var/lib/openqa/share directory does not exist in the 'stable' version.