Differential D1126

Dockerize taskotron
AbandonedPublic
Actions

Authored by jsedlak on Feb 10 2017, 1:23 PM.

Details

Reviewers
jskladan
Summary

This creates necessary files for making Docker image from
libtaskotron's git repo.

Test Plan

run docker build -t fedoraqa/libtaskotron-git .

Diff Detail

Repository
rLTRN libtaskotron
Branch
feature/dockerize (branched from develop)
Lint
No Linters Available
Unit
Unit Test Errors
Build Status
Buildable 1050
Build 1050: arc lint + arc unit

Unit TestsBroken
View All

Excuse: !srorre desiar gnitset tinU
TimeTest
0 ms.testing.functest_koji_utils
testing/functest_koji_utils.py:8: in <module> from dingus import Dingus E ImportError: No module named dingus
0 ms.testing.functest_resultsdb_directive
testing/functest_resultsdb_directive.py:7: in <module> from dingus import Dingus E ImportError: No module named dingus
0 ms.testing.functest_yumrepoinfo_directive
testing/functest_yumrepoinfo_directive.py:8: in <module> from dingus import Dingus E ImportError: No module named dingus
0 ms.testing.test_bodhi_directive
testing/test_bodhi_directive.py:9: in <module> from dingus import Dingus E ImportError: No module named dingus
0 ms.testing.test_bodhi_utils
testing/test_bodhi_utils.py:9: in <module> from dingus import Dingus E ImportError: No module named dingus
View Full Test Results (16 Broken · 277 Passed)
jsedlak retitled this revision from to Dockerize taskotron.Feb 10 2017, 1:23 PM
jsedlak updated this object.
jsedlak edited the test plan for this revision. (Show Details)
jsedlak added a reviewer: jskladan.
jsedlak added inline comments.Feb 10 2017, 1:26 PM
conf/namespaces.yaml.docker
9–11

This is necessary because how our dockerized buildbot master works - it git clones from local filesystem, so these values are shown as origin url. We might consider changing buildbot master so that it copies instead of cloning (we could then just usenamespaces.yaml.example) or consider dropping this completely.

jskladan requested changes to this revision.Feb 10 2017, 10:18 PM
jskladan added inline comments.
Dockerfile
4

Why gcc exactly? Most of the stuff that would need compiling (I guess it's here so you can pip install -r) will need header files anyway. I'd rather install the deps directly from the repos.

6–9

maybe put these all in one RUN?

11–13

Add can work with URL's maybe we could get around needing to duplicate the configs by using the config files from the taskotron-docker repo?
In the end, I'd like for the "latest" (i.e. built from repos) and "develop" (built from gits) docker images to be as close to each other as reasonably possible, so the usage and user experience is pretty much the same.

This said, here is a crazy idea - how about we don't base the 'devel' images on Fedora, but on the respective fedoraqa/$projectname:latest image instead.
That would give us the same configs, and stuff like that. All we'd do would be replacing the app data by installing via pip over the old stuff instead (kind of like upgrading by yum, but without building the rpm).

Thoughts?

21

I guess this is an equivalent of python setup.py install

conf/namespaces.yaml.docker
9–11

Honestly this will need to change quite a lot anyway in the future - this solution (I mean as a whole in libtaskotron) is sub-par. For the scope of the docker stuff, I'd just put all the relevant namespaces to the namespaces_safe list.

conf/taskotron.yaml.docker
76

Given the rest of the urls, you probably wanted localhost instead of resultsdb.
But it does not really matter, honesly, because for the CI thing, we will need to unify the urls anyway, and I'd go with resultsdb, execdb. etc anyway.
We'll discuss this more in person next week.

This revision now requires changes to proceed.Feb 10 2017, 10:18 PM
jsedlak updated this revision to Diff 2888.Feb 13 2017, 1:26 PM
  • changes
jsedlak abandoned this revision.Wed, Aug 2, 10:28 AM
jsedlak marked 2 inline comments as done.

This was done already, see https://pagure.io/taskotron/taskotron-docker

Revision Contents

PathPackages
AMDockerfile (22 lines)
AMconf/namespaces.yaml.docker (5 lines)
AMconf/taskotron.yaml.docker (7 lines)

Diff 2888

View Options

Dockerfile

  • This file was added.
1​from fedoraqa/buildbot-worker
2​
3​RUN dnf install -y createrepo koji mash python-configparser python-fedora python-hawkey \
4​ python-jinja2 python-munch python-pip python-progressbar \
jskladanUnsubmitted
Done

Why gcc exactly? Most of the stuff that would need compiling (I guess it's here so you can pip install -r) will need header files anyway. I'd rather install the deps directly from the repos.

5​ python-requests python-resultsdb_api python-setuptools python-xunitparser \
6​ PyYAML rpm-python sudo tar
7​RUN mkdir -p /var/log/taskotron /srv/taskotron/artifacts && dnf clean all
8​RUN install -d -m 775 /var/tmp/taskotron /var/log/taskotron \
9​ /var/cache/taskotron /var/lib/taskotron /var/lib/taskotron/artifacts \
jskladanUnsubmitted
Not Done

maybe put these all in one RUN?

10​ /var/lib/taskotron/images
11​
12​ADD conf/yumrepoinfo.conf.example /etc/taskotron/yumrepoinfo.conf
13​ADD conf/taskotron.yaml.docker /etc/taskotron/taskotron.yaml
jskladanUnsubmitted
Not Done

Add can work with URL's maybe we could get around needing to duplicate the configs by using the config files from the taskotron-docker repo?
In the end, I'd like for the "latest" (i.e. built from repos) and "develop" (built from gits) docker images to be as close to each other as reasonably possible, so the usage and user experience is pretty much the same.

This said, here is a crazy idea - how about we don't base the 'devel' images on Fedora, but on the respective fedoraqa/$projectname:latest image instead.
That would give us the same configs, and stuff like that. All we'd do would be replacing the app data by installing via pip over the old stuff instead (kind of like upgrading by yum, but without building the rpm).

Thoughts?

14​ADD conf/namespaces.yaml.docker /etc/taskotron/namespaces.yaml
15​
16​ENV LANG C.utf8
17​ENV LC_ALL C.utf8
18​
19​COPY . /usr/src/libtaskotron
20​WORKDIR /usr/src/libtaskotron
21​RUN pip install -r requirements.txt &&\
jskladanUnsubmitted
Done

I guess this is an equivalent of python setup.py install

22​ pip install .
View Options

conf/namespaces.yaml.docker

  • This file was added.
1​namespaces_safe:
2​ - scratch
3​ - qa
4​ - pkg
5​ - dist
jsedlakAuthorUnsubmitted
Not Done

This is necessary because how our dockerized buildbot master works - it git clones from local filesystem, so these values are shown as origin url. We might consider changing buildbot master so that it copies instead of cloning (we could then just usenamespaces.yaml.example) or consider dropping this completely.

jskladanUnsubmitted
Not Done

Honestly this will need to change quite a lot anyway in the future - this solution (I mean as a whole in libtaskotron) is sub-par. For the scope of the docker stuff, I'd just put all the relevant namespaces to the namespaces_safe list.

View Options

conf/taskotron.yaml.docker

  • This file was added.
1​profile: production
2​runtask_mode: local
3​resultsdb_server: http://resultsdb:5001/api/v2.0
4​resultsdb_frontend: http://localhost:5002
5​execdb_server: http://localhost:5003
6​taskotron_master: http://localhost:8010
7​artifacts_baseurl: http://localhost:8081
jskladanUnsubmitted
Not Done

Given the rest of the urls, you probably wanted localhost instead of resultsdb.
But it does not really matter, honesly, because for the CI thing, we will need to unify the urls anyway, and I'd go with resultsdb, execdb. etc anyway.
We'll discuss this more in person next week.