Differential D645

Use bare host as taskotron client with multiple buildslaves
ClosedPublic
Actions

Authored by mkrizek on Nov 11 2015, 3:10 PM.

Details

Reviewers
tflink
Test Plan

Will be tested once playbooks are run.

Diff Detail

Lint
Lint Skipped
Unit
Unit Tests Skipped
mkrizek retitled this revision from to Use bare host as taskotron client with multiple buildslaves.Nov 11 2015, 3:10 PM
mkrizek updated this object.
mkrizek edited the test plan for this revision. (Show Details)
mkrizek added a reviewer: tflink.
mkrizek added a project: infrastructure.
tflink requested changes to this revision.Nov 11 2015, 5:35 PM

looks good for the most part, one concern about ssh keys, though

roles/taskotron/taskotron-client/templates/settings.py.testcloud.j2
30

Doesn't this need to be filled out for ssh access?

This revision now requires changes to proceed.Nov 11 2015, 5:35 PM
mkrizek added inline comments.Nov 11 2015, 11:08 PM
roles/taskotron/taskotron-client/templates/settings.py.testcloud.j2
30

Wow, I wonder what happened here with the diff. Yeah, it's supposed to be "ssh-authorized-keys: {{ buildslave_pubkey }}".

mkrizek updated this revision to Diff 1681.Nov 12 2015, 11:46 AM

Fixing botched diff. Sorry about that.

tflink accepted this revision.Nov 19 2015, 1:04 AM

Hasn't this been pushed to the infra repo already? Either way, LGTM

This revision is now accepted and ready to land.Nov 19 2015, 1:04 AM
mkrizek closed this revision.Nov 19 2015, 8:10 AM

Revision Contents

Diff 1681

View Options

inventory/group_vars/taskotron-dev-client-hosts

Context not available.
2nrpe_procs_warn: 900 2​nrpe_procs_warn: 900
3nrpe_procs_crit: 1000 3​nrpe_procs_crit: 1000
4 4​
5​
6​############################################################
7​# general config
8​############################################################
9​
10​deployment_type: dev
11​tcp_ports: [ "{{ buildslave_port }}" ]
12​sudoers: "{{ private }}/files/sudo/qavirt-sudoers"
13​freezes: false
14​
15​# this enables extra repos during dnf install but that's broken right now, remember to re-enable
16​# extra_enablerepos: 'infrastructure-testing'
17​extra_enablerepos: ''
18​
19​
20​############################################################
21​# buildslave config
22​############################################################
23​
24​slaves_group: buildslaves
25​buildmaster: 10.5.124.181
26​buildslave_port: 9989
27​taskotron_admin_email: taskotron-admin-members@fedoraproject.org
28​
29​
30​############################################################
31​# taskotron config
32​############################################################
33​
34​taskotron_fas_user: taskotron
35​# stuff is kinda not working for stg bodhi right now. workaround by having empty fas password
36​#taskotron_fas_password: '{{ dev_taskotron_fas_password}}'
37​taskotron_fas_password: ''
38​
39​execdb_external_url: https://taskotron-dev.fedoraproject.org/execdb
40​resultsdb_server: http://resultsdb-dev01.qa.fedoraproject.org/resultsdb_api/api/v1.0
41​bodhi_server: https://bodhi.fedoraproject.org/
42​kojihub_url: http://koji.fedoraproject.org/kojihub
43​taskotron_master: https://taskotron-dev.fedoraproject.org/taskmaster
44​resultsdb_external_url: https://taskotron-dev.fedoraproject.org/resultsdb
45​artifacts_base_url: https://taskotron-dev.fedoraproject.org/artifacts
46​
47​client_report_to_bodhi: 'False'
48​
49​
50​############################################################
51​# testcloud config
52​############################################################
53​
54​buildslave_pubkey: "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA4EOTNfPIvIjCLNRYauVquS2LA8CZvCi6f87TASDZv4moFTv3mrCIdA3ycYLg+r+ODdPDjF9Cjpw1os/8L07XegTt9gAJVIpxzu3ZgQDkSQbx/hlDi+RG4EOIHL8nNJmu27dOVd1tb3k3aOkP5twO4uEq9R
55​
56​
57​############################################################
58​# task git mirror config
59​############################################################
60​
61​buildslave_private_sshkey_file: dev-buildslave-sshkey/dev_buildslave
62​buildslave_public_sshkey_file: dev-buildslave-sshkey/dev_buildslave.pub
63​buildmaster_pubkey: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0DR5IwJZ4Fdwj8xe0h36jRKAL2KtCtIJbjwG+hQrnoRDC+l4WtTm4yiE8/HJPSD+022ciRCgcwweoycc9Q/vXqOnUE1ZNnWFQaYU/7HvVblFxtUjF11y+K3h/yXHGCKseAYeaFvaQVBG7UUn0PE57J
Context not available.
View Options

inventory/host_vars/qa11.qa.fedoraproject.org

Context not available.
4sudoers: "{{ private }}/files/sudo/qavirt-sudoers" 4​sudoers: "{{ private }}/files/sudo/qavirt-sudoers"
5nrpe_procs_warn: 900 5​nrpe_procs_warn: 900
6nrpe_procs_crit: 1000 6​nrpe_procs_crit: 1000
7​
8​fas_client_groups: sysadmin-qa,sysadmin-main
9​short_hostname: qa11.qa
10​slaves:
11​ - { user: "{{ short_hostname }}-1", home: "/home/{{ short_hostname }}-1", dir: "/home/{{ short_hostname }}-1/slave" }
12​ - { user: "{{ short_hostname }}-2", home: "/home/{{ short_hostname }}-2", dir: "/home/{{ short_hostname }}-2/slave" }
13​ - { user: "{{ short_hostname }}-3", home: "/home/{{ short_hostname }}-3", dir: "/home/{{ short_hostname }}-3/slave" }
14​ - { user: "{{ short_hostname }}-4", home: "/home/{{ short_hostname }}-4", dir: "/home/{{ short_hostname }}-4/slave" }
Context not available.
View Options

inventory/host_vars/taskotron-dev01.qa.fedoraproject.org

Context not available.
47buildmaster: 10.5.124.181 47​buildmaster: 10.5.124.181
48 48​
49buildslaves: 49​buildslaves:
50 - taskotron-client26-1 50​ - qa11.qa-1
51 - taskotron-client26-2 51​ - qa11.qa-2
52 - taskotron-client27-1 52​ - qa11.qa-3
53 - taskotron-client27-2 53​ - qa11.qa-4
54 - taskotron-client28-1 ​
55 - taskotron-client28-2 ​
56 - taskotron-client29-1 ​
57 - taskotron-client29-2 ​
58i386_buildslaves: 54​i386_buildslaves:
59 - taskotron-client29-1 55​ - qa11.qa-4
60 - taskotron-client29-2 ​
61x86_64_buildslaves: 56​x86_64_buildslaves:
62 - taskotron-client26-1 57​ - qa11.qa-1
63 - taskotron-client26-2 58​ - qa11.qa-2
64 - taskotron-client27-1 59​ - qa11.qa-3
65 - taskotron-client27-2 ​
66 - taskotron-client28-1 ​
67 - taskotron-client28-2 ​
Context not available.
View Options

playbooks/groups/taskotron-client-hosts.yml

Context not available.
34 handlers: 34​ handlers:
35 - include: "{{ handlers }}/restart_services.yml" 35​ - include: "{{ handlers }}/restart_services.yml"
36 36​
37#- name: configure taskotron client host 37​- name: configure taskotron client host
38# hosts: taskotron-dev-client-hosts:taskotron-stg-client-hosts:taskotron-prod-client-hosts 38​ hosts: taskotron-dev-client-hosts:taskotron-stg-client-hosts:taskotron-prod-client-hosts
39# user: root 39​ user: root
40# gather_facts: True 40​ gather_facts: True
41# 41​
42# vars_files: 42​ vars_files:
43# - /srv/web/infra/ansible/vars/global.yml 43​ - /srv/web/infra/ansible/vars/global.yml
44# - "/srv/private/ansible/vars.yml" 44​ - "/srv/private/ansible/vars.yml"
45# - /srv/web/infra/ansible/vars/{{ ansible_distribution }}.yml 45​ - /srv/web/infra/ansible/vars/{{ ansible_distribution }}.yml
46# 46​
47# roles: 47​ roles:
48# - { role: taskotron/something, tags: ['something'] } 48​ - { role: taskotron/buildslave, tags: ['buildslave'] }
49# 49​ - { role: taskotron/taskotron-client, tags: ['taskotronclient'] }
50# handlers: 50​ - { role: taskotron/buildslave-configure, tags: ['buildslaveconfigure'] }
51# - include: "{{ handlers }}/restart_services.yml" 51​
52​ handlers:
53​ - include: "{{ handlers }}/restart_services.yml"
Context not available.
View Options

roles/taskotron/taskotron-client/tasks/main.yml

Context not available.
14 - PyYAML 14​ - PyYAML
15 - libtaskotron 15​ - libtaskotron
16 - resultsdb_api 16​ - resultsdb_api
17​ - testcloud
17 18​
18- name: ensure packages required for taskotron tasks are installed (yum) 19​- name: ensure packages required for taskotron tasks are installed (yum)
19 yum: name={{ item }} state=latest enablerepo={{ extra_enablerepos }} 20​ yum: name={{ item }} state=latest enablerepo={{ extra_enablerepos }}
Context not available.
34- name: generate taskotron.yaml config file 35​- name: generate taskotron.yaml config file
35 template: src=taskotron.yaml.j2 dest=/etc/taskotron/taskotron.yaml owner=root group=root mode=0644 36​ template: src=taskotron.yaml.j2 dest=/etc/taskotron/taskotron.yaml owner=root group=root mode=0644
36 37​
38​- name: generate testcloud config file
39​ template: src=settings.py.testcloud.j2 dest=/etc/testcloud/settings.py owner=root group=root mode=0644
40​
37# hotfixing correct yumrepoinfo until libtaskotron is updated 41​# hotfixing correct yumrepoinfo until libtaskotron is updated
38- name: upload yumrepoinfo.conf 42​- name: upload yumrepoinfo.conf
39 copy: src=yumrepoinfo.conf dest=/etc/taskotron/yumrepoinfo.conf owner=root group=root mode=0644 43​ copy: src=yumrepoinfo.conf dest=/etc/taskotron/yumrepoinfo.conf owner=root group=root mode=0644
Context not available.
View Options

roles/taskotron/taskotron-client/templates/settings.py.testcloud.j2

  • This file was added.
1​# Commented out default values with details are displayed below. To override
2​# these default values, uncomment the values, rename the file to settings.py
3​# and place it in a supported config location
4​
5​# Do not modify this file directly, it will not be picked up unless the filename
6​# is changed to settings.py
7​
8​#DOWNLOAD_PROGRESS = True
9​#LOG_FILE = None
10​
11​# Directories for data and cached downloaded images
12​#DATA_DIR = "/var/lib/testcloud/"
13​#CACHE_DIR = "/var/lib/testcloud/cache"
14​
15​# Data for cloud-init
16​
17​#PASSWORD = 'passw0rd'
18​#HOSTNAME = 'testcloud'
19​
20​#META_DATA = """instance-id: iid-123456
21​#local-hostname: %s
22​#"""
23​USER_DATA = """#cloud-config
24​users:
25​ - default
26​ - name: root
27​ password: %s
28​ chpasswd: { expire: False }
29​ ssh-authorized-keys:
30​ - {{ buildslave_pubkey }}
tflinkUnsubmitted
Not Done

Doesn't this need to be filled out for ssh access?

mkrizekAuthorUnsubmitted
Not Done

Wow, I wonder what happened here with the diff. Yeah, it's supposed to be "ssh-authorized-keys: {{ buildslave_pubkey }}".

31​"""
32​#ATOMIC_USER_DATA = """#cloud-config
33​#password: %s
34​#chpasswd: { expire: False }
35​#ssh_pwauth: True
36​#runcmd:
37​# - [ sh, -c, 'echo -e "ROOT_SIZE=4G\nDATA_SIZE=10G" > /etc/sysconfig/docker-storage-setup']
38​#"""
39​#
40​# Extra cmdline args for the qemu invocation.
41​# Customize as needed :)
42​
43​#CMD_LINE_ARGS = []
View Options

roles/taskotron/taskotron-client/templates/taskotron.yaml.j2

Context not available.
32## Local execution is the default mode for development profile and remote 32​## Local execution is the default mode for development profile and remote
33## execution for production profile. 33​## execution for production profile.
34## [choices: local, libvirt; default: local for development, libvirt for production] 34​## [choices: local, libvirt; default: local for development, libvirt for production]
35​{% if deployment_type == 'dev' %}
36​runtask_mode: libvirt
37​{% elif deployment_type in ['stg', 'prod', 'local'] %}
35runtask_mode: local 38​runtask_mode: local
39​{% endif %}
36 40​
37 41​
38## ==== SCHEDULING section ==== 42​## ==== SCHEDULING section ====
Context not available.