Differential D989

use compose repository (not master repo) for most tests
ClosedPublic
Actions

Authored by adamwill on Aug 31 2016, 1:29 AM.

Details

Reviewers
garretraziel
jskladan
Commits
rOPENQATESTSef689e75a93f: use compose repository (not master repo) for most tests
Summary

we have a long-standing problem with all the tests that hit
the repositories. The tests are triggered as soon as a compose
completes. At this point in time, the compose is not synced to
the mirrors, where the default 'fedora' repo definition looks;
the sync happens after the compose completes, and there is also
a metadata sync step that must happen after *that* before any
operation that uses the 'fedora' repository definition will
actually use the packages from the new compose. Thus all net
install tests and tests that installed packages have been
effectively testing the previous compose, not the current one.

We have some thoughts about how to fix this 'properly' (such
that the openQA tests wouldn't have to do anything special,
but their 'fedora' repository would somehow reflect the compose
under test), but none of them is in place right now or likely
to happen in the short term, so in the mean time this should
deal with most of the issues. With this change, everything but
the default_install tests for the netinst images should use
the compose-under-test's Everything tree instead of the 'fedora'
repository, and thus should install and test the correct
packages.

This relies on a corresponding change to openqa_fedora_tools
to set the LOCATION openQA setting (which is simply the base
location of the compose under test).

Test Plan

Do a full test run, check (as far as you can) tests run sensibly
and use appropriate repositories.

Diff Detail

Repository
rOPENQATESTS os-autoinst-distri-fedora
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
adamwill retitled this revision from to use compose repository (not master repo) for most tests.Aug 31 2016, 1:29 AM
adamwill updated this object.
adamwill edited the test plan for this revision. (Show Details)
adamwill added reviewers: jskladan, garretraziel.
adamwill updated this revision to Diff 2524.Sep 1 2016, 7:04 AM

I, uh, don't remember, but just making sure this is synced

garretraziel accepted this revision.Sep 1 2016, 2:10 PM
This revision is now accepted and ready to land.Sep 1 2016, 2:10 PM
Closed by commit rOPENQATESTSef689e75a93f: use compose repository (not master repo) for most tests (authored by adamwill). · Explain WhySep 1 2016, 3:23 PM
This revision was automatically updated to reflect the committed changes.
adamwill mentioned this in rOPENQA5cb778a66dff: pass compose location into openQA.

Revision Contents

Diff 2525

View Options

lib/anacondatest.pm

Show First 20 Lines • Show All 192 Lines • ▼ Show 20 Line(s)
193} 193​}
194 194​
195sub get_full_repo { 195​sub get_full_repo {
196 my ($self, $repourl) = @_; 196​ my ($self, $repourl) = @_;
197 # trivial thing we kept repeating: fill out an HTTP or HTTPS 197​ # trivial thing we kept repeating: fill out an HTTP or HTTPS
198 # repo URL with flavor and arch, leave NFS ones alone (as for 198​ # repo URL with flavor and arch, leave NFS ones alone (as for
199 # NFS tests we just use a mounted ISO and the URL is complete) 199​ # NFS tests we just use a mounted ISO and the URL is complete)
200 if ($repourl !~ m/^nfs/) { 200​ if ($repourl !~ m/^nfs/) {
201 $repourl .= "/".lc(get_var("VERSION"))."/Everything/".get_var("ARCH")."/os"; 201​ $repourl .= "/Everything/".get_var("ARCH")."/os";
202 } 202​ }
203 return $repourl; 203​ return $repourl;
204} 204​}
205 205​
206sub get_mirrorlist_url { 206​sub get_mirrorlist_url {
207 return "mirrors.fedoraproject.org/mirrorlist?repo=fedora-" . lc(get_var("VERSION")) . "&arch=" . get_var('ARCH') 207​ return "mirrors.fedoraproject.org/mirrorlist?repo=fedora-" . lc(get_var("VERSION")) . "&arch=" . get_var('ARCH')
208} 208​}
209 209​
2101; 210​1;
211 211​
212# vim: set sw=4 et: 212​# vim: set sw=4 et:
View Options

lib/installedtest.pm

Show First 20 Lines • Show All 81 Lines • ▼ Show 20 Line(s)81 if ($login) {
82 type_string "root"; 82​ type_string "root";
83 send_key "tab"; 83​ send_key "tab";
84 type_string get_var("ROOT_PASSWORD", "weakpassword"); 84​ type_string get_var("ROOT_PASSWORD", "weakpassword");
85 send_key "ret"; 85​ send_key "ret";
86 assert_screen "cockpit_main"; 86​ assert_screen "cockpit_main";
87 } 87​ }
88} 88​}
89 89​
90​sub repo_setup {
91​ # disable updates-testing and use the compose location rather than
92​ # mirrorlist, so we're testing the right packages
93​ my $location = get_var("LOCATION");
94​ assert_script_run 'dnf config-manager --set-disabled updates-testing';
95​ # we use script_run here as the rawhide repo file won't always exist
96​ # and we don't want to bother testing or predicting its existence;
97​ # assert_script_run doesn't buy you much with sed anyway as it'll
98​ # return 0 even if it replaced nothing
99​ script_run "sed -i -e 's,^metalink,#metalink,g' -e 's,^#baseurl.*basearch,baseurl=${location}/Everything/\$basearch,g' /etc/yum.repos.d/{fedora,fedora-rawhide}.repo";
100​ script_run "cat /etc/yum.repos.d/{fedora,fedora-rawhide}.repo";
101​}
102​
901; 103​1;
91 104​
92# vim: set sw=4 et: 105​# vim: set sw=4 et:
View Options

needles/anaconda/install_source/https_selected.json

1{ 1​{
2​ "area": [
3​ {
4​ "height": 19,
5​ "type": "match",
6​ "width": 105,
7​ "xpos": 56,
8​ "ypos": 303
9​ }
10​ ],
2 "properties": [], 11​ "properties": [],
3 "tags": [ 12​ "tags": [
4 "anaconda_install_source_selected", 13​ "anaconda_install_source_selected",
5 "ENV-DISTRI-fedora", 14​ "ENV-DISTRI-fedora",
6 "ENV-FLAVOR-server" 15​ "ENV-FLAVOR-server"
7 ], ​
8 "area": [ ​
9 { ​
10 "width": 115, ​
11 "xpos": 56, ​
12 "height": 21, ​
13 "type": "match", ​
14 "ypos": 163 ​
15 } ​
16 ] 16​ ]
17} 17​}
18​No newline at end of file
View Options

needles/anaconda/install_source/https_selected.png

  • This is a binary file.
View Options

templates

Show First 20 Lines • Show All 1475 Lines • ▼ Show 20 Line(s)1475 {
1476 name => "install_mirrorlist_graphical", 1476​ name => "install_mirrorlist_graphical",
1477 settings => [ 1477​ settings => [
1478 { key => "MIRRORLIST_GRAPHICAL", value => "1" }, 1478​ { key => "MIRRORLIST_GRAPHICAL", value => "1" },
1479 ], 1479​ ],
1480 }, 1480​ },
1481 { 1481​ {
1482 name => "install_repository_http_graphical", 1482​ name => "install_repository_http_graphical",
1483 settings => [ 1483​ settings => [
1484 { key => "REPOSITORY_GRAPHICAL", value => "http://dl.fedoraproject.org/pub/fedora/linux/development" }, 1484​ { key => "REPOSITORY_GRAPHICAL", value => "%LOCATION%" },
1485 ], 1485​ ],
1486 }, 1486​ },
1487 { 1487​ {
1488 name => "install_repository_nfs_graphical", 1488​ name => "install_repository_nfs_graphical",
1489 settings => [ 1489​ settings => [
1490 { key => "REPOSITORY_GRAPHICAL", value => "nfs:10.0.2.110:/repo" }, 1490​ { key => "REPOSITORY_GRAPHICAL", value => "nfs:10.0.2.110:/repo" },
1491 { key => "PARALLEL_WITH", value => "support_server" }, 1491​ { key => "PARALLEL_WITH", value => "support_server" },
1492 { key => "INSTALL_UNLOCK", value => "support_ready" }, 1492​ { key => "INSTALL_UNLOCK", value => "support_ready" },
1493 { key => "NICTYPE", value => "tap" }, 1493​ { key => "NICTYPE", value => "tap" },
1494 { key => "WORKER_CLASS", value => "tap" }, 1494​ { key => "WORKER_CLASS", value => "tap" },
1495 ], 1495​ ],
1496 }, 1496​ },
1497 { 1497​ {
1498 name => "install_repository_http_variation", 1498​ name => "install_repository_http_variation",
1499 settings => [ 1499​ settings => [
1500 { key => "REPOSITORY_VARIATION", value => "http://dl.fedoraproject.org/pub/fedora/linux/development" }, 1500​ { key => "REPOSITORY_VARIATION", value => "%LOCATION%" },
1501 ], 1501​ ],
1502 }, 1502​ },
1503 { 1503​ {
1504 name => "install_repository_nfs_variation", 1504​ name => "install_repository_nfs_variation",
1505 settings => [ 1505​ settings => [
1506 { key => "REPOSITORY_VARIATION", value => "nfs:10.0.2.110:/repo" }, 1506​ { key => "REPOSITORY_VARIATION", value => "nfs:10.0.2.110:/repo" },
1507 { key => "PARALLEL_WITH", value => "support_server" }, 1507​ { key => "PARALLEL_WITH", value => "support_server" },
1508 { key => "INSTALL_UNLOCK", value => "support_ready" }, 1508​ { key => "INSTALL_UNLOCK", value => "support_ready" },
▲ Show 20 Lines • Show All 283 Lines • ▼ Show 20 Line(s)1785 settings => [
1792 { key => "WORKER_CLASS", value => "tap" }, 1792​ { key => "WORKER_CLASS", value => "tap" },
1793 ], 1793​ ],
1794 }, 1794​ },
1795 { 1795​ {
1796 name => "install_package_set_kde", 1796​ name => "install_package_set_kde",
1797 settings => [ 1797​ settings => [
1798 { key => "DESKTOP", value => "kde" }, 1798​ { key => "DESKTOP", value => "kde" },
1799 { key => "PACKAGE_SET", value => "kde" }, 1799​ { key => "PACKAGE_SET", value => "kde" },
1800 { key => "REPOSITORY_VARIATION", value => "http://dl.fedoraproject.org/pub/fedora/linux/development" }, 1800​ { key => "REPOSITORY_VARIATION", value => "%LOCATION%" },
1801 { key => "USER_LOGIN", value => "false" }, 1801​ { key => "USER_LOGIN", value => "false" },
1802 ], 1802​ ],
1803 }, 1803​ },
1804 { 1804​ {
1805 name => "base_selinux", 1805​ name => "base_selinux",
1806 settings => [ 1806​ settings => [
1807 { key => "ENTRYPOINT", value => "base_selinux" }, 1807​ { key => "ENTRYPOINT", value => "base_selinux" },
1808 { key => "START_AFTER_TEST", value => "install_default_upload" }, 1808​ { key => "START_AFTER_TEST", value => "install_default_upload" },
▲ Show 20 Lines • Show All 186 Lines • Show Last 20 Lines
View Options

tests/freeipa_password_change_postinstall.pm

Show All 35 Lines6sub run {
36 # we don't get back to a prompt instantly and keystrokes while X 36​ # we don't get back to a prompt instantly and keystrokes while X
37 # is still shutting down are swallowed, so wait_still_screen before 37​ # is still shutting down are swallowed, so wait_still_screen before
38 # finishing (and handing off to freeipa_client_postinstall) 38​ # finishing (and handing off to freeipa_client_postinstall)
39 wait_still_screen 5; 39​ wait_still_screen 5;
40 # check we can kinit with changed password 40​ # check we can kinit with changed password
41 assert_script_run 'printf "loremipsum" | kinit test1'; 41​ assert_script_run 'printf "loremipsum" | kinit test1';
42 # change password via CLI (back to batterystaple, as that's what 42​ # change password via CLI (back to batterystaple, as that's what
43 # freeipa_client test expects) 43​ # freeipa_client test expects)
44 assert_script_run 'dnf -y --nogpgcheck install freeipa-admintools'; 44​ assert_script_run 'dnf -y install freeipa-admintools';
45 assert_script_run 'printf "batterystaple\nbatterystaple" | ipa user-mod test1 --password'; 45​ assert_script_run 'printf "batterystaple\nbatterystaple" | ipa user-mod test1 --password';
46 # check we can kinit again 46​ # check we can kinit again
47 assert_script_run 'printf "batterystaple" | kinit test1'; 47​ assert_script_run 'printf "batterystaple" | kinit test1';
48 # we just stay here - freeipa_client will pick right up 48​ # we just stay here - freeipa_client will pick right up
49} 49​}
50 50​
51sub test_flags { 51​sub test_flags {
52 # without anything - rollback to 'lastgood' snapshot if failed 52​ # without anything - rollback to 'lastgood' snapshot if failed
53 # 'fatal' - whole test suite is in danger if this fails 53​ # 'fatal' - whole test suite is in danger if this fails
54 # 'milestone' - after this test succeeds, update 'lastgood' 54​ # 'milestone' - after this test succeeds, update 'lastgood'
55 # 'important' - if this fails, set the overall state to 'fail' 55​ # 'important' - if this fails, set the overall state to 'fail'
56 return {}; 56​ return {};
57} 57​}
58 58​
591; 59​1;
View Options

tests/install_source_graphical.pm

1use base "anacondatest"; 1​use base "anacondatest";
2use strict; 2​use strict;
3use testapi; 3​use testapi;
4 4​
5sub run { 5​sub run {
6 my $self = shift; 6​ my $self = shift;
7 # Anaconda hub 7​ # Anaconda hub
8 assert_screen "anaconda_main_hub", 300; # 8​ assert_screen "anaconda_main_hub", 300; #
9 9​
10 # Go into the Install Source spoke 10​ # Go into the Install Source spoke
11 assert_and_click "anaconda_main_hub_installation_source"; 11​ assert_and_click "anaconda_main_hub_installation_source";
12 12​
13 # select appropriate protocol on the network 13​ # select appropriate protocol on the network
14 assert_and_click "anaconda_install_source_on_the_network"; 14​ assert_and_click "anaconda_install_source_on_the_network";
15 send_key "tab"; 15​ send_key "tab";
16 # if we have an NFS repo select NFS (one 'up'), otherwise HTTP (four 'ups') 16​ # if we have an NFS repo select NFS (one 'up'), otherwise HTTPS (three 'ups')
17 my $num; 17​ my $num;
18 $num = get_var("REPOSITORY_GRAPHICAL") =~ m/^nfs:/ ? 1 : 4; 18​ $num = get_var("REPOSITORY_GRAPHICAL") =~ m/^nfs:/ ? 1 : 3;
19 for (my $i=0; $i<$num; $i++){ 19​ for (my $i=0; $i<$num; $i++){
20 send_key "up"; 20​ send_key "up";
21 } 21​ }
22 # let's just accept either NFS or HTTP here, if it's the wrong one the 22​ # let's just accept either NFS or HTTP here, if it's the wrong one the
23 # test will fail soon anyhow 23​ # test will fail soon anyhow
24 assert_screen "anaconda_install_source_selected"; 24​ assert_screen "anaconda_install_source_selected";
25 25​
26 26​
Show All 37 Lines
View Options

tests/realmd_join_sssd_postinstall.pm

Show All 11 Lines6sub run {
12 # set up networking 12​ # set up networking
13 $self->setup_tap_static("10.0.2.103", "client003.domain.local"); 13​ $self->setup_tap_static("10.0.2.103", "client003.domain.local");
14 # use FreeIPA server as DNS server 14​ # use FreeIPA server as DNS server
15 assert_script_run "printf 'search domain.local\nnameserver 10.0.2.100' > /etc/resolv.conf"; 15​ assert_script_run "printf 'search domain.local\nnameserver 10.0.2.100' > /etc/resolv.conf";
16 # wait for the server to be ready (do it now just to make sure name 16​ # wait for the server to be ready (do it now just to make sure name
17 # resolution is working before we proceed) 17​ # resolution is working before we proceed)
18 mutex_lock "freeipa_ready"; 18​ mutex_lock "freeipa_ready";
19 mutex_unlock "freeipa_ready"; 19​ mutex_unlock "freeipa_ready";
20 # disable updates-testing 20​ # use compose repo, disable u-t, etc.
21 assert_script_run 'dnf config-manager --set-disabled updates-testing'; 21​ $self->repo_setup();
22 # do the enrolment 22​ # do the enrolment
23 assert_script_run "echo 'monkeys123' | realm join --user=admin ipa001.domain.local", 300; 23​ assert_script_run "echo 'monkeys123' | realm join --user=admin ipa001.domain.local", 300;
24} 24​}
25 25​
26sub test_flags { 26​sub test_flags {
27 # without anything - rollback to 'lastgood' snapshot if failed 27​ # without anything - rollback to 'lastgood' snapshot if failed
28 # 'fatal' - whole test suite is in danger if this fails 28​ # 'fatal' - whole test suite is in danger if this fails
29 # 'milestone' - after this test succeeds, update 'lastgood' 29​ # 'milestone' - after this test succeeds, update 'lastgood'
30 # 'important' - if this fails, set the overall state to 'fail' 30​ # 'important' - if this fails, set the overall state to 'fail'
31 return { fatal => 1 }; 31​ return { fatal => 1 };
32} 32​}
33 33​
341; 34​1;
35 35​
36# vim: set sw=4 et: 36​# vim: set sw=4 et:
View Options

tests/role_deploy_domain_controller.pm

Show All 14 Lines7sub run {
15 # clone host's /etc/hosts (for phx2 internal routing to work) 15​ # clone host's /etc/hosts (for phx2 internal routing to work)
16 # must come *before* setup_tap_static or else it would overwrite 16​ # must come *before* setup_tap_static or else it would overwrite
17 # its changes 17​ # its changes
18 $self->clone_host_file("/etc/hosts"); 18​ $self->clone_host_file("/etc/hosts");
19 # set up networking 19​ # set up networking
20 $self->setup_tap_static("10.0.2.100", "ipa001.domain.local"); 20​ $self->setup_tap_static("10.0.2.100", "ipa001.domain.local");
21 # clone host's resolv.conf to get name resolution 21​ # clone host's resolv.conf to get name resolution
22 $self->clone_host_file("/etc/resolv.conf"); 22​ $self->clone_host_file("/etc/resolv.conf");
23 # we don't want updates-testing for validation purposes 23​ # use compose repo, disable u-t, etc.
24 assert_script_run 'dnf config-manager --set-disabled updates-testing'; 24​ $self->repo_setup();
25 # we need a lot of entropy for this, and we don't care how good 25​ # we need a lot of entropy for this, and we don't care how good
26 # it is, so let's use haveged 26​ # it is, so let's use haveged
27 assert_script_run 'dnf -y --nogpgcheck install haveged', 300; 27​ assert_script_run 'dnf -y install haveged', 300;
28 assert_script_run 'systemctl start haveged.service'; 28​ assert_script_run 'systemctl start haveged.service';
29 # read DNS server IPs from host's /etc/resolv.conf for passing to 29​ # read DNS server IPs from host's /etc/resolv.conf for passing to
30 # rolectl 30​ # rolectl
31 my @forwards = $self->get_host_dns(); 31​ my @forwards = $self->get_host_dns();
32 # we are now gonna work around a stupid bug in rolekit. we want to 32​ # we are now gonna work around a stupid bug in rolekit. we want to
33 # pass it a list of ipv4 DNS forwarders and have no ipv6 DNS 33​ # pass it a list of ipv4 DNS forwarders and have no ipv6 DNS
34 # forwarders. but it won't allow you to have a dns_forwarders array 34​ # forwarders. but it won't allow you to have a dns_forwarders array
35 # with a "ipv4" list but no "ipv6" list, any values in the "ipv6" 35​ # with a "ipv4" list but no "ipv6" list, any values in the "ipv6"
▲ Show 20 Lines • Show All 82 Lines • Show Last 20 Lines
View Options

tests/server_cockpit_default_postinstall.pm

1use base "installedtest"; 1​use base "installedtest";
2use strict; 2​use strict;
3use testapi; 3​use testapi;
4 4​
5sub run { 5​sub run {
6 my $self=shift; 6​ my $self = shift;
7 # check cockpit appears to be enabled and running and firewall is setup 7​ # check cockpit appears to be enabled and running and firewall is setup
8 assert_script_run 'systemctl is-enabled cockpit.socket'; 8​ assert_script_run 'systemctl is-enabled cockpit.socket';
9 assert_script_run 'systemctl is-active cockpit.socket'; 9​ assert_script_run 'systemctl is-active cockpit.socket';
10 assert_script_run 'firewall-cmd --query-service cockpit'; 10​ assert_script_run 'firewall-cmd --query-service cockpit';
11 # we don't want updates-testing for validation purposes 11​ # use compose repo, disable u-t, etc.
12 assert_script_run 'dnf config-manager --set-disabled updates-testing'; 12​ $self->repo_setup();
13 # install a desktop and firefox so we can actually try it 13​ # install a desktop and firefox so we can actually try it
14 assert_script_run 'dnf -y --nogpgcheck groupinstall "base-x"', 300; 14​ assert_script_run 'dnf -y groupinstall "base-x"', 300;
15 assert_script_run 'dnf -y --nogpgcheck install firefox', 120; 15​ assert_script_run 'dnf -y install firefox', 120;
16 $self->start_cockpit(0); 16​ $self->start_cockpit(0);
17 # quit firefox (return to console) 17​ # quit firefox (return to console)
18 send_key "ctrl-q"; 18​ send_key "ctrl-q";
19} 19​}
20 20​
21sub test_flags { 21​sub test_flags {
22 # without anything - rollback to 'lastgood' snapshot if failed 22​ # without anything - rollback to 'lastgood' snapshot if failed
23 # 'fatal' - whole test suite is in danger if this fails 23​ # 'fatal' - whole test suite is in danger if this fails
24 # 'milestone' - after this test succeeds, update 'lastgood' 24​ # 'milestone' - after this test succeeds, update 'lastgood'
25 # 'important' - if this fails, set the overall state to 'fail' 25​ # 'important' - if this fails, set the overall state to 'fail'
26 return { fatal => 1 }; 26​ return { fatal => 1 };
27} 27​}
28 28​
291; 29​1;
30 30​
31# vim: set sw=4 et: 31​# vim: set sw=4 et:
View Options

tests/upgrade_run.pm

1use base "installedtest"; 1​use base "installedtest";
2use strict; 2​use strict;
3use testapi; 3​use testapi;
4 4​
5sub run { 5​sub run {
6 my $self = shift; 6​ my $self = shift;
7 my $release = lc(get_var("VERSION")); 7​ my $release = lc(get_var("VERSION"));
8 # disable screen blanking (download can take a long time) 8​ # disable screen blanking (download can take a long time)
9 script_run "setterm -blank 0"; 9​ script_run "setterm -blank 0";
10 10​
11 assert_script_run "dnf -y --nogpgcheck --releasever=${release} system-upgrade download", 6000; 11​ # use compose repo
12​ $self->repo_setup();
13​ my $params = "-y --releasever=${release}";
14​ if ($release eq "rawhide") {
15​ $params .= " --nogpgcheck";
16​ }
17​ assert_script_run "dnf ${params} system-upgrade download", 6000;
12 18​
13 upload_logs "/var/log/dnf.log"; 19​ upload_logs "/var/log/dnf.log";
14 upload_logs "/var/log/dnf.rpm.log"; 20​ upload_logs "/var/log/dnf.rpm.log";
15 21​
16 script_run "dnf system-upgrade reboot"; 22​ script_run "dnf system-upgrade reboot";
17 # fail immediately if we see a DNF error message 23​ # fail immediately if we see a DNF error message
18 die "DNF reported failure" if (check_screen "upgrade_fail", 15); 24​ die "DNF reported failure" if (check_screen "upgrade_fail", 15);
19 if (get_var("ENCRYPT_PASSWORD")) { 25​ if (get_var("ENCRYPT_PASSWORD")) {
Show All 22 Lines