Differential D922

add encrypted workstation upgrade tests (current and previous)
ClosedPublic
Actions

Authored by adamwill on Jul 5 2016, 8:00 AM.

Details

Reviewers
garretraziel
Commits
rOPENQATESTSaacd01ea8b4e: add encrypted workstation upgrade tests (current and previous)
Summary

This requires us to handle decryption each time we reboot in
the upgrade process, so factor that little block out into the
base class so we don't have to keep pasting it. It's also a
bit tricky to integrate into the 'catch a boot loop' code we
have to deal with #1349721, but I think this should work. There
is a matching openqa_fedora_tools diff to generate the disk
image.

Test Plan

Run the tests, check that they work, run the other
upgrade and encrypted install tests and check they still work
properly too.

Diff Detail

Repository
rOPENQATESTS os-autoinst-distri-fedora
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
adamwill retitled this revision from to add encrypted workstation upgrade tests (current and previous).Jul 5 2016, 8:00 AM
adamwill updated this object.
adamwill edited the test plan for this revision. (Show Details)
adamwill added a reviewer: garretraziel.
garretraziel accepted this revision.Jul 8 2016, 10:22 AM

I wasn't able to test this thoroughly, because createhdds is building F23 and F24 images for me (and I naturally cannot upgrade F24 disk), so I've tested only upgrading F23 -> F24 but it works without problem.

tests/disk_guided_encrypted_postinstall.pm
10

Long waiting on two different places isn't nice, but I guess that there's no way around.

This revision is now accepted and ready to land.Jul 8 2016, 10:22 AM
adamwill added a comment.Jul 8 2016, 3:54 PM

How do you mean you 'naturally' cannot upgrade f24 disks? Building f23 and f24 is correct, that's what it's intended to do ATM, because current tests are for post-f24 Rawhide. The prod tests at the moment test upgrading from f23->rawhide and f24->rawhide, and I don't see why you can't test that too?

adamwill added inline comments.Jul 8 2016, 3:56 PM
tests/disk_guided_encrypted_postinstall.pm
10

it doesn't actually do a long wait in both places. note that I also changed _graphical_wait_login (it's just above, in Phab) so the wait_time there will only be 300 for encrypted upgrades; I tweaked it so it only gets extended to 6000 if UPGRADE is set *and `ENCRYPT_PASSWORD is not*.

Closed by commit rOPENQATESTSaacd01ea8b4e: add encrypted workstation upgrade tests (current and previous) (authored by adamwill). · Explain WhyJul 8 2016, 3:57 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

Diff 2363

View Options

lib/fedorabase.pm

Show First 20 Lines • Show All 218 Lines • ▼ Show 20 Line(s)215sub get_host_dns {
219 while (<FH>) { 219​ while (<FH>) {
220 if ($_ =~ m/^nameserver +(.+)/) { 220​ if ($_ =~ m/^nameserver +(.+)/) {
221 push @forwards, $1; 221​ push @forwards, $1;
222 } 222​ }
223 } 223​ }
224 return @forwards; 224​ return @forwards;
225} 225​}
226 226​
227​sub boot_decrypt {
228​ # decrypt storage during boot; arg is timeout (in seconds)
229​ my $self = shift;
230​ my $timeout = shift || 60;
231​ assert_screen "boot_enter_passphrase", $timeout; #
232​ type_string get_var("ENCRYPT_PASSWORD");
233​ send_key "ret";
234​}
235​
2271; 236​1;
228 237​
229# vim: set sw=4 et: 238​# vim: set sw=4 et:
View Options

needles/console/boot_enter_passphrase-20160705.json

  • This file was added.
1​{
2​ "area": [
3​ {
4​ "width": 207,
5​ "xpos": 0,
6​ "ypos": 169,
7​ "type": "match",
8​ "height": 31
9​ }
10​ ],
11​ "tags": [
12​ "boot_enter_passphrase"
13​ ],
14​ "properties": []
15​}
16​No newline at end of file
View Options

needles/console/boot_enter_passphrase-20160705.png

  • This binary file was added.
View Options

needles/console/boot_enter_passphrase-graphical.json

  • This file was added.
1​{
2​ "area": [
3​ {
4​ "xpos": 395,
5​ "ypos": 362,
6​ "width": 33,
7​ "height": 40,
8​ "type": "match"
9​ }
10​ ],
11​ "properties": [],
12​ "tags": [
13​ "boot_enter_passphrase"
14​ ]
15​}
16​No newline at end of file
View Options

needles/console/boot_enter_passphrase-graphical.png

  • This binary file was added.
View Options

templates

Show First 20 Lines • Show All 805 Lines • ▼ Show 20 Line(s)804 product => {
806 distri => "fedora", 806​ distri => "fedora",
807 flavor => "universal", 807​ flavor => "universal",
808 version => "*", 808​ version => "*",
809 }, 809​ },
810 test_suite => { name => "upgrade_kde_64bit" }, 810​ test_suite => { name => "upgrade_kde_64bit" },
811 }, 811​ },
812 { 812​ {
813 machine => { name => "64bit" }, 813​ machine => { name => "64bit" },
814​ prio => 30,
815​ product => {
816​ arch => "x86_64",
817​ distri => "fedora",
818​ flavor => "universal",
819​ version => "*",
820​ },
821​ test_suite => { name => "upgrade_desktop_encrypted_64bit" },
822​ },
823​ {
824​ machine => { name => "64bit" },
814 prio => 50, 825​ prio => 50,
815 product => { 826​ product => {
816 arch => "x86_64", 827​ arch => "x86_64",
817 distri => "fedora", 828​ distri => "fedora",
818 flavor => "universal", 829​ flavor => "universal",
819 version => "*", 830​ version => "*",
820 }, 831​ },
821 test_suite => { name => "upgrade_2_minimal_64bit" }, 832​ test_suite => { name => "upgrade_2_minimal_64bit" },
Show All 28 Lines859 product => {
850 distri => "fedora", 861​ distri => "fedora",
851 flavor => "universal", 862​ flavor => "universal",
852 version => "*", 863​ version => "*",
853 }, 864​ },
854 test_suite => { name => "upgrade_2_kde_64bit" }, 865​ test_suite => { name => "upgrade_2_kde_64bit" },
855 }, 866​ },
856 { 867​ {
857 machine => { name => "64bit" }, 868​ machine => { name => "64bit" },
869​ prio => 50,
870​ product => {
871​ arch => "x86_64",
872​ distri => "fedora",
873​ flavor => "universal",
874​ version => "*",
875​ },
876​ test_suite => { name => "upgrade_2_desktop_encrypted_64bit" },
877​ },
878​ {
879​ machine => { name => "64bit" },
858 prio => 40, 880​ prio => 40,
859 product => { 881​ product => {
860 arch => "x86_64", 882​ arch => "x86_64",
861 distri => "fedora", 883​ distri => "fedora",
862 flavor => "universal", 884​ flavor => "universal",
863 version => "*", 885​ version => "*",
864 }, 886​ },
865 test_suite => { name => "install_updates_img_local" }, 887​ test_suite => { name => "install_updates_img_local" },
▲ Show 20 Lines • Show All 707 Lines • ▼ Show 20 Line(s)1592 settings => [
1573 { key => "USER_PASSWORD", value => "weakpassword" }, 1595​ { key => "USER_PASSWORD", value => "weakpassword" },
1574 { key => "BOOTFROM", value => "c" }, 1596​ { key => "BOOTFROM", value => "c" },
1575 { key => "HDD_1", value => "disk_f%CURRREL%_kde_3_x86_64.img" }, 1597​ { key => "HDD_1", value => "disk_f%CURRREL%_kde_3_x86_64.img" },
1576 { key => "UPGRADE", value => "1" }, 1598​ { key => "UPGRADE", value => "1" },
1577 { key => "DESKTOP", value => "kde" }, 1599​ { key => "DESKTOP", value => "kde" },
1578 ], 1600​ ],
1579 }, 1601​ },
1580 { 1602​ {
1603​ name => "upgrade_desktop_encrypted_64bit",
1604​ settings => [
1605​ { key => "ROOT_PASSWORD", value => "weakpassword" },
1606​ { key => "USER_LOGIN", value => "test" },
1607​ { key => "USER_PASSWORD", value => "weakpassword" },
1608​ { key => "BOOTFROM", value => "c" },
1609​ { key => "HDD_1", value => "disk_f%CURRREL%_desktopencrypt_x86_64.img" },
1610​ { key => "UPGRADE", value => "1" },
1611​ { key => "DESKTOP", value => "gnome" },
1612​ { key => "ENCRYPT_PASSWORD", value => "weakpassword" },
1613​ ],
1614​ },
1615​ {
1581 name => "upgrade_2_minimal_64bit", 1616​ name => "upgrade_2_minimal_64bit",
1582 settings => [ 1617​ settings => [
1583 { key => "ROOT_PASSWORD", value => "weakpassword" }, 1618​ { key => "ROOT_PASSWORD", value => "weakpassword" },
1584 { key => "USER_LOGIN", value => "false" }, 1619​ { key => "USER_LOGIN", value => "false" },
1585 { key => "BOOTFROM", value => "c" }, 1620​ { key => "BOOTFROM", value => "c" },
1586 { key => "HDD_1", value => "disk_f%PREVREL%_minimal_2_x86_64.img" }, 1621​ { key => "HDD_1", value => "disk_f%PREVREL%_minimal_2_x86_64.img" },
1587 { key => "UPGRADE", value => "1" }, 1622​ { key => "UPGRADE", value => "1" },
1588 ], 1623​ ],
Show All 30 Lines1651 settings => [
1619 { key => "USER_PASSWORD", value => "weakpassword" }, 1654​ { key => "USER_PASSWORD", value => "weakpassword" },
1620 { key => "BOOTFROM", value => "c" }, 1655​ { key => "BOOTFROM", value => "c" },
1621 { key => "HDD_1", value => "disk_f%PREVREL%_kde_3_x86_64.img" }, 1656​ { key => "HDD_1", value => "disk_f%PREVREL%_kde_3_x86_64.img" },
1622 { key => "UPGRADE", value => "1" }, 1657​ { key => "UPGRADE", value => "1" },
1623 { key => "DESKTOP", value => "kde" }, 1658​ { key => "DESKTOP", value => "kde" },
1624 ], 1659​ ],
1625 }, 1660​ },
1626 { 1661​ {
1662​ name => "upgrade_2_desktop_encrypted_64bit",
1663​ settings => [
1664​ { key => "ROOT_PASSWORD", value => "weakpassword" },
1665​ { key => "USER_LOGIN", value => "test" },
1666​ { key => "USER_PASSWORD", value => "weakpassword" },
1667​ { key => "BOOTFROM", value => "c" },
1668​ { key => "HDD_1", value => "disk_f%PREVREL%_desktopencrypt_x86_64.img" },
1669​ { key => "UPGRADE", value => "1" },
1670​ { key => "DESKTOP", value => "gnome" },
1671​ { key => "ENCRYPT_PASSWORD", value => "weakpassword" },
1672​ ],
1673​ },
1674​ {
1627 name => "upgrade_desktop_32bit", 1675​ name => "upgrade_desktop_32bit",
1628 settings => [ 1676​ settings => [
1629 { key => "ROOT_PASSWORD", value => "weakpassword" }, 1677​ { key => "ROOT_PASSWORD", value => "weakpassword" },
1630 { key => "USER_LOGIN", value => "test" }, 1678​ { key => "USER_LOGIN", value => "test" },
1631 { key => "USER_PASSWORD", value => "weakpassword" }, 1679​ { key => "USER_PASSWORD", value => "weakpassword" },
1632 { key => "BOOTFROM", value => "c" }, 1680​ { key => "BOOTFROM", value => "c" },
1633 { key => "HDD_1", value => "disk_f%CURRREL%_desktop_3_i686.img" }, 1681​ { key => "HDD_1", value => "disk_f%CURRREL%_desktop_3_i686.img" },
1634 { key => "UPGRADE", value => "1" }, 1682​ { key => "UPGRADE", value => "1" },
▲ Show 20 Lines • Show All 258 Lines • Show Last 20 Lines
View Options

tests/_graphical_wait_login.pm

1use base "installedtest"; 1​use base "installedtest";
2use strict; 2​use strict;
3use testapi; 3​use testapi;
4 4​
5sub run { 5​sub run {
6 # If KICKSTART is set, then the wait_time needs to consider the 6​ # If KICKSTART is set, then the wait_time needs to consider the
7 # install time. if UPGRADE, we have to wait for the entire upgrade 7​ # install time. if UPGRADE, we have to wait for the entire upgrade
8​ # unless ENCRYPT_PASSWORD is set (in which case the postinstall
9​ # test does the waiting)
8 my $wait_time = 300; 10​ my $wait_time = 300;
9 $wait_time = 1800 if (get_var("KICKSTART")); 11​ $wait_time = 1800 if (get_var("KICKSTART"));
10 $wait_time = 6000 if (get_var("UPGRADE")); 12​ $wait_time = 6000 if (get_var("UPGRADE") && !get_var("ENCRYPT_PASSWORD"));
11 13​
12 # Wait for the login screen 14​ # Wait for the login screen
13 assert_screen "graphical_login", $wait_time; 15​ assert_screen "graphical_login", $wait_time;
14 # handle the qemu display buffer sometimes showing the DM from the 16​ # handle the qemu display buffer sometimes showing the DM from the
15 # *previous* boot - https://openqa.stg.fedoraproject.org/tests/17116 17​ # *previous* boot - https://openqa.stg.fedoraproject.org/tests/17116
16 wait_still_screen; 18​ wait_still_screen;
17 assert_screen "graphical_login"; 19​ assert_screen "graphical_login";
18 # do user login unless USER_LOGIN is set to string 'false' 20​ # do user login unless USER_LOGIN is set to string 'false'
▲ Show 20 Lines • Show All 48 Lines • Show Last 20 Lines
View Options

tests/disk_guided_encrypted_postinstall.pm

1use base "installedtest"; 1​use base "installedtest";
2use strict; 2​use strict;
3use testapi; 3​use testapi;
4 4​
5sub run { 5​sub run {
6​ my $self = shift;
7​ my $wait_time = 300;
8​ # if we're running an upgrade, we must wait for the entire upgrade
9​ # process to run
10​ $wait_time = 6000 if (get_var("UPGRADE"));
garretrazielUnsubmitted
Not Done

Long waiting on two different places isn't nice, but I guess that there's no way around.

adamwillAuthorUnsubmitted
Not Done

it doesn't actually do a long wait in both places. note that I also changed _graphical_wait_login (it's just above, in Phab) so the wait_time there will only be 300 for encrypted upgrades; I tweaked it so it only gets extended to 6000 if UPGRADE is set *and `ENCRYPT_PASSWORD is not*.

6 # decrypt disks during boot 11​ # decrypt disks during boot
7 assert_screen "boot_enter_passphrase", 300; # 12​ $self->boot_decrypt($wait_time);
8 type_string get_var("ENCRYPT_PASSWORD"); ​
9 send_key "ret"; ​
10} 13​}
11 14​
12sub test_flags { 15​sub test_flags {
13 # without anything - rollback to 'lastgood' snapshot if failed 16​ # without anything - rollback to 'lastgood' snapshot if failed
14 # 'fatal' - whole test suite is in danger if this fails 17​ # 'fatal' - whole test suite is in danger if this fails
15 # 'milestone' - after this test succeeds, update 'lastgood' 18​ # 'milestone' - after this test succeeds, update 'lastgood'
16 # 'important' - if this fails, set the overall state to 'fail' 19​ # 'important' - if this fails, set the overall state to 'fail'
17 return { fatal => 1 }; 20​ return { fatal => 1 };
18} 21​}
19 22​
201; 23​1;
21 24​
22# vim: set sw=4 et: 25​# vim: set sw=4 et:
View Options

tests/upgrade_preinstall.pm

1use base "installedtest"; 1​use base "installedtest";
2use strict; 2​use strict;
3use testapi; 3​use testapi;
4 4​
5sub run { 5​sub run {
6 my $self = shift; 6​ my $self = shift;
7​ # decrypt disks during boot if necessary
8​ if (get_var("ENCRYPT_PASSWORD")) {
9​ $self->boot_decrypt(60);
10​ }
7 11​
8 # wait for either graphical or text login 12​ # wait for either graphical or text login
9 if (get_var('DESKTOP')) { 13​ if (get_var('DESKTOP')) {
10 $self->boot_to_login_screen("graphical_login", 30, 90); # DM takes time to load 14​ $self->boot_to_login_screen("graphical_login", 30, 90); # DM takes time to load
11 } else { 15​ } else {
12 $self->boot_to_login_screen(); 16​ $self->boot_to_login_screen();
13 } 17​ }
14 # switch to TTY3 for both, graphical and console tests 18​ # switch to TTY3 for both, graphical and console tests
15 $self->root_console(tty=>3); 19​ $self->root_console(tty=>3);
16 # disable screen blanking (update can take a long time) 20​ # disable screen blanking (update can take a long time)
17 script_run "setterm -blank 0"; 21​ script_run "setterm -blank 0";
18 22​
19 # upgrader should be installed on up-to-date system 23​ # upgrader should be installed on up-to-date system
20 24​
21 assert_script_run 'dnf -y update', 1800; 25​ assert_script_run 'dnf -y update', 1800;
22 26​
23 script_run "reboot"; 27​ script_run "reboot";
24 28​
29​ # decrypt if necessary
30​ if (get_var("ENCRYPT_PASSWORD")) {
31​ $self->boot_decrypt(60);
32​ }
33​
25 if (get_var('DESKTOP')) { 34​ if (get_var('DESKTOP')) {
26 $self->boot_to_login_screen("graphical_login", 30, 90); # DM takes time to load 35​ $self->boot_to_login_screen("graphical_login", 30, 90); # DM takes time to load
27 } else { 36​ } else {
28 $self->boot_to_login_screen(); 37​ $self->boot_to_login_screen();
29 } 38​ }
30 $self->root_console(tty=>3); 39​ $self->root_console(tty=>3);
31 40​
32 my $update_command = 'dnf -y install dnf-plugin-system-upgrade'; 41​ my $update_command = 'dnf -y install dnf-plugin-system-upgrade';
Show All 15 Lines
View Options

tests/upgrade_run.pm

Show All 17 Lines5sub run {
18 18​
19 assert_script_run "dnf -y system-upgrade download ${args}", 6000; 19​ assert_script_run "dnf -y system-upgrade download ${args}", 6000;
20 20​
21 upload_logs "/var/log/dnf.log"; 21​ upload_logs "/var/log/dnf.log";
22 upload_logs "/var/log/dnf.rpm.log"; 22​ upload_logs "/var/log/dnf.rpm.log";
23 23​
24 script_run "dnf system-upgrade reboot"; 24​ script_run "dnf system-upgrade reboot";
25 # fail immediately if we see a DNF error message 25​ # fail immediately if we see a DNF error message
26 die "DNF reported failure" if (check_screen "upgrade_fail"); 26​ die "DNF reported failure" if (check_screen "upgrade_fail", 15);
27​ if (get_var("ENCRYPT_PASSWORD")) {
28​ $self->boot_decrypt(60);
29​ }
27 # try and catch if we hit RHBZ #1349721 and work around it 30​ # try and catch if we hit RHBZ #1349721 and work around it
28 if (check_screen "bootloader") { 31​ if (check_screen "bootloader") {
29 # wait some secs for the screen to clear 32​ # wait some secs for the screen to clear
30 sleep 10; 33​ sleep 10;
34​ if (get_var("ENCRYPT_PASSWORD")) {
35​ $self->boot_decrypt(60);
36​ }
31 if (check_screen "bootloader") { 37​ if (check_screen "bootloader") {
32 record_soft_failure; 38​ record_soft_failure;
33 $self->do_bootloader(postinstall=>1, params=>"enforcing=0"); 39​ $self->do_bootloader(postinstall=>1, params=>"enforcing=0");
40​ if (get_var("ENCRYPT_PASSWORD")) {
41​ $self->boot_decrypt(60);
42​ }
43​ }
34 } 44​ }
45​ # in encrypted case we need to wait a bit so postinstall test
46​ # doesn't bogus match on the encryption prompt we just completed
47​ # before it disappears from view
48​ if (get_var("ENCRYPT_PASSWORD")) {
49​ sleep 5;
35 } 50​ }
36} 51​}
37 52​
38 53​
39sub test_flags { 54​sub test_flags {
40 # without anything - rollback to 'lastgood' snapshot if failed 55​ # without anything - rollback to 'lastgood' snapshot if failed
41 # 'fatal' - whole test suite is in danger if this fails 56​ # 'fatal' - whole test suite is in danger if this fails
42 # 'milestone' - after this test succeeds, update 'lastgood' 57​ # 'milestone' - after this test succeeds, update 'lastgood'
43 # 'important' - if this fails, set the overall state to 'fail' 58​ # 'important' - if this fails, set the overall state to 'fail'
44 return { fatal => 1 }; 59​ return { fatal => 1 };
45} 60​}
46 61​
471; 62​1;
48 63​
49# vim: set sw=4 et: 64​# vim: set sw=4 et: