convert upgrade tests to dnf-plugin-system-upgrade
ClosedPublic

Authored by adamwill on Sep 9 2015, 7:47 AM.

Details

Summary

This is a first cut which more or less works for now. Issues:

  1. We're not really testing the BUILD, here. All the test does

is try and upgrade to the specified VERSION - so it'll be using
the latest 'stable' for the given VERSION at the time the test
runs. This isn't really that terrible, but especially for TC/RC
validation, we might want to make things a bit more elaborate
and set up the repo for the actual BUILD (and disable the main
repos).

  1. We'd actually need --nogpgcheck for non-Rawhide, at one

specific point in the release cycle - after Branching but
before Bodhi activation (which is when we can be sure all
packages are signed). This won't matter until 24 branches, and
maybe releng will have it fixed by then...if not, I'll tweak
it.

  1. We don't really test that the upgrade actually *happened*

for desktop, at the moment - the only thing in the old test
that really checked that was where we checked for the fedup
boot menu entry, but that has no analog in dnf. What we should
probably do is check that GUI login works, then switch to a
console and check /etc/fedora-release just as the minimal test
does.

Test Plan

Run the tests. Note that creating the desktop disk
image doesn't work ATM, so I can't verify the desktop test
works, but the minimal one seems to (with D565). There'll be
a matching diff for openqa_fedora_tools to update the test
case names there.

Diff Detail

Repository
rOPENQATESTS os-autoinst-distri-fedora
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
adamwill retitled this revision from to convert upgrade tests to dnf-plugin-system-upgrade.Sep 9 2015, 7:47 AM
adamwill updated this object.
adamwill edited the test plan for this revision. (Show Details)
adamwill added reviewers: jskladan, garretraziel.
jskladan accepted this revision.Sep 9 2015, 8:17 AM
This revision is now accepted and ready to land.Sep 9 2015, 8:17 AM
adamwill updated this revision to Diff 1484.Sep 9 2015, 5:58 PM

tweak needle, nogpgcheck for branched, check version on desktop

The version ident string can apparently vary a bit in Rawhide(?)
so make the needle a bit looser. To cover the pre-Bodhi-enable
phase, use --nogpgcheck for all 'branched' builds (that is, all
branched nightlies). And check that the upgrade actually
happened, in the desktop case.

adamwill updated this revision to Diff 1485.Sep 9 2015, 9:30 PM

more robust installed release check

The 'MOTD' string can change between flavors, so it's not good
to rely on it, and in particular you can't use it to identify
Rawhide Workstation. We could cat /etc/fedora-release, which is
part of fedora-release and doesn't change with the flavors, but
looking at the fedora-release spec, I think this is probably
the most robust way: this value should always be what we want
it to be.

adamwill updated this revision to Diff 1486.Sep 9 2015, 9:31 PM

drop the now-unneeded console_*_installed needles

adamwill updated this revision to Diff 1487.Sep 9 2015, 9:44 PM

fix the branched nogpgcheck check

it's the milestone, not the release.

adamwill updated this revision to Diff 1488.Sep 9 2015, 9:53 PM

document check_release

I hit a couple of problems in testing Workstation upgrades right now: it seems like the GDM "Sign In" button is not blue any more which is breaking that needle match. But it looks like it would work without those glitches. Minimal upgrade from 22 to both 23 and Rawhide works.

adamwill updated this revision to Diff 1489.Sep 10 2015, 12:44 AM

drop login button match from graphical_login_gdm

We don't actually need to match the login button, there's no
reason to check it's there and we're not clicking it (we hit
'enter'). In F23 and F24 the button is not sensitive until
you've typed at least one character of the password, so if we
kept this match, we'd need a new needle. To avoid this, just
drop the area entirely.

garretraziel accepted this revision.Sep 10 2015, 11:38 AM

Yup, seems to work OK.

lib/installedtest.pm
42

I like this, it's smart.

This revision was automatically updated to reflect the committed changes.