Differential D1004

add a desktop notifications test
ClosedPublic
Actions

Authored by adamwill on Sep 16 2016, 1:36 AM.

Details

Reviewers
garretraziel
jskladan
Commits
rOPENQATESTSa067d0655fa2: add a desktop notifications test
Summary

this more or less covers desktop_error_checks and desktop_
update_notification, though it can't really distinguish
between them easily. All we know is that if both the live and
postinstall versions of this test pass, both of those tests
pass. Any fails will have to be investigated manually.

Test Plan

Run the tests for both KDE and Workstation, see
what happens. Workstation will fail for F25 and Rawhide at
present, due to SELinux/abrt notifications.

Diff Detail

Repository
rOPENQATESTS os-autoinst-distri-fedora
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
adamwill retitled this revision from to add a desktop notifications test.Sep 16 2016, 1:36 AM
adamwill updated this object.
adamwill edited the test plan for this revision. (Show Details)
adamwill added reviewers: garretraziel, jskladan.
garretraziel added a comment.Sep 20 2016, 9:55 AM

I'm getting

+++ worker notes +++
end time: 2016-09-20 08:45:13
result: setup failure: /var/lib/openqa/share/factory/hdd/disk_Workstation-live-iso_64bit.qcow2 does not exist!
uploading autoinst-log.txt

for desktop_notifications_postinstall test, but I guess it's problem somewhere in my openQA instance. Have you tried running it on staging?

adamwill added a comment.Sep 20 2016, 4:24 PM

yeah...well, at least, I did for KDE - e.g. https://openqa.stg.fedoraproject.org/tests/42974 - not totally sure if I did on Workstation. But it uses the same settings as desktop_terminal so I can't see how it would go wrong. It could just be that you tested with an image where the default_upload test fails? I don't recall precisely how the children's logs look in that case, and it may differ between openQA versions...

it looks like GNOME 'getting started' is working again in Rawhide today, so I'll do a run there just to check.

garretraziel accepted this revision.Sep 21 2016, 5:27 AM

Then it's settled.

This revision is now accepted and ready to land.Sep 21 2016, 5:27 AM
adamwill added a comment.EditedSep 23 2016, 7:22 AM

I landed this, but there's an issue with the KDE needles that I'll try and fix tomorrow.

Closed by commit rOPENQATESTSa067d0655fa2: add a desktop notifications test (authored by adamwill). · Explain WhySep 23 2016, 11:03 PM
This revision was automatically updated to reflect the committed changes.
adamwill added a comment.Sep 23 2016, 11:33 PM

OK, should be good now. Turns out the systray icons on KDE move all over the place so the 'no notifications' needle for KDE was not safe, but we can get a reliable 'no notifications' check by opening the 'extended' systray and clicking on Notifications. In fact I think we don't strictly speaking need to click on Notifications - simply its presence there implies that there are no fresh notifications, otherwise it's not there - but this seems safest in case upstream changes the interface or anything.

Revision Contents

Diff 2588

View Options

needles/gnome/desktop_no_notifications.json

  • This file was added.
1​{
2​ "area": [
3​ {
4​ "height": 47,
5​ "type": "match",
6​ "width": 47,
7​ "xpos": 345,
8​ "ypos": 249
9​ }
10​ ],
11​ "tags": [
12​ "desktop_no_notifications",
13​ "DESKTOP-gnome"
14​ ]
15​}
View Options

needles/gnome/desktop_no_notifications.png

  • This binary file was added.
View Options

needles/gnome/desktop_update_notification_only.json

  • This file was added.
1​{
2​ "area": [
3​ {
4​ "height": 95,
5​ "type": "match",
6​ "width": 40,
7​ "xpos": 170,
8​ "ypos": 90
9​ }
10​ ],
11​ "tags": [
12​ "desktop_update_notification_only",
13​ "DESKTOP-gnome"
14​ ]
15​}
View Options

needles/gnome/desktop_update_notification_only.png

  • This binary file was added.
View Options

needles/kde/desktop_expand_notifications.json

  • This file was moved to needles/kde/desktop_expand_systray.json.
View Options

needles/kde/desktop_expand_notifications.png

  • This file was moved to needles/kde/desktop_expand_systray.png.
View Options

needles/kde/desktop_expand_systray.json

  • This file was moved from needles/kde/desktop_expand_notifications.json.
1{ 1​{
2 "area": [ 2​ "area": [
3 { 3​ {
4 "xpos": 846, 4​ "xpos": 846,
5 "ypos": 743, 5​ "ypos": 743,
6 "width": 17, 6​ "width": 17,
7 "height": 18, 7​ "height": 18,
8 "type": "match" 8​ "type": "match"
9 } 9​ }
10 ], 10​ ],
11 "tags": [ 11​ "tags": [
12 "DESKTOP-kde", 12​ "DESKTOP-kde",
13 "desktop_expand_notifications" 13​ "desktop_expand_systray"
14 ], 14​ ],
15 "properties": [] 15​ "properties": []
16} 16​}
View Options

needles/kde/desktop_expand_systray.png

  • This file was moved from needles/kde/desktop_expand_notifications.png.
The contents of this file were not changed.
View Options

needles/kde/desktop_no_notifications.json

  • This file was added.
1​{
2​ "area": [
3​ {
4​ "xpos": 651,
5​ "ypos": 436,
6​ "width": 198,
7​ "height": 25,
8​ "type": "match"
9​ }
10​ ],
11​ "tags": [
12​ "desktop_no_notifications"
13​ ],
14​ "properties": []
15​}
16​No newline at end of file
View Options

needles/kde/desktop_no_notifications.png

  • This binary file was added.
View Options

needles/kde/desktop_systray_notifications.json

  • This file was added.
1​{
2​ "properties": [],
3​ "tags": [
4​ "DESKTOP-kde",
5​ "desktop_systray_notifications"
6​ ],
7​ "area": [
8​ {
9​ "xpos": 641,
10​ "ypos": 443,
11​ "width": 83,
12​ "height": 16,
13​ "type": "match"
14​ }
15​ ]
16​}
17​No newline at end of file
View Options

needles/kde/desktop_systray_notifications.png

  • This binary file was added.
View Options

needles/kde/desktop_update_notification_only.json

  • This file was added.
1​{
2​ "area": [
3​ {
4​ "height": 18,
5​ "type": "match",
6​ "width": 18,
7​ "xpos": 663,
8​ "ypos": 743
9​ },
10​ {
11​ "height": 18,
12​ "type": "match",
13​ "width": 18,
14​ "xpos": 767,
15​ "ypos": 743
16​ }
17​ ],
18​ "tags": [
19​ "desktop_update_notification_only",
20​ "DESKTOP-kde"
21​ ]
22​}
View Options

needles/kde/desktop_update_notification_only.png

  • This binary file was added.
View Options

templates

Show First 20 Lines • Show All 369 Lines • ▼ Show 20 Line(s)361 {
370 test_suite => { name => "desktop_browser" }, 370​ test_suite => { name => "desktop_browser" },
371 }, 371​ },
372 { 372​ {
373 machine => { name => "64bit" }, 373​ machine => { name => "64bit" },
374 prio => 30, 374​ prio => 30,
375 product => { 375​ product => {
376 arch => "x86_64", 376​ arch => "x86_64",
377 distri => "fedora", 377​ distri => "fedora",
378​ flavor => "Workstation-live-iso",
379​ version => "*",
380​ },
381​ test_suite => { name => "desktop_notifications_live" },
382​ },
383​ {
384​ machine => { name => "64bit" },
385​ prio => 32,
386​ product => {
387​ arch => "x86_64",
388​ distri => "fedora",
389​ flavor => "KDE-live-iso",
390​ version => "*",
391​ },
392​ test_suite => { name => "desktop_notifications_live" },
393​ },
394​ {
395​ machine => { name => "64bit" },
396​ prio => 30,
397​ product => {
398​ arch => "x86_64",
399​ distri => "fedora",
400​ flavor => "Workstation-live-iso",
401​ version => "*",
402​ },
403​ test_suite => { name => "desktop_notifications_postinstall" },
404​ },
405​ {
406​ machine => { name => "64bit" },
407​ prio => 32,
408​ product => {
409​ arch => "x86_64",
410​ distri => "fedora",
411​ flavor => "KDE-live-iso",
412​ version => "*",
413​ },
414​ test_suite => { name => "desktop_notifications_postinstall" },
415​ },
416​ {
417​ machine => { name => "64bit" },
418​ prio => 30,
419​ product => {
420​ arch => "x86_64",
421​ distri => "fedora",
378 flavor => "universal", 422​ flavor => "universal",
379 version => "*", 423​ version => "*",
380 }, 424​ },
381 test_suite => { name => "install_package_set_minimal" }, 425​ test_suite => { name => "install_package_set_minimal" },
382 }, 426​ },
383 { 427​ {
384 machine => { name => "64bit" }, 428​ machine => { name => "64bit" },
385 prio => 30, 429​ prio => 30,
▲ Show 20 Lines • Show All 1737 Lines • ▼ Show 20 Line(s)2165 {
2123 settings => [ 2167​ settings => [
2124 { key => "POSTINSTALL", value => "desktop_browser" }, 2168​ { key => "POSTINSTALL", value => "desktop_browser" },
2125 { key => "START_AFTER_TEST", value => "install_default_upload" }, 2169​ { key => "START_AFTER_TEST", value => "install_default_upload" },
2126 { key => "BOOTFROM", value => "c" }, 2170​ { key => "BOOTFROM", value => "c" },
2127 { key => "HDD_1", value => "disk_%FLAVOR%_%MACHINE%.qcow2" }, 2171​ { key => "HDD_1", value => "disk_%FLAVOR%_%MACHINE%.qcow2" },
2128 ], 2172​ ],
2129 }, 2173​ },
2130 { 2174​ {
2175​ name => "desktop_notifications_postinstall",
2176​ settings => [
2177​ { key => "POSTINSTALL", value => "desktop_notifications" },
2178​ { key => "START_AFTER_TEST", value => "install_default_upload" },
2179​ { key => "BOOTFROM", value => "c" },
2180​ { key => "HDD_1", value => "disk_%FLAVOR%_%MACHINE%.qcow2" },
2181​ ],
2182​ },
2183​ {
2184​ name => "desktop_notifications_live",
2185​ settings => [
2186​ { key => "ENTRYPOINT", value => "desktop_notifications_postinstall" },
2187​ ],
2188​ },
2189​ {
2131 name => "install_kickstart_nfs", 2190​ name => "install_kickstart_nfs",
2132 settings => [ 2191​ settings => [
2133 { key => "KICKSTART", value => "1" }, 2192​ { key => "KICKSTART", value => "1" },
2134 { key => "GRUB", value => "inst.ks=nfs:10.0.2.110:/export/root-user-crypted-net.ks" }, 2193​ { key => "GRUB", value => "inst.ks=nfs:10.0.2.110:/export/root-user-crypted-net.ks" },
2135 { key => "USER_LOGIN", value => "false" }, 2194​ { key => "USER_LOGIN", value => "false" },
2136 { key => "ROOT_PASSWORD", value => "111111" }, 2195​ { key => "ROOT_PASSWORD", value => "111111" },
2137 { key => "PARALLEL_WITH", value => "support_server" }, 2196​ { key => "PARALLEL_WITH", value => "support_server" },
2138 { key => "INSTALL_UNLOCK", value => "support_ready" }, 2197​ { key => "INSTALL_UNLOCK", value => "support_ready" },
Show All 17 Lines
View Options

tests/desktop_notifications_postinstall.pm

  • This file was added.
1​use base "installedtest";
2​use strict;
3​use testapi;
4​use main_common;
5​use packagetest;
6​
7​# This test sort of covers QA:Testcase_desktop_update_notification
8​# and QA:Testcase_desktop_error_checks . If it fails, probably *one*
9​# of those failed, but we don't know which (deciphering which is
10​# tricky and involves likely-fragile needles to try and figure out
11​# what notifications we have).
12​
13​sub run {
14​ my $self = shift;
15​ # for the live image case, handle bootloader here
16​ unless (get_var("BOOTFROM")) {
17​ $self->do_bootloader(postinstall=>0);
18​ }
19​ assert_screen "graphical_desktop_clean", 300;
20​ # ensure we actually have some package updates available
21​ # we're kinda theoretically racing with the update check here,
22​ # but we have no great way to handle that especially live; let's
23​ # just assume we're gonna win
24​ $self->root_console(tty=>3);
25​ prepare_test_packages;
26​ $self->desktop_vt();
27​ # now, WE WAIT. this is just an unconditional wait - rather than
28​ # breaking if we see an update notification appear - so we catch
29​ # things that crash a few minutes after startup, etc.
30​ for my $n (1..5) {
31​ sleep 120;
32​ mouse_set 10, 10;
33​ mouse_hide;
34​ }
35​ my $desktop = get_var("DESKTOP");
36​ if ($desktop eq 'gnome') {
37​ # of course, we have no idea what'll be in the clock, so we just
38​ # have to click where we know it is
39​ mouse_set 512, 10;
40​ mouse_click;
41​ }
42​ elsif ($desktop eq 'kde' && !get_var("BOOTFROM")) {
43​ # the order and number of systray icons varies in KDE, so we
44​ # can't really just use a systray 'no notifications' needle.
45​ # instead open up the 'extended systray' thingy and click on
46​ # the notifications bit
47​ assert_and_click 'desktop_expand_systray';
48​ assert_and_click 'desktop_systray_notifications';
49​ }
50​ if (get_var("BOOTFROM")) {
51​ # we should see an update notification and no others
52​ check_screen "desktop_update_notification_only";
53​ }
54​ else {
55​ # for the live case there should be *no* notifications
56​ assert_screen "desktop_no_notifications";
57​ }
58​}
59​
60​
61​sub test_flags {
62​ # without anything - rollback to 'lastgood' snapshot if failed
63​ # 'fatal' - whole test suite is in danger if this fails
64​ # 'milestone' - after this test succeeds, update 'lastgood'
65​ # 'important' - if this fails, set the overall state to 'fail'
66​ return { fatal => 1 };
67​}
68​
69​1;
70​
71​# vim: set sw=4 et:
View Options

tests/desktop_update_graphical_postinstall.pm

Show All 10 Lines6sub run {
11 assert_script_run 'dnf config-manager --set-disabled updates-testing'; 11​ assert_script_run 'dnf config-manager --set-disabled updates-testing';
12 prepare_test_packages; 12​ prepare_test_packages;
13 # get back to the desktop 13​ # get back to the desktop
14 $self->desktop_vt(); 14​ $self->desktop_vt();
15 # run the updater 15​ # run the updater
16 if ($desktop eq 'kde') { 16​ if ($desktop eq 'kde') {
17 # KDE team tells me the 'preferred' update method is the 17​ # KDE team tells me the 'preferred' update method is the
18 # systray applet 18​ # systray applet
19 assert_and_click 'desktop_expand_notifications'; 19​ assert_and_click 'desktop_expand_systray';
20 } 20​ }
21 else { 21​ else {
22 # this launches GNOME Software on GNOME, dunno for any other 22​ # this launches GNOME Software on GNOME, dunno for any other
23 # desktop yet 23​ # desktop yet
24 $self->menu_launch_type('update'); 24​ $self->menu_launch_type('update');
25 } 25​ }
26 # GNOME Software has a welcome screen, get rid of it if it shows 26​ # GNOME Software has a welcome screen, get rid of it if it shows
27 # up (but don't fail if it doesn't, we're not testing that) 27​ # up (but don't fail if it doesn't, we're not testing that)
▲ Show 20 Lines • Show All 51 Lines • Show Last 20 Lines