Initial implementation of live image testing.
ClosedPublic
Actions

Authored by adamwill on Mar 17 2015, 5:58 PM.

Details

Reviewers

garretraziel
jskladan

Summary

This requires adding products, flavors and needles and test
cases, and tweaking some existing ones to handle the slightly
different behaviour of live images in shared tests.

The bootloader needles are simplified to match on some text
that exists on both (we need two different screenshots,
though, as for some reason the background color is different
on live vs. non-live).

The tweak to the 'Done' button needle allows it to match
when the button contains a faint dashed rectangle, as it
seems to on live installs, sometimes.

The anaconda_install_done needle's main match is the text,
which is the same for live and non-live. It captures a small
portion of the Quit/Reboot button in order to be able to
click it. openQA documents that when a needle contains
multiple areas, assert_and_click will click the centre of
the smallest area. It cannot capture the whole button
because the button's text is different in live vs. non-live.

To handle the different main hub screens in live and non-live,
a less stringent needle is added which is unregistered for
non-live tests, so they don't match on it before they've
finished updating repository metadata.

There are a few small bugfix tweaks in this too, like some
delays in user creation to try and avoid intermittent failures
there.

We also add new root_logged_in needle for the new Rawhide
console font:
https://fedoraproject.org/wiki/Changes/NewDefaultConsoleFont

Test Plan

Trigger runs against various composes and ensure that all
expected images are tested and the tests are not broken. I
have already tested this quite a bit.

Diff Detail

Repository

rOPENQATESTS os-autoinst-distri-fedora

Branch

live

Lint

No Linters Available

Unit

No Unit Test Coverage

adamwill retitled this revision from to Initial implementation of live image testing..Mar 17 2015, 5:58 PM

adamwill updated this object.

adamwill edited the test plan for this revision. (Show Details)

adamwill added reviewers: jskladan, garretraziel.

Herald added a subscriber: tflink. · View Herald TranscriptMar 17 2015, 5:58 PM

oops, probably the CPU should be changed back to 2, changing it to 1 was just a local site customization for me. I find it works better that way, though...

Drop change of QEMUCPUS value (which is specific to me). Not
sure if the QEMUVGA and QEMURAM changes belong or not, but
leaving them for now.

Note: this goes along with https://phab.qadevel.cloud.fedoraproject.org/D309 .

Fine with me. Please squash the commits into more reasonable aggregates before pushing into develop.
I'm still not sold on the Live-login needle, but since it works... :)

This revision is now accepted and ready to land.Mar 18 2015, 12:11 PM

Okay, this also looks good to me.

Was merged into development branch.

Revision Contents

			Path
M			lib/anacondalog.pm (18 lines)
A	M		lib/fedoradistribution.pm (26 lines)
M			main.pm (51 lines)
M			needles/anaconda_install_done.json (41 lines)
P			needles/anaconda_install_finish.json Deleted after being copied to multiple locations: needles/bootloader_bios_live.json needles/bootloader_bios_offline.json
D	M	(img)	needles/anaconda_install_finish.png
A	M		needles/anaconda_main_hub_live.json (25 lines)
P		(img)	needles/anaconda_main_hub_live.png Copied from needles/anaconda_main_hub_server.png
A	M		needles/anaconda_main_hub_nonlive.json (32 lines)
V		(img)	needles/{anaconda_main_hub_nonlive.png ← anaconda_main_hub_server.png}
D	M		needles/anaconda_main_hub_server.json (59 lines)
P		(img)	needles/anaconda_main_hub_server.png Deleted after being copied to multiple locations: needles/anaconda_main_hub_live.png needles/anaconda_main_hub_nonlive.png
M			needles/anaconda_spoke_done.json (10 lines)
P			needles/bootloader_bios_live.json (12 lines) Copied from needles/anaconda_install_finish.json
A	M	(img)	needles/bootloader_bios_live.png
V			needles/{bootloader_bios_offline.json ← anaconda_install_finish.json} (12 lines)
V		(img)	needles/{bootloader_bios_offline.png ← bootloader_server.png}
D	M		needles/bootloader_server.json (31 lines)
V		(img)	needles/{bootloader_server.png → bootloader_bios_offline.png}
A	M		needles/gnome_desktop_clean.json (24 lines)
A	M	(img)	needles/gnome_desktop_clean.png
A	M		needles/graphical_login_gdm.json (17 lines)
A	M	(img)	needles/graphical_login_gdm.png
P			needles/root_logged_in_rawhide20150311.json (10 lines) Copied from needles/text_console_login.json
A	M	(img)	needles/root_logged_in_rawhide20150311.png
P			needles/text_console_login.json (2 lines) Copied to needles/root_logged_in_rawhide20150311.json
A	M		needles/workstation_live_initial.json (17 lines)
A	M	(img)	needles/workstation_live_initial.png
M			templates (122 lines)
M			tests/_boot_to_anaconda.pm (16 lines)
A	M		tests/_console_wait_login.pm (57 lines)
M			tests/_do_install_and_reboot.pm (7 lines)
A	M		tests/_graphical_wait_login.pm (35 lines)
D	M		tests/_wait_for_login_screen.pm (62 lines)

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	843	f8f6d2f		Mar 17 2015, 5:58 PM	★	★
Diff 2	847	f8f6d2f	Drop change of QEMUCPUS value (which is specific to me). Not	Mar 17 2015, 6:37 PM	★	★

Commit	Tree	Parents	Author	Summary	Date
3e240e8a586a	10a92db05708	453c022fc7a2	Adam Williamson	rename 'gdm' needle to 'graphical_login_gdm' per jskladan	Mar 17 2015, 4:40 PM
453c022fc7a2	38f51a53ee10	d165c536176d	Adam Williamson	simplify bootloader needles to match on 'Press Tab' (Show More…)	Mar 13 2015, 11:52 PM
d165c536176d	bbc57ba2a57e	e21607c1d860	Adam Williamson	remove unused anaconda_install_finished needle	Mar 13 2015, 11:50 PM
e21607c1d860	6b95f28c9332	147d7038a10a	Adam Williamson	fix failure log submission (dumb perl think-o)	Mar 11 2015, 10:38 PM
147d7038a10a	afab42c35318	19e2c430c2f2	Adam Williamson	add new root_logged_in needle for new Rawhide console font (Show More…)	Mar 11 2015, 10:07 PM
19e2c430c2f2	f20965f27e50	f8f6d2f8bc64	Adam Williamson	Initial implementation of live image testing. (Show More…)	Mar 11 2015, 12:42 AM

1	package anacondalog;	1	package anacondalog;
2	use base 'basetest';	2	use base 'basetest';
3		3
4	use testapi;	4	use testapi;
5		5
6	sub post_fail_hook {	6	sub post_fail_hook {
7	my $self = shift;	7	my $self = shift;
8	send_key "ctrl-alt-f2";	8	send_key "ctrl-alt-f2";
9	if (check_screen "anaconda_console", 10) {	9	my $logged_in = 0;
		10	if (get_var("LIVE") && check_screen "text_console_login", 20) {
		11	# On live installs, we need to log in
		12	type_string "root";
		13	send_key "ret";
		14	if (check_screen "root_logged_in", 10) {
		15	$logged_in = 1;
		16	}
		17	}
		18	elsif (check_screen "anaconda_console", 10) {
		19	$logged_in = 1;
		20	}
		21
		22	if ($logged_in == 1) {
10	upload_logs "/tmp/X.log";	23	upload_logs "/tmp/X.log";
11	upload_logs "/tmp/anaconda.log";	24	upload_logs "/tmp/anaconda.log";
12	upload_logs "/tmp/packaging.log";	25	upload_logs "/tmp/packaging.log";
13	upload_logs "/tmp/storage.log";	26	upload_logs "/tmp/storage.log";
14	upload_logs "/tmp/syslog";	27	upload_logs "/tmp/syslog";
15	upload_logs "/tmp/program.log";	28	upload_logs "/tmp/program.log";
16	upload_logs "/tmp/dnf.log";	29	upload_logs "/tmp/dnf.log";
17		30
18	# Upload all ABRT logs	31	# Upload all ABRT logs
19	type_string "cd /var/tmp/abrt && tar czvf abrt.tar.gz *";	32	type_string "cd /var/tmp/abrt && tar czvf abrt.tar.gz *";
20	send_key "ret";	33	send_key "ret";
21	upload_logs "/var/tmp/abrt/abrt.tar.gz";	34	upload_logs "/var/tmp/abrt/abrt.tar.gz";
22		35
23	# Upload Anaconda logs	36	# Upload Anaconda logs
24	type_string "tar czvf /tmp/anaconda_tb.tar.gz /tmp/anaconda-tb-*";	37	type_string "tar czvf /tmp/anaconda_tb.tar.gz /tmp/anaconda-tb-*";
25	send_key "ret";	38	send_key "ret";
26	upload_logs "/tmp/anaconda_tb.tar.gz";	39	upload_logs "/tmp/anaconda_tb.tar.gz";
27	}	40	}

		1		package fedoradistribution;
		2		use base 'distribution';
		3
		4		# Fedora distribution class
		5
		6		use testapi qw(send_key type_string);
		7
		8		sub init() {
		9		my ($self) = @_;
		10
		11		$self->SUPER::init();
		12		}
		13
		14		sub x11_start_program($$$) {
		15		my ($self, $program, $timeout, $options) = @_;
		16		# TODO: take screenshots of every goddamn 'run command' dialog in
		17		# every goddamn desktop and keep them updated forever
		18		send_key "alt-f2";
		19		sleep 3;
		20		type_string $program;
		21		sleep 1;
		22		send_key "ret", 1;
		23		}
		24
		25		1;
		26		# vim: set sw=4 et:

Show All 11 Lines
12	#	12	#
13	# You should have received a copy of the GNU General Public License along	13	# You should have received a copy of the GNU General Public License along
14	# with this program; if not, write to the Free Software Foundation, Inc.,	14	# with this program; if not, write to the Free Software Foundation, Inc.,
15	# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.	15	# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
16		16
17	use strict;	17	use strict;
18	use testapi;	18	use testapi;
19	use autotest;	19	use autotest;
		20	use needle;
		21
		22	# distribution-specific implementations of expected methods
		23	my $distri = testapi::get_var("CASEDIR") . '/lib/fedoradistribution.pm';
		24	require $distri;
		25	testapi::set_distribution(fedoradistribution->new());
		26
		27	# Stolen from openSUSE.
		28	sub unregister_needle_tags($) {
		29	my $tag = shift;
		30	my @a = @{ needle::tags($tag) };
		31	for my $n (@a) { $n->unregister(); }
		32	}
		33
		34	sub cleanup_needles() {
		35	if (!get_var('LIVE')) {
		36	## Unregister live-only installer needles. The main issue is the
		37	## hub: on non-live we want to wait for repository setup to complete,
		38	## but if we match that spoke's "ready" icon, it breaks live because
		39	## it doesn't have that spoke. So we have a live needle which doesn't
		40	## match on that icon, but we unregister it for non-live installs so
		41	## they don't match on it too soon.
		42	unregister_needle_tags("ENV-INSTALLER-live");
		43	}
		44	}
		45	$needle::cleanuphandler = \&cleanup_needles;
		46
		47	if (get_var('LIVE')) {
		48	# No package set selection for lives.
		49	set_var('PACKAGE_SET', "default");
		50	}

1	{	1	{
2	"area": [	2	"area": [
3	{	3	{
4	"xpos": 910,	4	"xpos": 669,
5	"ypos": 680,	5	"ypos": 639,
6	"width": 89,	6	"height": 25,
7	"height": 46,	7	"width": 330,
8	"type": "match"	8	"type": "match"
		9	},
		10	{
		11	"type": "match",
		12	"width": 17,
		13	"ypos": 688,
		14	"height": 28,
		15	"xpos": 977
9	}	16	}
10	],	17	],
11	"tags": [	18	"tags": [
12	"anaconda_install_done",	19	"anaconda_install_done",
13	"ENV-DESKTOP-default",	20	"ENV-DESKTOP-default",
14	"ENV-DISTRI-fedora",	21	"ENV-DISTRI-fedora",
15	"ENV-INSTLANG-en_US"	22	"ENV-INSTLANG-en_US"
16	]	23	]
17	}	24	}
18	No newline at end of file

Initial implementation of live image testing.ClosedPublicActions

Details

Diff Detail

Revision Contents

Diff 847

lib/anacondalog.pm

lib/fedoradistribution.pm

main.pm

needles/anaconda_install_done.json

needles/anaconda_install_finish.json

needles/anaconda_install_finish.png

needles/anaconda_main_hub_live.json

needles/anaconda_main_hub_live.png

needles/anaconda_main_hub_nonlive.json

needles/anaconda_main_hub_nonlive.png

needles/anaconda_main_hub_server.json

needles/anaconda_main_hub_server.png

needles/anaconda_spoke_done.json

needles/bootloader_bios_live.json

needles/bootloader_bios_live.png

needles/bootloader_bios_offline.json

needles/bootloader_bios_offline.png

needles/bootloader_server.json

needles/bootloader_server.png

needles/gnome_desktop_clean.json

needles/gnome_desktop_clean.png

needles/graphical_login_gdm.json

needles/graphical_login_gdm.png

needles/root_logged_in_rawhide20150311.json

needles/root_logged_in_rawhide20150311.png

needles/text_console_login.json

needles/workstation_live_initial.json

needles/workstation_live_initial.png

templates

tests/_boot_to_anaconda.pm

tests/_console_wait_login.pm

tests/_do_install_and_reboot.pm

tests/_graphical_wait_login.pm

tests/_wait_for_login_screen.pm

Initial implementation of live image testing.
ClosedPublic
Actions