mention main.pm architecture in README
ClosedPublic
Actions

Authored by garretraziel on Aug 11 2015, 10:17 AM.

Details

Reviewers

kparal
adamwill
jskladan

Maniphest Tasks

T546: Describe the idea behind OpenQA main.pm

Commits

rOPENQATESTS5438b452d212: mention main.pm architecture in README

Summary

Mention main.pm modular architecture in separate chapter in README,
describing its parts, customization VARIABLES, automatically loaded tests.

Test Plan

read main.pm modular architecture to check if it makes sense

Diff Detail

Repository

rOPENQATESTS os-autoinst-distri-fedora

Lint

Automatic diff as part of commit; lint not applicable.

Unit

Automatic diff as part of commit; unit tests not applicable.

garretraziel retitled this revision from to mention main.pm architecture in README.Aug 11 2015, 10:17 AM

garretraziel updated this object.

garretraziel edited the test plan for this revision. (Show Details)

garretraziel added reviewers: jskladan, adamwill.

Herald added a subscriber: tflink. · View Herald TranscriptAug 11 2015, 10:17 AM

garretraziel added a task: T546: Describe the idea behind OpenQA main.pm.Aug 11 2015, 10:20 AM

garretraziel edited the test plan for this revision. (Show Details)Aug 11 2015, 10:52 AM

Looks readable to me.

This revision is now accepted and ready to land.Aug 11 2015, 11:10 AM

Closed by commit rOPENQATESTS5438b452d212: mention main.pm architecture in README (authored by garretraziel). · Explain WhyAug 11 2015, 11:31 AM

This revision was automatically updated to reflect the committed changes.

Revision Contents

			Path	Packages
M			README.md (81 lines)
M			VARIABLES.md (2 lines)

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	1303	cd57a72		Aug 11 2015, 10:17 AM	★	★
Diff 2	1304	8fde224	Commit rOPENQATESTS5438b452d212bafd31150ae7ee445c27eb83312b	Aug 11 2015, 11:30 AM	★	★

Commit	Tree	Parents	Author	Summary	Date
5c194448f8ac	907366784fc2	cd57a721bc44	Jan Sedlák	mention main.pm architecture in README (Show More…)	Aug 11 2015, 10:15 AM

Diff 1304

View Options

README.md

1	OpenQA tests for the Fedora distribution	1	OpenQA tests for the Fedora distribution
2	========================================	2	========================================
3		3
4	This repository contains tests and images for testing [Fedora](https://getfedora.org/) with	4	This repository contains tests and images for testing [Fedora](https://getfedora.org/) with
5	[OpenQA](http://os-autoinst.github.io/openQA/). For additional tools, Installation Guide and	5	[OpenQA](http://os-autoinst.github.io/openQA/). For additional tools, Installation Guide and
6	Docker images, see [this repository](https://bitbucket.org/rajcze/openqa_fedora_tools).	6	Docker images, see [this repository](https://bitbucket.org/rajcze/openqa_fedora_tools).
7		7
8	Test development	8	Test development
9	----------------	9	----------------
10	See official documentation [on basic concept](https://github.com/os-autoinst/openQA/blob/master/docs/GettingStarted.asciidoc), [test development (including API specification)](https://github.com/os-autoinst/openQA/blob/master/docs/WritingTests.asciidoc), [needles specification](https://github.com/os-autoinst/os-autoinst/blob/master/doc/needles.txt) and [supported variables for backend](https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc). See	10	See official documentation [on basic concept](https://github.com/os-autoinst/openQA/blob/master/docs/GettingStarted.asciidoc),
		11	[test development (including API specification)](https://github.com/os-autoinst/openQA/blob/master/docs/WritingTests.asciidoc),
		12	[needles specification](https://github.com/os-autoinst/os-autoinst/blob/master/doc/needles.txt) and
		13	[supported variables for backend](https://github.com/os-autoinst/os-autoinst/blob/master/doc/backend_vars.asciidoc). See
11	[this example repo](https://github.com/os-autoinst/os-autoinst-distri-example) on how tests should be structured.	14	[this example repo](https://github.com/os-autoinst/os-autoinst-distri-example) on how tests should be structured.
12		15
13	In short, since OpenQA uses only one entrypoint for all tests (`main.pm`), we have decided to utilize	16	### main.pm modular architecture
14	this feature and make tests modular. It means that basic passing through `main.pm` (without any variables set)	17	Since OpenQA uses only one entrypoint for all tests (main.pm), we have decided to utilize
15	results in most basic installation test executed. Developer can then customize it with additional variables	18	this feature and make tests modular. It means that basic passing through main.pm (without any variables set)
		19	results in most basic installation test executed. Developer can customize it with additional variables
16	(for example by setting `PACKAGE_SET=minimal` to do installation only with minimal package set).	20	(for example by setting `PACKAGE_SET=minimal` to do installation only with minimal package set).
17		21
18	Make your test modular, so that it utilizes `_boot_to_anaconda()` and `_do_install_and_reboot()`	22	Fedora installation (and consequently main.pm) consists of several parts:
19	tests (that are loaded automatically). Break your test into smaller parts, each dealing with one	23
20	specific feature (e. g. partitioning, user creation...) and add their loading into `main.pm` based	24	1. booting into Anaconda or booting live image and starting Anaconda
21	on reasonable variable setting (so they can be used in other tests also).	25
		26	Since there isn't much variation between tests in this step, we have developed universal `_boot_to_anaconda.pm`
		27	test that is loaded automatically each time except when `ENTRYPOINT` or `UPGRADE` is set (see VARIABLES.md).
		28
		29	To customize this step, you can set following variables:
		30
		31	- `GRUB` is appended to kernel line before boot. You can set for example `inst.updates` here.
		32	- If `KICKSTART` is set, this part of installation ends here (program doesn't wait for Anaconda to appear).
		33	Note that you should set `inst.ks` yourself by setting `GRUB` variable.
		34	- If `LIVE` is set, program waits for desktop to appear and then clicks on "Install to Hard Drive" button.
		35
		36	2. customizing installation by interacting with Anaconda spokes
		37
		38	Most of the differences between tests take place in this part. If you want to add another installation test,
		39	you will probably put your variable checking and test loading here. All tests in this part should start on
		40	Anaconda's main hub and after they done its part, they should go back to Anaconda's main hub so that next
		41	test could be executed. In this phase, universal `_software_selection.pm` test is loaded that handles
		42	selecting what software to install.
		43
		44	To customize this step, you can set following variables:
		45
		46	- Set `PACKAGE_SET` to install required package set on "Software selection spoke" - you have to provide correct needles with the
		47	name of `anaconda_${PACKAGE_SET}_highlighted` and `anaconda_${PACKAGE_SET}_selected`.
		48	- Set `ENCRYPT_PASSWORD` to encrypt disk, value of this variable is used as an actual password.
		49
		50	3. installing Fedora and waiting for Fedora to reboot
		51
		52	After all customizations are finished, `_do_install_and_reboot.pm` test is automatically loaded.
		53	It starts installation, creates user and sets root password when required, waits for installation
		54	to finish and reboots into installed system. Only variables that control flow in this part are these:
		55
		56	- `ROOT_PASSWORD` to set root password to this value.
		57	- When set, `USER_LOGIN` and `USER_PASSWORD` are used to create user in Anaconda.
		58
		59	4. post-install phase
		60
		61	After installation is finished and installed system is fully booted, you can run additional tests
		62	as checks that installed system has correct attributes - that correct file system is used, that
		63	RAID is used etc.
		64
		65	Make your test modular, so that it utilizes `_boot_to_anaconda.pm`, `_software_selection.pm` and
		66	`_do_install_and_reboot.pm` tests (that are loaded automatically). Break your test into smaller parts,
		67	each dealing with one specific feature (e. g. partitioning, user creation...) and add their loading
		68	into main.pm based on reasonable variable setting (so they can be used in other tests also).
22		69
23	### Test inheritance	70	### Test inheritance
24	Your test can inherit from `basetest`, `fedorabase`, `installedtest` or `anacondatest`.	71	Your test can inherit from `basetest`, `fedorabase`, `installedtest` or `anacondatest`.
25		72
26	* `basetest` is basic class provided by os-autoinst - it has empty `post_fail_hook()` and doesn't set any flags.	73	- `basetest` is basic class provided by os-autoinst - it has empty `post_fail_hook()` and doesn't set any flags.
27	* `fedorabase` doesn't neither set flags nor does anything in `post_fail_hook()`, but it provides basic functions	74	- `fedorabase` doesn't neither set flags nor does anything in `post_fail_hook()`, but it provides basic functions
28	that will be useful during testing Fedora, like `console_login()` or `boot_to_login_screen()`. It should be used	75	that will be useful during testing Fedora, like `console_login()` or `boot_to_login_screen()`. It should be used
29	when no other, more specific class can be used.	76	when no other, more specific class can be used.
30	* `anacondatest` should be used in tests where Anaconda is running. It uploads Anaconda logs (for example	77	- `anacondatest` should be used in tests where Anaconda is running. It uploads Anaconda logs (for example
31	`anaconda.log` or `packaging.log`) in `post_fail_hook()`. It also provides convenient methods for Anaconda	78	`anaconda.log` or `packaging.log`) in `post_fail_hook()`. It also provides convenient methods for Anaconda
32	like `select_disks()`.	79	like `select_disks()`.
33	* `installedtest` should be used in tests that are running on installed system (either in postinstall phase	80	- `installedtest` should be used in tests that are running on installed system (either in postinstall phase
34	or in upgrade tests). It uploads `/var/log` in `post_fail_hook()`.	81	or in upgrade tests). It uploads `/var/log` in `post_fail_hook()`.
35		82
36	### Test development checklist	83	### New test development workflow
37		84
38	1. Select test from [this document](https://bitbucket.org/rajcze/openqa_fedora_tools/src/develop/PhaseSeparation.md) or from [phabricator page](https://phab.qadevel.cloud.fedoraproject.org/maniphest/?statuses=open%28%29&projects=PHID-PROJ-epofbmazit3u2rndqccd#R)	85	1. Select test from [this document](https://bitbucket.org/rajcze/openqa_fedora_tools/src/develop/PhaseSeparation.md) or from
		86	[phabricator page](https://phab.qadevel.cloud.fedoraproject.org/maniphest/?statuses=open%28%29&projects=PHID-PROJ-epofbmazit3u2rndqccd#R)
39	2. Put each part of your test as a separate file into `tests/` directory, reimplementing `run()` method	87	2. Put each part of your test as a separate file into `tests/` directory, reimplementing `run()` method
40	and `test_flags()` method, inheriting from one of the classes mentioned above.	88	and `test_flags()` method, inheriting from one of the classes mentioned above.
41	3. Set correct variables (so that all test parts you have made are executed) in [WebUI -> Test suites](https://localhost:8080/admin/test_suites).	89	3. Set correct variables (so that all test parts you have made are executed) in [WebUI -> Test suites](https://localhost:8080/admin/test_suites).
42	4. Link your newly created Test suite to medium type in [WebUI -> Job groups](https://localhost:8080/admin/groups).	90	4. Link your newly created Test suite to medium type in [WebUI -> Job groups](https://localhost:8080/admin/groups).
43	5. Run test (see [openqa_fedora_tools repository](https://bitbucket.org/rajcze/openqa_fedora_tools)).	91	5. Run test (see [openqa_fedora_tools repository](https://bitbucket.org/rajcze/openqa_fedora_tools)).
44	6. Create needles (images) by using interactive mode and needles editor in WebUI.	92	6. Create needles (images) by using interactive mode and needles editor in WebUI.
45	7. Add new Job template and Test suite into `templates` file.	93	7. Add new Job template and Test suite into `templates` file.
46	8. Add new Test suite and Test case into [`conf_test_suites.py`](https://bitbucket.org/rajcze/openqa_fedora_tools/src/develop/tools/openqa_trigger/conf_test_suites.py) file in openqa_fedora_tools repository.	94	8. Add new Test suite and Test case into [`conf_test_suites.py`](https://bitbucket.org/rajcze/openqa_fedora_tools/src/develop/tools/openqa_trigger/conf_test_suites.py)
		95	file in openqa_fedora_tools repository.
47	9. Mark your test in PhaseSeparation.md as done.	96	9. Mark your test in PhaseSeparation.md as done.
48	10. Open differential request via phabricator.	97	10. Open differential request via phabricator, set openqa_fedora as a project and repository.