1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
|
Dummy (or Destructive) Testsuite Framework, DTF
===============================================
Testsuite supported by this framework is expected¹ to be destructive, may
require root permissions or basically whatever. So do not run such testsuite on
your machine unless you *really* know what you are doing. It is always better
to create temporary virtual machine to run them there (we have some tooling for
OpenStack VM provisioning).
WORKFLOW
--------
In general, we expect these steps for running the testsuite:
* generate self-standing testsuite tarball
* start (temporary) virtual machine
* copy the test tarball on the new machine and perform testing
* collect the results from VM
* terminate the VM
However, the testsuite is designed so it may be developed (and thus run)
locally.
OVERVIEW
--------
The framework consists of basically two parts, 'tester' and 'controller'.
The 'tester' component is something like tool-set to help users with writing
theirs 'testsuites'. That is (a) convenient (Bash) library available for
for testsuite, (b) main running script (named 'run', it is also part of DTF API)
and (c) tooling which helps with testsuite initialization.
The 'controller' part is tool-set which helps you to automatize running of the
testsuite on remote boxes (currently on OpenStack VMs only).
PORTABILITY
-----------
Currently, the tools here are mostly written in Bash (not shell) and Perl and
thats the reason why you need² to have these languages working on your system.
Also, it was tested only on GNU/Linux operating system.
CREATE TESTSUITE
----------------
Each testsuite must be self-standing, distributable and ideally non-destructive.
Creating testsuite should follow this cook-book:
* Create the directory structure:
$ mkdir YOURPKG_TEST_DIR
$ dtf-prepare-testsuite YOURPKG_TEST_DIR
Then prepares the 'dtflib', 'run' script and basic dummy task for you. The
dummy test-case created is in 'YOURPKG_TEST_DIR/tasks/sample/runtest.sh'.
Be aware of the fact that 'dtf-prepare-testsuite' is expected to be always run
against latest GIT version of that script. The 'dtflib' and 'run' files (and
probably other) will then be copied from GIT.
* Test the sample:
$ ./YOURPKG_TEST_DIR/run
RUN dummy-sample ... [ OK ]
The 'runtest.sh' file must have the 'run()' function defined and should follow
the exit-status API defined in 'run' script.
* To distribute the tarball, just run:
$ ./YOURPKG_TEST_DIR/run --dist | gzip > testsuite.tar.gz
Also, please look at the `YOURPKG_TEST_DIR/run --help` output.
REMOTE RUN
----------
To perform this task easily, we expect you to have access to some OpenStack
instance. You need to have OpenStack credentials file in
'~/.dtf/private/os/OSID.yml' (note the OSID identifier). The file should look
like (be sure that only *you* can read that file):
$ cat ~/.dtf/private/os/OSID.yml
---
os_auth_url: http://control.example.com:5000/v2.0
os_tenant_id: 9df30fc192f5xxxxxxxxxxxxxxxx0110
os_tenant_name: Your OS Project
os_username: os1username
os_nova_password: your_os_password
Also, you must have your distribution images prepared available in OpenStack.
This needs to be configured for DTF in '~/.dtf/os/OSID.sh'. See the
'OSID.sh.template' doc file for example configuration.
Then it should be enough to run:
$ dtf-run-remote --distro=fedora --distro-version=20 --taskdir YOUR_TEST_DIR
...
REQUIREMENTS
------------
Packages needed to successful run
* ansible (for remote runs)
* Perl
* Bash
* perl(Text::Xslate)
* perl(YAML::Syck)
----
¹ However, making tests it non-destructive is highly recommended.
² So far there was no need to make it more portable, however it should be
possible.
|
