Differential D928

docker docs: improve bash and python example scripts
ClosedPublic
Actions

Authored by kparal on Jul 8 2016, 12:45 PM.

Details

Reviewers
lbrabec
jskladan
Commits
rLTRNcc5decf5565b: docker docs: improve bash and python example scripts
Summary

Make the first bash example closer to what docker uses (set -e). Make
the second bash example use checkname to distinguish tests rather than
note. Use long arguments for taskotron_result to make it easier to
read. Use subprocess directly in the Python example.

This took the changes from D927 and extended it with some further improvements.

Test Plan

examples not tested

Diff Detail

Repository
rLTRN libtaskotron
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
kparal retitled this revision from to docker docs: improve bash and python example scripts.Jul 8 2016, 12:45 PM
kparal updated this object.
kparal edited the test plan for this revision. (Show Details)
kparal added reviewers: lbrabec, jskladan.
This revision is now accepted and ready to land.Jul 8 2016, 12:52 PM
Closed by commit rLTRNcc5decf5565b: docker docs: improve bash and python example scripts (authored by kparal). · Explain WhyJul 8 2016, 12:53 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathPackages
Mdocs/source/docker.rst (95 lines)

Diff 2362

View Options

docs/source/docker.rst

1===================== 1​=====================
2Testing Docker Images 2​Testing Docker Images
3===================== 3​=====================
4 4​
5In Taskotron, we support two approaches on how to test Docker images: shell script or pytest. 5​In Taskotron, we support two approaches on how to test Docker images: shell script or pytest.
6 6​
7 7​
8Shell script 8​Shell script
9============ 9​============
10Shell scripts are commonly used in testing, Docker official images have tests written in bash and 10​Shell scripts are commonly used in testing, Docker official images have checks written in bash and
11we assume that shell script approach will be the default choice. 11​we assume that shell script approach will be the default choice.
12 12​
13Example bellow shows what could tests in shell can look like: 13​Here's an example of how a docker check in shell often looks like. The script makes sure that docker
14​service is running, that image is pulled from repository and then runs two checks. Finally,
15​the container is stopped and removed.
14 16​
15.. code-block:: bash 17​.. code-block:: bash
16 18​
17 #!/bin/bash 19​ #!/bin/bash
18 20​
21​ # exit on any error
22​ set -eo pipefail
23​
19 # setup 24​ # setup
20 systemctl start docker 25​ systemctl start docker
21 docker pull httpd 26​ docker pull httpd
22 container=`docker run -d -p 80:80 httpd` 27​ container=`docker run -d -p 80:80 httpd`
23 # wait for container 28​ # wait for container
24 sleep 1s 29​ sleep 1s
25 30​
26 # test 1 31​ # test 1
27 output=`netstat -lnt | awk '$6 == "LISTEN" && $4 ~ "80$"'` 32​ output=`netstat -lnt | awk '$6 == "LISTEN" && $4 ~ "80$"'`
28 outcome1=$([ "$output" != "" ] && echo "PASSED" || echo "FAILED") 33​ [ "$output" != "" ]
29 34​
30 # test 2 35​ # test 2
31 curl localhost 36​ curl localhost
32 outcome2=$([ "$?" == 0 ] && echo "PASSED" || echo "FAILED") ​
33 37​
34 # teardown 38​ # teardown
35 docker rm -f $container 39​ docker rm -f $container
36 40​
37The script makes sure that docker service is running, that image is pulled from repository and then 41​For running this check in Taskotron, we need slight adjustments. We will no longer crash the whole
38runs two tests. Finally, container is stopped and removed. For propagating the test outcomes 42​script on any individual command error, instead we will check their output/exit code and generate
39further, we prepared `taskotron_result`, a tool that generates :ref:`resultyaml-format`. 43​results accordingly (an alternative is to put parts of code into separate shell scripts and run
40Consecutive usage of `taskotron_result` will append the outcome to the specified output file. 44​them in a subshell). We will check a particular docker image version, instead of the latest one,
45​and identify it in task results. We will save the results in a file (which is later passed to a
46​reporting directive). In order to save task results easily, we'll use ``taskotron_result``, a tool
47​that generates :ref:`resultyaml-format`. Consecutive usage of ``taskotron_result`` will append
48​another result to the specified output file.
41 49​
42.. code-block:: bash 50​.. code-block:: bash
43 51​
44 #!/bin/bash 52​ #!/bin/bash
45 53​
46 # variable $1 has to contain path to output file 54​ # test item is passed from task formula and is in "name:tag" format, e.g. "httpd:2.4"
55​ ITEM=$1
56​ # where to save results, again passed from task formula
57​ OUTFILE=$2
58​ # how to name our task results
59​ CHECKNAME="httpd_sanity"
47 60​
48 # setup 61​ # setup
49 systemctl start docker 62​ systemctl start docker
50 docker pull httpd 63​ docker pull $ITEM
51 container=`docker run -d -p 80:80 httpd` 64​ container=`docker run -d -p 80:80 $ITEM`
52 # wait for container 65​ # wait for container
53 sleep 1s 66​ sleep 1s
54 67​
55 # test 1 68​ # test 1
56 output=`netstat -lnt | awk '$6 == "LISTEN" && $4 ~ "80$"'` 69​ output=`netstat -lnt | awk '$6 == "LISTEN" && $4 ~ "80$"'`
57 outcome1=$([ "$output" != "" ] && echo "PASSED" || echo "FAILED") 70​ outcome1=$([ "$output" != "" ] && echo "PASSED" || echo "FAILED")
58 taskotron_result -f $1 -i httpd -t docker_image -o $outcome1 -n "listening port 80" 71​ taskotron_result --file $OUTFILE --item $ITEM --report_type docker_image --outcome $outcome1 \
72​ --checkname "$CHECKNAME.netstat"
59 73​
60 # test 2 74​ # test 2
61 curl localhost 75​ curl localhost
62 outcome2=$([ "$?" == 0 ] && echo "PASSED" || echo "FAILED") 76​ outcome2=$([ "$?" == 0 ] && echo "PASSED" || echo "FAILED")
63 taskotron_result -f $1 -i httpd -t docker_image -o $outcome2 -n "working http" 77​ taskotron_result --file $OUTFILE --item $ITEM --report_type docker_image --outcome $outcome2 \
78​ --checkname "$CHECKNAME.curl"
64 79​
65 # teardown 80​ # teardown
66 docker rm -f $container 81​ docker rm -f $container
67 82​
68Output file now contains this ResultYAML output: 83​Output file now contains this ResultYAML output:
69 84​
70.. code-block:: yaml 85​.. code-block:: yaml
71 86​
72 results: 87​ results:
73 - item: httpd 88​ - item: <item from command line>
74 note: listening port 80 ​
75 outcome: PASSED ​
76 type: docker_image 89​ type: docker_image
77 - item: httpd 90​ checkname: httpd_sanity.netstat
78 note: working http ​
79 outcome: PASSED 91​ outcome: PASSED
92​ - item: <item from command line>
80 type: docker_image 93​ type: docker_image
94​ checkname: httpd_sanity.curl
95​ outcome: PASSED
81 96​
82 97​We'll use this task formula to execute this task in Taskotron:
83Accompanying formula to run in taskotron: ​
84 98​
85.. code-block:: yaml 99​.. code-block:: yaml
86 100​
87 name: docker_httpd 101​ name: httpd_sanity
88 102​
89 environment: 103​ environment:
90 rpm: 104​ rpm:
91 - docker 105​ - docker
92 106​
93 actions: 107​ actions:
94 - bash: 108​ - bash:
95 command: bash test.sh ${workdir}/test_output 109​ command: bash test.sh ${item} ${workdir}/test_output
96 110​
97 - bash: 111​ - bash:
98 command: cat ${workdir}/test_output 112​ command: cat ${workdir}/test_output
99 export: test_results 113​ export: test_results
100 114​
101 - resultsdb: 115​ - resultsdb:
102 results: ${test_results} 116​ results: ${test_results}
103 117​
104 118​
105 ​
106Pytest 119​Pytest
107====== 120​======
108Pytest provides richer coding experience (e.g. fixtures, setup methods, teardown methods etc.) and 121​Pytest provides richer coding experience (e.g. fixtures, setup methods, teardown methods etc.) and
109there are projects that already use pytest for docker image testing. For ease of reusability, 122​there are projects that already use pytest for docker image testing. For ease of reusability,
110taskotron can run test suites written in pytest. For seamless integration we support xUnit format 123​Taskotron can run test suites written in pytest. For seamless integration we support xUnit format
111to propagate test results further. 124​to propagate check results further.
112 125​
113Example of above mentioned test, now written in python:: 126​Example of the aforementioned check, now written in Python.
127​
128​.. code-block:: python
114 129​
115 #!/usr/bin/env python 130​ #!/usr/bin/env python
116 131​
117 from time import sleep 132​ from time import sleep
118 import os ​
119 import pytest ​
120 import subprocess 133​ import subprocess
121 134​
122 def system(cmd): ​
123 ret = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE, ​
124 stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True) ​
125 out, err = ret.communicate() ​
126 return out, err, ret.returncode ​
127 ​
128 class TestDockerHttpdSimple(object): 135​ class TestDockerHttpdSimple(object):
129 def setup_method(self, test_method): 136​ def setup_method(self, test_method):
130 out, err, ret = system('docker run -d -p 80:80 httpd') 137​ self.container = subprocess.check_output(['docker', 'run', '-d', '-p', '80:80', 'httpd'])
131 sleep(1) # wait for container 138​ sleep(1) # wait for container
132 self.container = out ​
133 139​
134 def teardown_method(self, test_method): 140​ def teardown_method(self, test_method):
135 out, err, ret = system('docker rm -f %s' % self.container) 141​ subprocess.check_call(['docker', 'rm', '-f', self.container])
136 142​
137 def test_listening_port80(self): 143​ def test_listening_port80(self):
138 out, err, ret = system("netstat -lnt | awk '$6 == \"LISTEN\" && $4 ~ \"80$\"'") 144​ out = subprocess.check_output("netstat -lnt | awk '$6 == \"LISTEN\" && $4 ~ \"80$\"'",
145​ shell=True)
139 assert out != "" 146​ assert out != ""
140 147​
141 def test_working_http(self): 148​ def test_working_http(self):
142 out, err, ret = system("curl localhost") 149​ subprocess.check_call(['curl', 'localhost'])
143 assert ret == 0 ​
144 ​
145 150​
146Accompanying formula: 151​We'll use this task formula to execute this task in Taskotron:
147 152​
148.. code-block:: yaml 153​.. code-block:: yaml
149 154​
150 name: docker_httpd 155​ name: httpd_sanity
151 156​
152 environment: 157​ environment:
153 rpm: 158​ rpm:
154 - docker 159​ - docker
155 - python2-pytest 160​ - python2-pytest
156 161​
157 actions: 162​ actions:
158 - bash: 163​ - bash:
159 command: systemctl start docker 164​ command: systemctl start docker
160 165​
161 - bash: 166​ - bash:
162 command: docker pull httpd 167​ command: docker pull ${item}
163 168​
164 - bash: 169​ - bash:
165 command: py.test dockertest.py --junitxml=${artifactsdir}/output.xml 170​ command: py.test dockertest.py --junitxml=${artifactsdir}/output.xml
166 171​
167 - xunit: 172​ - xunit:
168 file: ${artifactsdir}/output.xml 173​ file: ${artifactsdir}/output.xml
169 agregation: none 174​ aggregation: none
170 export: test_results 175​ export: test_results
171 176​
172 - resultsdb: 177​ - resultsdb:
173 results: ${test_results} 178​ results: ${test_results}
174 179​
175 180​
176Other possibilities 181​Other possibilities
177=================== 182​===================
178Taskotron is capable of running arbitrary code (e.g. via bash directive) in any language. If you 183​Taskotron is capable of running arbitrary code (e.g. via bash directive) in any language. If you
179cannot choose between shell script and pytest, you can always write your docker tests in any other 184​cannot choose between shell script and pytest, you can always write your docker tests in any other
180language you prefer. Just make sure that the test results are in format we understand -- 185​language you prefer. Just make sure that the test results are in format we understand --
181:ref:`resultyaml-format` or xUnit. 186​:ref:`resultyaml-format` or xUnit.