Added a bash directive
ClosedPublic

Authored by lbrabec on Apr 20 2016, 8:03 AM.

Details

Summary

This diff introduces a new bash directive, it executes given command in
subprocess. Stdout of executed command is exportable in formula.

If we want to use this directive as main check of the task, @kparal suggested
that we should introduce a tool for generating results yaml. A simple script or
shell function that from passed arguments creates a output we understand.

Also we need to discuss how to pass the test output from executed command or
script other than stdout. Almost every command in shell prints something to
stdout and thus bloats the output (which should contain only results yaml at
the moment).

Here is example how it works now, the task pulls httpd docker image, runs
container and tries curl localhost (the example is little too verbose,
all the bash command would probably end in test.sh file):

name: test httpd docker image

environment:
  rpm:
    - docker

actions:
  - bash:
      command: systemctl start docker

  - bash:
      command: docker pull httpd

  - bash:
      command: docker run -d -p 80:80 --name docker_test_httpd httpd

  - bash:
      command: sh test.sh
    export: test_output

  - bash:
      command: docker rm -f docker_test_httpd

  - name: report
    resultsdb:
      results: ${test_output}

The file test.sh contains:

#!/bin/bash
{
  #wait for container to start
  sleep 1s
  #test it
  curl localhost
  outcome=$?

} &> /dev/null


cat << EOF
results:
  - item: httpd
    type: docker_image
    outcome: $([ "$outcome" == 0 ] && echo "PASSED" || echo "FAILED")
EOF
Test Plan

tests will follow

Diff Detail

Repository
rLTRN libtaskotron
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
lbrabec retitled this revision from to Added a bash directive.Apr 20 2016, 8:03 AM
lbrabec updated this object.
lbrabec edited the test plan for this revision. (Show Details)
lbrabec added reviewers: tflink, kparal, jskladan, mkrizek.
lbrabec added a subscriber: kparal.

Other than the lint errors, it looks fine to me.

WRT getting output, I wonder if we should either change the resultsdb directive to parse files on the filesystem or if we want to add a directive that does that specially.

I don't think that attempting to extract YAML/XML/TAP from stdout is going to be wildly successful and reading from a specified file shouldn't be too much more effort for task authors.

In D819#15587, @tflink wrote:

WRT getting output, I wonder if we should either change the resultsdb directive to parse files on the filesystem or if we want to add a directive that does that specially.

This was one of the things that sparked quite a heated discussion yesterday. tldr - if we decide to parse files, it should be a directive of its own - does not really make sense to limit file-parsing ability to just resultsdb directive. For one, it is not systematic, but more importantly - you might want to pass _any_ data from the bash (or any other, for that matter) directive, not necessarily only result yaml, thus making fileparsing solely resultsdb directive's ability is IMO not the approach to take.

Also, with the bash directive being present, we effectively do have a "read file" directive:

  - name: read results from file
	​bash:
	​    command: cat /path/to/the/result/file.yml
	​export: task_results

I don't think that attempting to extract YAML/XML/TAP from stdout is going to be wildly successful and reading from a specified file shouldn't be too much more effort for task authors.

Agreed, even though we might be somewhat successful, I'd rather not. And although needing to have a hardcoded file path in both the formula (to read the results) and the task/wrapper script (to store them), is a huge code smell for me, I was not able to make case for any other solution, so we'll probably be going forward with this approach. At least for the start.

lbrabec updated this revision to Diff 2110.Apr 27 2016, 8:37 AM
  • tests

When I try to run your example as a task (D819.diff is the raw diff from this review)

$ python runtask.py -t compose -i madeupstuff-1234 --libvirt -p D819.diff ../task-dockershell-example/dockershell.yml

I end up with:

[libtaskotron:executor.py:81] 2016-04-27 17:25:21 DEBUG   Current workdir: /var/tmp/taskotron/root/task-hEaRES
[libtaskotron:executor.py:141] 2016-04-27 17:25:21 DEBUG   Executing directive: bash
[libtaskotron:bash_directive.py:77] 2016-04-27 17:25:21 INFO    Executing Bash: ['systemctl', 'start', 'docker']
[libtaskotron:logger.py:88] 2016-04-27 17:25:21 CRITICAL Traceback (most recent call last):
  File "/usr/bin/runtask", line 9, in <module>
    load_entry_point('libtaskotron==0.4.11', 'console_scripts', 'runtask')()
  File "/usr/lib/python2.7/site-packages/libtaskotron/main.py", line 163, in main
    overlord.start()
  File "/usr/lib/python2.7/site-packages/libtaskotron/overlord.py", line 92, in start
    runner.execute()
  File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 57, in execute
    self._run()
  File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 91, in _run
    self._do_actions()
  File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 129, in _do_actions
    self._do_single_action(action)
  File "/usr/lib/python2.7/site-packages/libtaskotron/executor.py", line 150, in _do_single_action
    self.arg_data)
  File "/usr/lib/python2.7/site-packages/libtaskotron/directives/bash_directive.py", line 84, in process
    raise TaskotronDirectiveError(e)
TaskotronDirectiveError: [Errno 2] No such file or directory: ''

[libtaskotron:minion.py:208] 2016-04-27 17:25:22 WARNING Task execution was interrupted by an error, doing emergency cleanup.
[libtaskotron:minion.py:216] 2016-04-27 17:25:22 INFO    Shutting down the minion...
[libtaskotron:vm.py:173] 2016-04-27 17:25:22 DEBUG   Stopping instance taskotron-20160427_172428_628154
[testcloud.instance:instance.py:417] 2016-04-27 17:25:22 DEBUG   stopping instance taskotron-20160427_172428_628154.
[libtaskotron:vm.py:185] 2016-04-27 17:25:24 DEBUG   destroying instance taskotron-20160427_172428_628154
[testcloud.instance:instance.py:429] 2016-04-27 17:25:24 DEBUG   removing instance taskotron-20160427_172428_628154 from libvirt.
[testcloud.instance:instance.py:446] 2016-04-27 17:25:24 DEBUG   Unregistering domain from libvirt.
[testcloud.instance:instance.py:448] 2016-04-27 17:25:24 DEBUG   removing instance /var/lib/testcloud/instances/taskotron-20160427_172428_628154 from disk
[libtaskotron:logger.py:88] 2016-04-27 17:25:24 CRITICAL Traceback (most recent call last):
  File "runtask.py", line 10, in <module>
    main.main()
  File "/home/tflink/code/taskotron/libtaskotron/libtaskotron/main.py", line 163, in main
    overlord.start()
  File "/home/tflink/code/taskotron/libtaskotron/libtaskotron/overlord.py", line 92, in start
    runner.execute()
  File "/home/tflink/code/taskotron/libtaskotron/libtaskotron/minion.py", line 203, in execute
    self._run()
  File "/home/tflink/code/taskotron/libtaskotron/libtaskotron/minion.py", line 127, in _run
    self.exitcode = self.ssh.cmd(' '.join(task_cmd))
  File "/home/tflink/code/taskotron/libtaskotron/libtaskotron/remote_exec.py", line 150, in cmd
    (cmd, self.username, self.hostname, retcode))
TaskotronRemoteProcessError: Command "cd /var/tmp/taskotron/taskdir && runtask --item madeupstuff-1234 --jobid -1 --type compose --uuid 20160427_172428_628154 --local dockershell.yml" on root@192.168.124.189 exited with code 1

Am I doing something wrong to run this?

lbrabec updated this revision to Diff 2116.Apr 28 2016, 7:39 AM
  • path fix
tflink accepted this revision.Apr 28 2016, 12:43 PM
This revision is now accepted and ready to land.Apr 28 2016, 12:43 PM
Closed by commit rLTRN629f0770a9a1: Added a bash directive (authored by Lukas Brabec <lbrabec@redhat.com>). · Explain WhyApr 28 2016, 1:16 PM
This revision was automatically updated to reflect the committed changes.