Make task YAML file support non-string (and nested) variables
ClosedPublic

Authored by kparal on May 23 2014, 12:18 PM.

Details

Summary

This fixes T169 and T148. It also improves T181.

As described in T169, I had to get rid of jinja, because it operates on text only,
and we need to work with objects. The variable substitutions are now done manually,
but you can now easily pass objects in the task yaml.

This also fixes variable nesting, you can now easily use variables deep in the
structure, like this:

task:
    - name: run upgradepath
      python:
          file: upgradepath.py
          callable: main
          custom_args: [--debug, --bodhi-hack, "${bodhi_id}", faketag]
      export: upgradepath_output

The variable syntax changed to ${var}, because dollar is recognized by yaml
as a beginning of string. No quotes are therefore needed in most cases (quite
the opposite, they would break your non-string objects):

key: ${value}

This doesn't work in YAML:

key: {{value}}

because it's recognized as a dictionary start.

Please note that all our task yaml files will need to be adjusted for this
change (however, it's very simple).

This is not complete.

There was a request to support ${ var } style variables. This will requires
some changes in regular expression handling, and I'm afraid it will complicate
the code quite a bit. Patches welcome, or I can propose a patch early next week,
if requested.

Also, if this is approved, I'll also need to remove jinja from dependencies and
adjust our documentation. I haven't done that yet.

Test Plan

Tested with new unit tests and also with rpmlint and upgradepath.

Diff Detail

Repository
rLTRN libtaskotron
Lint
Lint Skipped
Unit
Unit Tests Skipped
kparal retitled this revision from to Make task YAML file support non-string (and nested) variables.May 23 2014, 12:18 PM
kparal updated this object.
kparal edited the test plan for this revision. (Show Details)
kparal added reviewers: tflink, jskladan, mkrizek.

Given the description of the review, it looks good to me so far

jskladan added inline comments.May 23 2014, 1:38 PM
jskladan added inline comments.May 23 2014, 1:40 PM
libtaskotron/taskyaml.py
51

*sigh* it is not safe_replace but safe_substitute of course... Next time I'll check docs before posting, I promise :)

kparal updated this revision to Diff 302.May 27 2014, 10:25 AM
  • use string.Template instead
  • adjust documentation

Good call, @jskladan, thank you. I still need to do some things manually (when I need to return an object, not a string), but it simplifies the code. And I can use a regex expression from the standard library rather than my own.

This has several effects:

  • From now on, both $variable and ${variable} syntaxes are supported now.
  • You need to to escape dollar signs by doubling them $$not_a_variable. I completely forgot about escaping in my initial patch. This gives folks the possibility to do so.
  • Invalid variable syntax is rejected immediately, rather than silently ignored. I consider this a benefit.
  • I don't plan to support whitespace inside curly braces, I'd need to diverge in a big way from the standard library functions, I don't see it cost-effective.

I have adjusted documentation and dependencies as well.

kparal updated this revision to Diff 303.May 27 2014, 10:38 AM
  • simplify pattern matching, in our case it works the same
tflink accepted this revision.May 27 2014, 1:28 PM

Looks good to me.

Tested with WIP depcheck (dict passing between actions) and it works great.

This revision is now accepted and ready to land.May 27 2014, 1:28 PM
kparal closed this revision.May 27 2014, 2:43 PM
kparal updated this revision to Diff 307.

Closed by commit rLTRN1f9cc58e1a88 (authored by @kparal).