It bugs me that it's not very visible if we've messed up wiki
reporting: we really have to be paying attention to spot that
we don't have reporting hooked up for a test, or if we have it
hooked up but we didn't quite get the settings right and
wikitcms won't manage to report the result. The latter case is
partly logged, but that still means we have to go poke through
the server logs to spot it, and there are in fact cases where
wikitcms won't report insufficient / incorrect restups (I
should probably fix that), and it means we don't catch the
issue till after we've committed the changes; we can't check
before commit.
So I fixed it! Here's a script that checks the TESTSUITES and
TESTCASES dicts correspond, that we have an entry in TESTSUITES
for every test suite in the openQA templates, and that wikitcms
can actually find a resultrow and environment for a set of
faked-up 'passed jobs' derived from the openQA templates. Now
we can use this to sanity check commits that add wiki reports.
We also fix an error it discovered, for KDE_Package_Install.
The result would actually have gotten reported anyway in this
case (as wikitcms doesn't bother checking the restup's 'env'
value if the resultrow only has *one* env), but it's good to
be correct.
I guess this is *one* way to spend an 8 hour train ride...