allows easier reproduction of depcheck runs. after installing doit, 'doit buildid=<BUILDID> depcheck' will spit out the required command to reproduce the run specified by the BUILDID of the run you're interested in on taskotron.fp.o
Details
Diff Detail
- Repository
- rDEPCK task-depcheck
- Branch
- feature/reproducing-runs-doit
- Lint
No Linters Available - Unit
No Unit Test Coverage
had a typo in the initial description - should have been doit buildid=<SOME BUILDID> depcheck
This uses doit to do the following:
- get buildinfo from taskotron buildbot (run date, koji tag etc.)
- find and download archived repodata that was used for that run
- download all rpms used in the original run (by scraping the output in taskotron.log)
- mash downloaded rpms
- print out commands needed to do the depcheck run locally
The files will be stored in a local directory named for the date of original run and the tag used (localruns/<tag>-<date>/)
Example using a run on f22-updates-testing:
$ doit buildid=56193 depcheck Getting closest branched base release repo Finding closest mash for f22-updates-testing . makedirs . fetchtaskotronlog . downloadrpms . mash . depcheck i386 depcheck command: python run_depcheck.py -f updates -d -a i386 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/repos/f22/i386 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/repos/f22-updates-testing/i386 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/newrepo-i386 /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/newrepo-i386 x86_64 depcheck command: python run_depcheck.py -f updates -d -a x86_64 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/repos/f22/x86_64 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/repos/f22-updates-testing/x86_64 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/newrepo-x86_64 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/repos/f22/i386 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/repos/f22-updates-testing/i386 -r /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/newrepo-i386 /home/tflink/code/taskotron/task-depcheck/localruns/f22-updates-testing-pending.201504081117/newrepo-x86_64
Thanks, that's useful. It seems it fails to download repo metadata for me, but I haven't debugged why:
$ doit buildid=60951 depcheck Finding closest mash for f21-updates . makedirs . fetchtaskotronlog fetching https://taskotron.fedoraproject.org/taskmaster/builders/x86_64/builds/60951/steps/runtask/logs/taskotron.log/text as /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/taskotron.log . downloadrpms . mash . depcheck i386 depcheck command: python run_depcheck.py -f updates -d -a i386 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21-updates/i386 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-i386 /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-i386 x86_64 depcheck command: python run_depcheck.py -f updates -d -a x86_64 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/x86_64 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21-updates/x86_64 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-x86_64 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21-updates/i386 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-i386 /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-x86_64 $ python run_depcheck.py -f updates -d -a i386 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21-updates/i386 -r /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-i386 /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-i386 DEBUG:depcheck:Preparing metadata download DEBUG:depcheck: - '/home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-i386': 'i686', '/home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/newrepo-i386' DEBUG:depcheck: - '/home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386': 'i686', '/home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386' DEBUG:depcheck: - '/home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21-updates/i386': 'i686', '/home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21-updates/i386' INFO:depcheck:Getting metadata... Traceback (most recent call last): File "run_depcheck.py", line 59, in <module> depcheck.depcheck_cli() File "/home/kparal/devel/taskotron/task-depcheck/depcheck/__init__.py", line 168, in depcheck_cli run(args.rpm, arch, repos, args.format, args.workdir) File "/home/kparal/devel/taskotron/task-depcheck/depcheck/__init__.py", line 115, in run repomd_files = md_handle.repodata File "/home/kparal/devel/taskotron/task-depcheck/depcheck/metadata.py", line 253, in repodata self.verify() File "/home/kparal/devel/taskotron/task-depcheck/depcheck/metadata.py", line 243, in verify self._handle_repo(reponame) File "/home/kparal/devel/taskotron/task-depcheck/depcheck/metadata.py", line 228, in _handle_repo repo_targetdir) File "/home/kparal/devel/taskotron/task-depcheck/depcheck/metadata.py", line 192, in _handle_local_repofiles local_repofiles = self._find_repo_mdfiles(repopath) File "/home/kparal/devel/taskotron/task-depcheck/depcheck/metadata.py", line 147, in _find_repo_mdfiles raise Exception("Not able to find requested metadata file for %s in %s" % (mdtype, repodir)) Exception: Not able to find requested metadata file for filelists in /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386 $ find /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386/ /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386/ /home/kparal/devel/taskotron/task-depcheck/localruns/f21-updates-pending.201504221402/repos/f21/i386/repodata
I wanted to fetch https://taskotron.fedoraproject.org/resultsdb/results/1380421 , because it is quite small.
dodo.py | ||
---|---|---|
6 | Could you perhaps just very shortly document the purpose of this script as a module docstring? Just copy&paste the description from the phabricator. Thanks. | |
38 | I guess this needs to be added to .gitignore. As well as .doit.db, it seems. | |
310 | This print doesn't seem to work. Could you also print some kind of progress, like Downloading (1/29): NVR? |
looks like there's a missing dep for task_depcheck - if you run 'fetchrepodata', it should work
I've been futzing with the script today, trying to make it behave more like taskotron, will post changes when I get them working better
Could you perhaps just very shortly document the purpose of this script as a module docstring? Just copy&paste the description from the phabricator. Thanks.