Differential D319

Symlink artefacts dir to 'all' directory to provide canonical path
ClosedPublic
Actions

Authored by mkrizek on Mar 25 2015, 2:45 PM.

Details

Reviewers
tflink
jskladan
Maniphest Tasks
T442: ansible-ize execdb
T445: Show link to Artifacts in ExecDB
Summary

When storing artefacts, there is no canonical path for a given UUID, because the directories are scattered in the per-day directories.
This patch makes sure, that there is a canonical path /all/UUID, so we can reference links without extra knowledge of the execution details.

Test Plan

Untested so far

Diff Detail

Lint
Lint Skipped
Unit
Unit Tests Skipped
jskladan retitled this revision from to Symlink artefacts dir to 'all' directory to provide canonical path.Mar 25 2015, 2:45 PM
jskladan updated this object.
jskladan edited the test plan for this revision. (Show Details)
jskladan added reviewers: tflink, mkrizek.
jskladan added a project: infrastructure.
jskladan updated this revision to Diff 873.Mar 25 2015, 3:02 PM

Found out a problem with ExecDB's settings on the way - copy-pasting from resultsdb was not the ideal idea...

mkrizek accepted this revision.Mar 26 2015, 9:51 AM

LGTM

This revision is now accepted and ready to land.Mar 26 2015, 9:51 AM
mkrizek commandeered this revision.Mar 31 2015, 10:37 AM
mkrizek edited reviewers, added: jskladan; removed: mkrizek.
mkrizek added a task: T442: ansible-ize execdb.

I need to push this to hopefully fix T442.

This revision now requires review to proceed.Mar 31 2015, 10:37 AM
tflink added a comment.Mar 31 2015, 1:19 PM

Doesn't this kind of undo the change to use date-based dirs? I realize that they're still around but we're still going to have a URL that will take forever to load before too long.

I wonder if we should disable the autoindex on the symlinked dir since it shouldn't be used for anything

mkrizek added a comment.Apr 1 2015, 10:41 AM
In D319#5815, @tflink wrote:

Doesn't this kind of undo the change to use date-based dirs? I realize that they're still around but we're still going to have a URL that will take forever to load before too long.

I wonder if we should disable the autoindex on the symlinked dir since it shouldn't be used for anything

Yeah, thanks for reminding me this, I disabled directory listing on the all dir.

jskladan accepted this revision.Apr 3 2015, 8:18 AM

Probably ready to be closed, since it was merged to the infra-ansible already.

This revision is now accepted and ready to land.Apr 3 2015, 8:18 AM
mkrizek closed this revision.Apr 3 2015, 8:26 AM

Revision Contents

Diff 873

View Options

inventory/group_vars/resultsdb-dev

Context not available.
30execdb_endpoint: 'execdb' 30​execdb_endpoint: 'execdb'
31execdb_db_name: execdb_dev 31​execdb_db_name: execdb_dev
32 32​
33​external_hostname: taskotron-dev.fedoraproject.org
34​
33allowed_hosts: 35​allowed_hosts:
34 - 10.5.124 36​ - 10.5.124
35 37​
Context not available.
View Options

roles/taskotron/buildmaster-configure/templates/taskotron.master.cfg.j2

Context not available.
199 return datetime.datetime.now().strftime("%Y%m%d") 199​ return datetime.datetime.now().strftime("%Y%m%d")
200 200​
201# move artifacts dir 201​# move artifacts dir
202factory.addStep(MasterShellCommand(command=Interpolate('mkdir -p -m 0755 {{ public_artifacts_dir }}/%(kw:today)s && mv {{ public_artifacts_dir }}/%(prop:uuid)s/ {{ public_artifacts_dir }}/%(kw:today)s/', today=today), 202​factory.addStep(MasterShellCommand(command=Interpolate('mkdir -p -m 0755 {{ public_artifacts_dir }}/%(kw:today)s && mkdir -p -m 0755 {{ public_artifacts_dir }}/all && mv {{ public_artifacts_dir }}/%(prop:uuid)s/ {{ public_artifacts_dir }}/%(kw:today)s/ && ln -s {{ public_artifacts_dir }}/%(kw:today)s/%(prop:uuid)s {{ public_artifacts_dir }}/all/', today=today),
203 descriptionDone=['Move artifacs dir'])) 203​ descriptionDone=['Move artifacs dir']))
204{% else %} 204​{% else %}
205# capture the taskotron log 205​# capture the taskotron log
Context not available.
View Options

roles/taskotron/execdb/templates/settings.py.j2

Context not available.
18LOGFILR = '/var/log/execdb/execdb.log' 18​LOGFILR = '/var/log/execdb/execdb.log'
19SYSLOG_LOGGING = False 19​SYSLOG_LOGGING = False
20STREAM_LOGGING = True 20​STREAM_LOGGING = True
21​
22​BUILDBOT_FRONTPAGE_URL = 'http://{{ external_hostname }}/taskmaster'
23​RESULTSDB_FRONTPAGE_URL = 'http://{{ external_hostname }}/resultsdb'
24​ARTIFACTS_BASE_URL = 'http://{{ external_hostname }}/artifacts/all'
Context not available.