Differential D336

koji_utils: configure koji connection timeout to 5 minutes
ClosedPublic
Actions

Authored by kparal on Apr 10 2015, 12:17 PM.

Details

Reviewers
tflink
mkrizek
jskladan
Commits
rLTRNf820b5f0bb4d: koji_utils: configure koji timeout to 5 minutes
Summary

Otherwise the connection could have been waiting for up to 12 hours by
default. That's not fit for our purposes.

I'm trying to find out why this timed out:
http://taskotron-dev.fedoraproject.org/taskmaster/builders/x86_64/builds/6635/steps/runtask/logs/stdio

This is in Koji source code:

# set a default 12 hour connection timeout.
# Some Koji operations can take a long time to return, but after 12
# hours we can assume something is seriously wrong.
timeout = self.opts.setdefault('timeout',  60 * 60 * 12)

I think that a koji connection got stuck and waited for too long. I've already added some debug messages to upgradepath to help debug it next time. By configuring the timeout in koji_utils I hope we will avoid the problem completely.

Test Plan

couldn't really test this, but at least ran upgradepath manually to see if it works normally

Diff Detail

Repository
rLTRN libtaskotron
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
kparal retitled this revision from to koji_utils: configure koji timeout to 5 minutes.Apr 10 2015, 12:17 PM
kparal updated this object.
kparal edited the test plan for this revision. (Show Details)
kparal added reviewers: jskladan, mkrizek, tflink.
kparal retitled this revision from koji_utils: configure koji timeout to 5 minutes to koji_utils: configure koji connection timeout to 5 minutes.Apr 10 2015, 2:30 PM
mkrizek accepted this revision.Apr 10 2015, 3:34 PM
This revision is now accepted and ready to land.Apr 10 2015, 3:34 PM
jskladan accepted this revision.Apr 14 2015, 8:56 AM

If the timeout still triggers retry, it's good for me.

Closed by commit rLTRNf820b5f0bb4d: koji_utils: configure koji timeout to 5 minutes (authored by kparal). · Explain WhyApr 14 2015, 9:10 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

PathPackages
Mlibtaskotron/koji_utils.py (12 lines)

Diff 916

View Options

libtaskotron/koji_utils.py

Show All 26 Lines22class KojiClient(object):
27 27​
28 def __init__(self, koji_session=None): 28​ def __init__(self, koji_session=None):
29 '''Create a new KojiClient 29​ '''Create a new KojiClient
30 30​
31 :param koji_session: an existing Koji session instance or ``None`` if 31​ :param koji_session: an existing Koji session instance or ``None`` if
32 you want a new default session to be created 32​ you want a new default session to be created
33 :type koji_session: :class:`koji.ClientSession` 33​ :type koji_session: :class:`koji.ClientSession`
34 ''' 34​ '''
35 # make koji retry even for non-logged-in connections 35​ opts = {
36 # retry behavior can be configured via 'max_retries' (default 30) and 36​ # make koji retry even for non-logged-in connections.
37 # 'retry_interval' (default 20) opts keys 37​ # retry behavior can be configured via 'max_retries' (default 30)
38 opts = {'anon_retry': True} 38​ # and 'retry_interval' (default 20) opts keys
39​ 'anon_retry': True,
40​ # default Koji connection timeout is 12 hours, that's too high.
41​ # let's make it 5 minutes
42​ 'timeout': 60 * 5}
39 43​
40 self.session = (koji_session or 44​ self.session = (koji_session or
41 koji.ClientSession(config.get_config().koji_url, opts=opts)) 45​ koji.ClientSession(config.get_config().koji_url, opts=opts))
42 46​
43 47​
44 def rpms_to_build(self, rpms): 48​ def rpms_to_build(self, rpms):
45 '''Get list of koji build objects for the rpms. Order of koji objects 49​ '''Get list of koji build objects for the rpms. Order of koji objects
46 in this list is the same as order of the respective rpm objects. 50​ in this list is the same as order of the respective rpm objects.
▲ Show 20 Lines • Show All 213 Lines • Show Last 20 Lines