summaryrefslogtreecommitdiffstats
path: root/roles/mirrormanager
Commit message (Collapse)AuthorAgeFilesLines
...
* Change the MM public key to user_home_tAdrian Reber2015-12-171-0/+7
| | | | Signed-off-by: Adrian Reber <adrian@lisas.de>
* Missing quotesAdrian Reber2015-12-171-1/+1
|
* Add mirrormanager public key to frontends to transfer statistics resultsAdrian Reber2015-12-171-0/+8
| | | | Signed-off-by: Adrian Reber <adrian@lisas.de>
* Install mirrormanager2-statistics on the backendAdrian Reber2015-12-171-0/+2
| | | | Signed-off-by: Adrian Reber <adrian@lisas.de>
* Switch to new propagation scanningAdrian Reber2015-12-172-16/+40
| | | | Signed-off-by: Adrian Reber <adrian@lisas.de>
* Disable statistics until the necessary data is availableAdrian Reber2015-12-171-3/+3
| | | | Signed-off-by: Adrian Reber <adrian@lisas.de>
* The mirrorlist_server no longer supports --debugAdrian Reber2015-12-161-1/+1
| | | | | | | | With the '--debug' switch the client access were logged to syslog and to a logfile. The latest MM2 no longer logs this to syslog and removed the '--debug' switch. Signed-off-by: Adrian Reber <adrian@lisas.de>
* Revert mirrorlist_server hotfix which is included in the new releaseAdrian Reber2015-12-161-6/+0
| | | | Signed-off-by: Adrian Reber <adrian@lisas.de>
* Prepare for new MM2 releaseAdrian Reber2015-12-163-4/+50
| | | | | | | The new MM2 release includes different usage visualizations. This prepares the MM2 frontend system to handle those visualizations. Signed-off-by: Adrian Reber <adrian@lisas.de>
* Run secondary umdl hourlyPatrick Uiterwijk2015-12-091-2/+2
| | | | Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
* Always require UMDL for secondary for nowPatrick Uiterwijk2015-12-091-1/+4
| | | | Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
* Make mirrorlists xz compress their gigantic messages logsKevin Fenzi2015-11-222-0/+20
|
* Enable mirrorlist-server loggingAdrian Reber2015-11-123-0/+19
| | | | | | | | | | With the logs from the mirrorlist-server logging it is possible to create country/repository/architecture statistics. The code which creates the actual statistics is partially already included into mirrormanager. Signed-off-by: Adrian Reber <adrian@lisas.de>
* Decrease number of parallel crawlers to 20Adrian Reber2015-11-121-2/+1
|
* Replace all restart httpd with reload httpdPatrick Uiterwijk2015-11-041-1/+1
| | | | Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
* Decrease the number of mm_crawler threadsPatrick Uiterwijk2015-11-011-1/+1
| | | | | | | | | | It seems like an update caused the crawler to use slightly more memory than before, meaning the previous tuning of 27 threads no longer fits in the server's memory. This patch brings it down to 23, which is for now known-good. We should look again at what values to use after freeze. Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
* Apply hotfix to make mirrorlist threadPatrick Uiterwijk2015-10-181-1/+1
|
* Up mirrorlist servers max apache workers to 900 from the default of 256. We ↵Kevin Fenzi2015-10-151-0/+3
| | | | are hitting this now with the last change.
* Increase the number of possible child processesAdrian Reber2015-10-151-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | The mirrorlist-server is the process which has the mirrorlist data loaded and which is accessed by the public facing mirrorlist_client.wsgi. The mirrorlist-server uses the ForkingUnixStreamServer which has a default of max_children = 40. (https://hg.python.org/cpython/file/2.7/Lib/SocketServer.py#l516) Looking at the code of ForkingUnixStreamServer it says at https://hg.python.org/cpython/file/2.7/Lib/SocketServer.py#l523 # If we're above the max number of children, wait and reap them until # we go back below threshold. Note that we use waitpid(-1) below to be # able to collect children in size(<defunct children>) syscalls instead # of size(<children>): the downside is that this might reap children # which we didn't spawn, which is why we only resort to this when we're # above max_children. As we are running the wsgi with processes=45 this sounds like it can lead to situation where it might just hang. This increases max_children to 80 and processes to 60. Signed-off-by: Adrian Reber <adrian@lisas.de>
* Move all puppet_private stuff to ansible private so we can stop using puppet ↵Kevin Fenzi2015-09-251-1/+1
| | | | private.
* Only do the propagation test on mm-crawler01Adrian Reber2015-09-081-0/+2
|
* Change propagation test scripts to run at 15 and 45Adrian Reber2015-09-081-4/+4
|
* Added propagation tests cronjobsAdrian Reber2015-09-081-0/+9
|
* Create directory used to store the result of the repomd.xml propagation testAdrian Reber2015-09-081-0/+6
|
* Try lowering it to 100Kevin Fenzi2015-09-051-1/+1
|
* Try setting the mm-frontend max requests down to 500.Kevin Fenzi2015-09-041-1/+1
|
* Reduce number of crawlers from 29 to 27Adrian Reber2015-08-301-2/+2
|
* Reduce the number of parallel crawled hosts to 29.Adrian Reber2015-07-291-2/+2
| | | | | | | There has been a report that the MM database was not updated correctly and dmesg on mm-crawler01 shows three OOM killed crawlers. https://fedorahosted.org/fedora-infrastructure/ticket/4845
* Decrease number of parallel crawls from 31 to 30.Adrian Reber2015-07-281-2/+2
|
* Install python-GeoIP for the new crawler.Adrian Reber2015-07-261-0/+1
| | | | | | | | | The latest crawler needs python-GeoIP for continent specific crawls. This should be a dependency of the mirrormanager2-crawler package but as it is not python-GeoIP has to be installed manually. For some reason the package was installed on mm-crawler02 but not on mm-crawler01. All mirrors that should have been crawled from mm-crawler01 have not been crawled for the last few days.
* Delete the frontend and pacake submodule of MirrorManager - they referred to MM1Pierre-Yves Chibon2015-07-156-265/+0
|
* Define the UMDL_PREFIX for umdl2 on stgPierre-Yves Chibon2015-07-151-0/+4
|
* Also switch to fedmsg based umdl crawl for SecondaryAdrian Reber2015-07-023-11/+34
|
* Run umdl for each category separately.Adrian Reber2015-06-293-12/+32
| | | | | | | | From now on each umdl category is run separately at different times. The categories 'Fedora Linux' and 'Fedora EPEL' are started every 30 minutes and if there has been a sync available fedmsg umdl is run for just that category. The remaining categories are run once per day at different times (00:00, 08:00, 16:00).
* Use the new script to better distribute the load between the crawlers.Adrian Reber2015-06-282-20/+4
|
* Reduce number of parallel crawlers from 32 to 31Adrian Reber2015-06-281-3/+3
| | | | Only wait 2 hours before starting the crawl on the second crawler.
* Renaming mm2_get-highest-active-host-id to mm2_get-mirrors-to-crawl.Adrian Reber2015-06-281-0/+0
|
* Better distribution of mirrors to crawl.Adrian Reber2015-06-281-2/+27
| | | | | | | | | | | The script mm2_get-highest-active-host-id used to return the highest ID of the active mirrors. This number was divided by the number of active crawlers and then each crawler got its share of mirrors to crawl. This did not take into account that more active mirrors are in the higher IDs as old mirror IDs are not re-used and thus one crawler was getting much more mirrors to crawl than another. The new script (which will be renamed) now divides the list correctly by returning exactly the fraction which each crawler should crawl.
* Revert "Disable umdl for a manual 'umdl --delete' run."Adrian Reber2015-06-281-1/+1
| | | | | | | This reverts commit 1b409809ebdc490a67e450a8d79ded2d00b28c3e. Manual umdl runs with '--delete' have finished. Now re-enabled cron based umdl runs.
* Disable umdl for a manual 'umdl --delete' run.Adrian Reber2015-06-251-1/+1
|
* Don't install this cron job in staging if we aren't setting up rsync in stg.Kevin Fenzi2015-06-221-0/+1
|
* Only sync current logs and not the old ones.Adrian Reber2015-06-221-1/+1
|
* Again the wrong order, first create the user, then chown.Adrian Reber2015-06-221-6/+6
|
* The mirrormanager2-frontend RPM does not create a mirrormanager userAdrian Reber2015-06-221-0/+6
| | | | Create the user manually for the crawler log sync.
* First create the directory than apply selinux changes.Adrian Reber2015-06-221-6/+6
|
* Forgot 'tags:'Adrian Reber2015-06-221-0/+1
|
* Sync crawler logs every hourAdrian Reber2015-06-223-0/+27
| | | | Sync crawler logs every hour from crawlers to the frontend.
* Add log/mirrormanager/crawler to apache configAdrian Reber2015-06-222-0/+11
|
* The crawler logs are now on the frontend systemAdrian Reber2015-06-222-33/+0
| | | | | and therefore apache and its configuration can be removed from the crawlers.
* Remove fedmsg-relay from mm2 backend and crawler.Ralph Bean2015-06-202-2/+0
|
generated by cgit (git 2.34.1) at 2025-09-30 18:45:48 +0000