Merge branch 'master' of git://git.fedorahosted.org/git/abrt

1 files changed, 87 insertions, 84 deletions

diff --git a/doc/abrt-retrace-server.texi b/doc/abrt-retrace-server.texi
index 95dca232..29d41414 100644
--- a/doc/abrt-retrace-server.texi
+++ b/doc/abrt-retrace-server.texi
@@ -50,11 +50,18 @@ Debugger (GDB), that is commonly used to analyze coredumps on free
 operating systems, expects that the system analyzing the coredump is
 identical to the system where the program crashed. Software updates
 often break this assumption even on the system where the crash occured,
-making the coredump analyzable only with significant effort.
-
-Retrace server solves this problem for Fedora 14+ and RHEL 6+ operating
-systems, and allows developers to analyze coredumps without having
-access to the machine where the crash occurred.
+making the coredump analyzable only with significant
+effort. Furthermore, older versions of software packages are often
+removed from software repositories, including the packages with
+debugging symbols, so the package with debugging symbols is often not
+available when user needs to install it for coredump analysis. Packages
+with the debugging symbols are large, requiring a lot of free space and
+causing problems with downloading them via unreliable internet
+connection.
+
+Retrace server solves these problems for Fedora 14+ and RHEL 6+
+operating systems, and allows developers to analyze coredumps without
+having access to the machine where the crash occurred.
 
 Retrace server is usually run as a service on a local network, or on
 Internet. A user sends a coredump together with some additional
@@ -385,93 +392,58 @@ the area GDB operates on.
 No special care is taken to avoid the possibility that GDB will not run
 with the set of packages (fixed versions) as provided by coredump. It is
 expected that any combination of packages user might use in a released
-system should satisfy the needs of some version of GDB. Yum selects the
+system satisfies the needs of some version of GDB. Yum selects the
 newest possible version which has its requirements satisfied.
 
 @node Task cleanup
 @chapter Task cleanup
 
-Tasks that were created more than 5 days ago are deleted, because tasks
-occupy disk space (not so much space, as the coredumps are deleted after
-the retrace, and only backtraces and configuration remain). A shell
-script @command{abrt-retrace-clean} must check the creation time and
-delete the directories in @file{/var/spool/abrt-retrace/}. It is
-supposed that the server administrator sets @command{cron} to call the
-script once a day. This assumption must be mentioned in the
-@command{abrt-retrace-clean} manual page.
-
-The database containing packages and processing times should also
-be regularly pruned to remain small and provide data quickly. The
-cleanup script should delete some rows for packages with too many
-entries:
-@enumerate
-@item
-get a list of packages from the database: @code{SELECT DISTINCT
-package, release FROM package_time}
-@item
-for every package, get the row count: @code{SELECT COUNT(*) FROM
-package_time WHERE package == '??' AND release == '??'}
-@item
-for every package with the row count larger than 100, some rows
-most be removed so that only the newest 100 rows remain in the
-database:
-@itemize
- @item
-  to get highest row id which should be deleted,
-  execute @code{SELECT id FROM package_time WHERE package == '??' AND
-  release == '??' ORDER BY id LIMIT 1 OFFSET ??}, where the
-  @code{OFFSET} is the total number of rows for that single
-  package minus 100
- @item
-  then all the old rows can be deleted by executing @code{DELETE
-  FROM package_time WHERE package == '??' AND release == '??' AND id
-  <= ??}
-@end itemize
-@end enumerate
+It is neccessary to watch and limit the resource usage of tasks for a
+retrace server to remain operational. This is performed by the
+@command{abrt-retrace-cleanup} tool. It is supposed that the server
+administrator sets @command{cron} to run the tool every hour.
+
+Tasks that were created more than 120 hours (5 days) ago are
+deleted. The limit can be changed by the @var{DeleteTaskAfter} option in
+the server configuration file. Coredumps are deleted when the retrace
+process is finished, and only backtraces, logs, and configuration remain
+available for every task until the cleanup. The
+@command{abrt-retrace-cleanup} checks the creation time and deletes the
+directories in @file{/var/spool/abrt-retrace/}.
+
+Tasks running for more than 1 hour are terminated and removed from the
+system. Tasks for which the @command{abrt-retrace-worker} crashed for
+some reason without marking the task as finished are also removed.
 
 @node Package repository
 @chapter Package repository
 
-We should support every Fedora release with all packages that ever
-made it to the updates and updates-testing repositories. In order to
-provide all that packages, a local repository is maintained for every
-supported operating system. The debuginfos might be provided by a
-debuginfo server in future (it will save the server disk space). We
-should support the usage of local debuginfo first, and add the
-debuginfofs support later.
+Retrace server is able to support every Fedora release with all packages
+that ever made it to the updates and updates-testing repositories. In
+order to provide all that packages, a local repository needs to be
+maintained for every supported operating system.
 
 A repository with Fedora packages must be maintained locally on the
 server to provide good performance and to provide data from older
-packages already removed from the official repositories. We need a
-package downloader, which scans Fedora servers for new packages, and
-downloads them so they are immediately available.
-
-Older versions of packages are regularly deleted from the updates
-and updates-testing repositories. We must support older versions of
-packages, because that is one of two major pain-points that the
-retrace server is supposed to solve (the other one is the slowness of
-debuginfo download and debuginfo disk space requirements).
-
-A script abrt-reposync must download packages from Fedora
-repositories, but it must not delete older versions of the
-packages. The retrace server administrator is supposed to call this
-script using cron every ~6 hours. This expectation must be documented
-in the abrt-reposync manual page. The script can use use wget, rsync,
-or reposync tool to get the packages. The remote yum source
-repositories must be configured from a configuration file or files
-(@file{/etc/yum.repos.d} might be used).
-
-When the abrt-reposync is used to sync with the Rawhide repository,
-unneeded packages (where a newer version exists) must be removed after
-residing one week with the newer package in the same repository.
-
-All the unneeded content from the newly downloaded packages should be
-removed to save disk space and speed up chroot creation. We need just
-the binaries and dynamic libraries, and that is a tiny part of package
-contents.
-
-The packages should be downloaded to a local repository in
-@file{/var/cache/abrt-repo/@{fedora12,fedora12-debuginfo,...@}}.
+packages already removed from the official repositories. Retrace server
+contains a tool @command{abrt-retrace-reposync}, which is a package
+downloader scanning Fedora servers for new packages, and downloading
+them so they are immediately available.
+
+Older versions of packages are regularly deleted from the updates and
+updates-testing repositories. Retrace server supports older versions of
+packages, as this is one of major pain-points that the retrace server is
+supposed to solve.
+
+The @command{abrt-retrace-reposync} downloads packages from Fedora
+repositories, and it does not delete older versions of the packages. The
+retrace server administrator is supposed to call this script using cron
+approximately every 6 hours. The script uses @command{rsync} to get the
+packages and @command{createrepo} to generate respository metadata.
+
+The packages are downloaded to a local repository in
+@file{/var/cache/abrt-retrace/}. The location can be changed via the
+@var{RepoDir} option in the server configuration file.
 
 @node Traffic and load estimation
 @chapter Traffic and load estimation
@@ -488,8 +460,7 @@ or down. New, untested versions of software are added to Fedora, but
 on the other side most software matures and becomes less crashy.  So
 let's assume that the bug count stays approximately the same.
 
-Test crashes (see that we should probably use @code{`xz -2`}
-to compress coredumps):
+Test crashes (see why we use @code{`xz -2`} to compress coredumps):
 @itemize
 @item
 firefox with 7 tabs (random pages opened), coredump size 172 MB
@@ -635,8 +606,8 @@ from within the chroot, unable to access the retrace server's
 environment. We should consider setting a disk quota to every chroot
 directory, and limit the GDB access to resources using cgroups.
 
-SELinux policy should be written for both the retrace server's HTTP
-interface, and for the retrace worker.
+SELinux policy exists for both the retrace server's HTTP interface, and
+for the retrace worker.
 
 @node Clients
 @section Clients
@@ -800,4 +771,36 @@ database provides good estimations after some time anyway
 (@ref{Task cleanup} chapter describes how the
 data are pruned).
 
+@section Keep the database with statistics small
+The database containing packages and processing times should also be
+regularly pruned to remain small and provide data quickly. The cleanup
+script should delete some rows for packages with too many entries:
+@enumerate
+@item
+get a list of packages from the database: @code{SELECT DISTINCT package,
+release FROM package_time}
+@item
+for every package, get the row count: @code{SELECT COUNT(*) FROM
+package_time WHERE package == '??' AND release == '??'}
+@item
+for every package with the row count larger than 100, some rows most be
+removed so that only the newest 100 rows remain in the database:
+@itemize
+@item
+to get highest row id which should be deleted, execute @code{SELECT id
+FROM package_time WHERE package == '??' AND release == '??' ORDER BY id
+LIMIT 1 OFFSET ??}, where the @code{OFFSET} is the total number of rows
+for that single package minus 100
+@item
+then all the old rows can be deleted by executing @code{DELETE FROM
+package_time WHERE package == '??' AND release == '??' AND id <= ??}
+@end itemize
+@end enumerate
+
+@section Support Fedora Rawhide
+When the @command{abrt-retrace-reposync} is used to sync with the
+Rawhide repository, unneeded packages (where a newer version exists)
+must be removed after residing one week with the newer package in the
+same repository.
+
 @bye