tdb: Reduce freelist contention

In a metadata-intensive benchmark we have seen the locking.tdb freelist to be
one of the central contention points. This patch removes most of the contention
on the freelist. Ages ago we already reduced freelist contention by using the
even much older DEAD records: If TDB_VOLATILE is set, don't directly put
deleted records on the freelist, but just mark a few of them just as DEAD. The
next new record can them re-use that space without consulting the freelist.

This patch builds upon the DEAD records: If we need space and the freelist is
busy, instead of doing a blocking wait on the freelist, start looking into
other chains for DEAD records and steal them from there. This way every hash
chain becomes a small freelist. Just wander around the hash chains as long as
the freelist is still busy.

With this patch and the tdb mutex patch (following hopefully some time soon)
you can see a heavily busy clustered smbd run without locking.tdb futex
syscalls.

Signed-off-by: Volker Lendecke <vl@samba.org>
Reviewed-by: Michael Adam <obnox@samba.org>
Reviewed-by: Stefan Metzmacher <metze@samba.org>

1 files changed, 1 insertions, 19 deletions

diff --git a/lib/tdb/common/tdb.c b/lib/tdb/common/tdb.c
index a7111dea96..ba1c98edbe 100644
--- a/lib/tdb/common/tdb.c
+++ b/lib/tdb/common/tdb.c
@@ -550,26 +550,8 @@ static int _tdb_store(struct tdb_context *tdb, TDB_DATA key,
 		}
 	}
 
-	/*
-	 * We have to allocate some space from the freelist, so this means we
-	 * have to lock it. Use the chance to purge all the DEAD records from
-	 * the hash chain under the freelist lock.
-	 */
-
-	if (tdb_lock(tdb, -1, F_WRLCK) == -1) {
-		goto fail;
-	}
-
-	if ((tdb->max_dead_records != 0)
-	    && (tdb_purge_dead(tdb, hash) == -1)) {
-		tdb_unlock(tdb, -1, F_WRLCK);
-		goto fail;
-	}
-
 	/* we have to allocate some space */
-	rec_ptr = tdb_allocate(tdb, key.dsize + dbuf.dsize, &rec);
-
-	tdb_unlock(tdb, -1, F_WRLCK);
+	rec_ptr = tdb_allocate(tdb, hash, key.dsize + dbuf.dsize, &rec);
 
 	if (rec_ptr == 0) {
 		goto fail;