summaryrefslogtreecommitdiffstats
path: root/lib/tdb/common
Commit message (Collapse)AuthorAgeFilesLines
* tdb: allow transactions on on tdb's with TDB_MUTEX_LOCKINGStefan Metzmacher2014-12-191-1/+1
| | | | | | | | | | | | | There's no real reason to disallow transactions as the allrecord lock is also available with mutexes enabled. E.g. ctdbd requires transactions also on non-persistent databases opened with TDB_CLEAR_IF_FIRST and TDB_MUTEX_LOCKING. Bug: https://bugzilla.samba.org/show_bug.cgi?id=11004 Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Amitay Isaacs <amitay@gmail.com>
* tdb: Fix tdb_runtime_check_for_robust_mutexes()Volker Lendecke2014-11-261-4/+4
| | | | | | | | | | When using exit() instead of _exit(), the child will flush buffered stdout (and other stdio) content that it inherited from the parent process. In make test, this led to duplicate output from net registry which then confused the blackbox selftest. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: allow tdb_open_ex() with O_RDONLY of TDB_FEATURE_FLAG_MUTEX tdbs.Stefan Metzmacher2014-11-201-9/+9
| | | | | | | Bug: https://bugzilla.samba.org/show_bug.cgi?id=10781 Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Michael Adam <obnox@samba.org>
* tdb: Fix a commentVolker Lendecke2014-09-301-1/+1
| | | | | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Jeremy Allison <jra@samba.org> Autobuild-Date(master): Tue Sep 30 23:08:07 CEST 2014 on sn-devel-104
* tdb: Improve wording in a commentVolker Lendecke2014-09-251-1/+1
| | | | | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Jeremy Allison <jra@samba.org> Autobuild-Date(master): Thu Sep 25 00:59:16 CEST 2014 on sn-devel-104
* tdb: defragment the freelist in tdb_allocate_from_freelist()Michael Adam2014-06-261-0/+55
| | | | | | | | | | | | | | | | | | | | While we are traversing the freelist anyways, merge a record with the left if it is also a free list record. That partially makes up for the fragmentation introduced by the lack of merging with right records in tdb_free(). Note there is a potential slight downside: If the left record we merge the current record into was earlier in the chain and has hence already been met in traverse, then we can not use the enlarged record even if it might be a new best fit. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org> Autobuild-User(master): Volker Lendecke <vl@samba.org> Autobuild-Date(master): Thu Jun 26 12:16:03 CEST 2014 on sn-devel-104
* tdb: use tdb_freelist_merge_adjacent in tdb_freelist_size()Michael Adam2014-06-261-4/+29
| | | | | | | | So that we automatically defragment the free list when freelist_size is called (unless the database is read only). Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: add tdb_freelist_merge_adjacent()Michael Adam2014-06-261-0/+59
| | | | | | | | | | | | | | | | This is intended to be called to reduce the fragmentation in the freelist. This is to make up the deficiency of the freelist to be not doubly linked. If the freelist were doubly linked, we could easily avoid the creation of adjacent freelist entries. But with the current singly linked list, it is only possible to cheaply merge a new free record into a freelist entry on the left, not on the right... This can be called periodically, e.g. in the vacuuming process of a ctdb cluster. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: add utility function check_merge_ptr_with_left_record()Michael Adam2014-06-261-0/+53
| | | | | | | | Variant of check_merge_with_left_record() that reads the record itself if necessary. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: simplify tdb_free() using check_merge_with_left_record()Michael Adam2014-06-261-17/+9
| | | | | Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: add utility function check_merge_with_left_record()Michael Adam2014-06-261-0/+50
| | | | | | | | Check whether the record left of a given freelist record is also a freelist record, and if so, merge the two records. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: improve comments for tdb_free().Michael Adam2014-06-261-7/+11
| | | | | Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: factor merge_with_left_record() out of tdb_free()Michael Adam2014-06-261-7/+37
| | | | | Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: fix debug message in tdb_free()Michael Adam2014-06-261-1/+1
| | | | | Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: reduce indentation in tdb_free() for merging leftMichael Adam2014-06-261-18/+23
| | | | | Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: increase readability of read_record_on_left()Michael Adam2014-06-261-27/+41
| | | | | | | | by using early returns and better variable names, and reducing indentation. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: factor read_record_on_left() out of tdb_free()Michael Adam2014-06-261-30/+49
| | | | | Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: add TDB_MUTEX_LOCKING supportVolker Lendecke2014-05-228-14/+1312
| | | | | | | | | | | | | | | | | | | | | | | | This adds optional support for locking based on shared robust mutexes. The caller can use the TDB_MUTEX_LOCKING flag together with TDB_CLEAR_IF_FIRST after verifying with tdb_runtime_check_for_robust_mutexes() that it's supported by the current system. The caller should be aware that using TDB_MUTEX_LOCKING implies some limitations, e.g. it's not possible to have multiple read chainlocks on a given hash chain from multiple processes. Note: that this doesn't make tdb thread safe! Pair-Programmed-With: Stefan Metzmacher <metze@samba.org> Pair-Programmed-With: Michael Adam <obnox@samba.org> Signed-off-by: Volker Lendecke <vl@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: introduce tdb->hdr_ofsVolker Lendecke2014-05-224-36/+143
| | | | | | | | | | | | | | This makes it possible to have some extra headers before the real tdb content starts in the file. This will be used used e.g. to implement locking based on robust mutexes. Pair-Programmed-With: Stefan Metzmacher <metze@samba.org> Pair-Programmed-With: Michael Adam <obnox@samba.org> Signed-off-by: Volker Lendecke <vl@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: introduce TDB_SUPPORTED_FEATURE_FLAGSStefan Metzmacher2014-05-224-2/+39
| | | | | | | | | | | | | This will allow to store a feature mask in the tdb header on disk, so that openers can check if they can handle the features other openers are using. Pair-Programmed-With: Volker Lendecke <vl@samba.org> Pair-Programmed-With: Michael Adam <obnox@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Signed-off-by: Volker Lendecke <vl@samba.org> Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: use asprintf() to simplify tdb_summary()Stefan Metzmacher2014-05-221-7/+6
| | | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: return ENOSYS if the tdb was created with spinlocks.Stefan Metzmacher2014-05-121-0/+1
| | | | | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Autobuild-User(master): Stefan Metzmacher <metze@samba.org> Autobuild-Date(master): Mon May 12 21:07:04 CEST 2014 on sn-devel-104
* tdb: consolidate tdb allocation code - re-use dead records at hash top.Michael Adam2014-04-092-43/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | When in tdb_store we re-use a dead record reactivated from the target hash chain itself, we currently leave it in its place in the chain. When we re-use a dead record from a different chain or from the freelist instead, we insert it at the beginning of the target chain. This patch changes the behaviour to always newly store a record at the beginning of the hash chain. This removes a special case and hence simplifies the allocation code. On the other hand side, it introduces two additioal tdb_ofs_write calls for the in-chain-case. Note the subtelty of the patch that by moving the case of the candidate record's chain as new case "i=0" into the for loop, we also reverse the order of the two steps in the for-loop body (non blocking freelist alloc and searching for dead record in a chain) in order to keep the overall order of execution identical. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Michael Adam <obnox@samba.org> Autobuild-Date(master): Wed Apr 9 10:37:08 CEST 2014 on sn-devel-104
* tdb: don't alter errno on success of tdb_open_ex()Stefan Metzmacher2014-04-021-0/+2
| | | | | Signed-off-by: Stefan Metzmacher <metze@samba.org> Reviewed-by: Andrew Bartlett <abartlet@samba.org>
* tdb: Reduce freelist contentionVolker Lendecke2014-03-183-30/+93
| | | | | | | | | | | | | | | | | | | | | | | In a metadata-intensive benchmark we have seen the locking.tdb freelist to be one of the central contention points. This patch removes most of the contention on the freelist. Ages ago we already reduced freelist contention by using the even much older DEAD records: If TDB_VOLATILE is set, don't directly put deleted records on the freelist, but just mark a few of them just as DEAD. The next new record can them re-use that space without consulting the freelist. This patch builds upon the DEAD records: If we need space and the freelist is busy, instead of doing a blocking wait on the freelist, start looking into other chains for DEAD records and steal them from there. This way every hash chain becomes a small freelist. Just wander around the hash chains as long as the freelist is still busy. With this patch and the tdb mutex patch (following hopefully some time soon) you can see a heavily busy clustered smbd run without locking.tdb futex syscalls. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: Make "tdb_purge_dead" internally publicVolker Lendecke2014-03-182-1/+2
| | | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: Make "tdb_find_dead" internally publicVolker Lendecke2014-03-182-3/+6
| | | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: Add "last_ptr" to tdb_find_deadVolker Lendecke2014-03-181-4/+13
| | | | | | | | Will be used soon to unlink a dead record from a chain Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: Move adding tailer space to tdb_find_deadVolker Lendecke2014-03-181-3/+4
| | | | | | | | | This aligns the tdb_find_dead API with the tdb_allocate API and thus makes it a bit easier to understand, at least for me. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: Do a best fit search for dead recordsVolker Lendecke2014-03-181-7/+13
| | | | | | | | | | Hash chains are (or can be made) short enough that a full search for the best-fitting dead record is feasible. The freelist can become much longer, there we don't do the full search but accept records which are too large. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: Don't purge records to a blocked freelistVolker Lendecke2014-03-181-1/+4
| | | | | | | | If the freelist is heavily contended, we should avoid accessing it Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: Fix a tdb corruptionVolker Lendecke2014-03-181-2/+5
| | | | | | | | | | tdb_purge_dead can change the next pointer of "rec" if we purge the record right behind the current record to be deleted. Just overwrite the magic, not the whole record with stale data. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Michael Adam <obnox@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
* tdb: always open internal databases with incompatible hash.Michael Adam2014-02-151-0/+5
| | | | | | | | | | | This makes them more efficient due to better distribution of keys across hash chains. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Jeremy Allison <jra@samba.org> Autobuild-Date(master): Sat Feb 15 08:26:07 CET 2014 on sn-devel-104
* tdb: in tdb_delete_hash, make lock/unlock bracket more obviousMichael Adam2014-02-151-1/+1
| | | | | | | | | | by using the same variable as hash as in the lock. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org> Autobuild-User(master): Jeremy Allison <jra@samba.org> Autobuild-Date(master): Sat Feb 15 03:21:07 CET 2014 on sn-devel-104
* tdb: simplify tdb_delete_hash() a bitMichael Adam2014-02-141-12/+5
| | | | | | | | | | | | | | | | | | | | Make the lock/unlock bracket more obvious by extracting locking (and finding) from the special cases to the top of the function. This also lets us take lock and find the record outside the special case branches (use dead records or not). There is a small semantic change implied: In the dead records case, the record to delete is looked up before the current dead records are potentially purged. Hence, if the record to delete is not found, the dead records are also not purge. This does not make a big difference though, because purging is only delayed until directly befor the next record to delete is in fact found. Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: tdbtool: dump record magic with fixed number of 8 hex digitsMichael Adam2014-02-141-1/+1
| | | | | Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: tdbtool: dump record hash with fixed number of 8 hex digitsMichael Adam2014-02-141-1/+1
| | | | | Signed-off-by: Michael Adam <obnox@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Avoid reallocs for lockrecsVolker Lendecke2013-12-143-17/+17
| | | | | | | | | | | In normal operations we have at most 3 entries in this array. Don't bother with shrinking. Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org> Autobuild-User(master): Stefan Metzmacher <metze@samba.org> Autobuild-Date(master): Sat Dec 14 13:19:47 CET 2013 on sn-devel-104
* lib/tdb: fix compiler warningsChristian Ambach2013-12-121-5/+5
| | | | | | | about a variable shadowing a global declaration Signed-off-by: Christian Ambach <ambi@samba.org> Reviewed-by: Jeremy Allison <jra@samba.org>
* tdb: Add another overflow check to tdb_expand_adjustVolker Lendecke2013-06-031-0/+6
| | | | | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Rusty Russell <rusty@rustcorp.com.au> Autobuild-User(master): Volker Lendecke <vl@samba.org> Autobuild-Date(master): Mon Jun 3 14:08:54 CEST 2013 on sn-devel-104
* tdb: Make tdb_recovery_allocate overflow-safeVolker Lendecke2013-06-031-1/+6
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
* tdb: Make tdb_recovery_size overflow-safeVolker Lendecke2013-06-032-7/+28
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
* tdb: add proper OOM/ENOSPC handling to tdb_expand()Stefan Metzmacher2013-06-031-8/+23
| | | | | | | | | | | Failing to do so will result in corrupt tdbs: We will overwrite the hash chain pointers with 0x42424242. Pair-Programmed-With: Volker Lendecke <vl@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
* tdb: add overflow detection to tdb_expand_adjust()Stefan Metzmacher2013-06-031-3/+31
| | | | | | | | | | | | | We round up at maximun to a new size of 4GB, but still return at least the given size. The caller has to deal with ENOSPC itself. Pair-Programmed-With: Volker Lendecke <vl@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
* tdb: add overflow/ENOSPC handling to tdb_expand_file()Stefan Metzmacher2013-06-031-1/+11
| | | | | | | | Pair-Programmed-With: Volker Lendecke <vl@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
* tdb: add a 'new_size' helper variable to tdb_expand_file()Stefan Metzmacher2013-06-031-4/+7
| | | | | | | | Pair-Programmed-With: Volker Lendecke <vl@samba.org> Signed-off-by: Stefan Metzmacher <metze@samba.org> Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
* tdb: Add overflow-checking tdb_add_off_tVolker Lendecke2013-06-032-0/+12
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Rusty Russell <rusty@rustcorp.com.au>
* tdb: fix logging of offets and lengths.Rusty Russell2013-05-287-47/+46
| | | | | | | | | | | | We can have offsets > 2G, so use unsigned values. Fixes other prints to be native types rather than casts, too. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Reviewed-by: Andrew Bartlett <abartlet@samba.org> Autobuild-User(master): Andrew Bartlett <abartlet@samba.org> Autobuild-Date(master): Tue May 28 11:22:14 CEST 2013 on sn-devel-104
* tdb: include information about hash function being used in tdbtool info outputChristian Ambach2013-05-141-0/+2
| | | | | | | | makes it possible to easily determine if the tdb under examination uses jenkins hash or not Signed-off-by: Christian Ambach <ambi@samba.org> Reviewed-by: Volker Lendecke <vl@samba.org>
* tdb: Little format changeVolker Lendecke2013-03-261-1/+1
| | | | | Signed-off-by: Volker Lendecke <vl@samba.org> Reviewed-by: Stefan Metzmacher <metze@samba.org>
generated by cgit (git 2.34.1) at 2025-06-13 09:12:40 +0000