| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
|
|
|
|
| |
SPR returns one page in one operation. Let the search_result_set keep the
current sizelimit and make the sizelimit work beyond operations.
|
|
|
|
|
| |
When sort request control is given, even if the search result is NULL,
sort response control should be created and passed to the client.
|
|
|
|
|
|
|
|
| |
First cut for implementing Entry USN.
See http://directory.fedoraproject.org/wiki/Entry_USN for the design details.
This change includes a bug fix for "db2ldif -r"; event queue system was not
shutdown before the plugins are closed, which could have crashed the command
line utility.
|
|
|
|
|
|
|
| |
When filter test is necessary against the search results and the test fails,
lookthroughcount attached to the search result structure should have been
decremented since the entry will not be sent to the client, but it was not.
This change fixes it.
|
|
|
|
|
|
|
|
| |
When importing an ldif with pre-encrypted attributes, we
need to skip the syntax check to avoid the import of those
entries being skipped. The fix makes a copy of an entry
with encrypted attributes, removes the encrypted attribtues,
and uses this trimmed copy for the syntax check.
|
|
|
|
|
|
|
|
|
|
|
| |
The current attrcrypt is failing because it attempts to store the encryption
symkey in the nsSymmetricKey attribute. This attribute is not defined in the
schema, so it defaults to DirectoryString syntax. Storing the value then fails
syntax validation because the binary values in the key do not conform to
DirectoryString. The code was poorly designed to handle and report errors of
this nature. The real fix is to add nsSymmetricKey as a BINARY syntax
attribute. I also cleaned up the error detection and reporting for this case.
Reviewed by: nkinder (Thanks!)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
valgrind is a very useful tool - however, the directory server produces a lot
of false positives that have to be suppressed in order to get to the useful
information. These patches attempt to reduce some of that noise.
1) aclparse - should calculate the length of the string _after_ trimming the
spaces
2) something about random number generation causes some of the bits to be uninitialized, and valgrind doesn't like it - this patch doesn't eliminate the error, just reduces it
3) use initialized memory when generating hashes - also remove "magic numbers"
4) bin.c - slapi_value_get_string must not be used with unterminated (binary) values
5) we get these odd valgrind reports from deep within bdb about invalid reads and uninitialized memory - I thought perhaps because we were initializing DBT structures with = {0} which the bdb docs says is not sufficient - they recommend memset or bzero
6) There are some small memory leaks during attrcrypt initialization and in error cases
7) error message in ldif2ldbm.c was attempting to print the Slapi_DN structure rather than getting the char *dn
8) After we call NSS_Initialize, we must call the NSS shutdown functions to clean up the caches and other data structures, otherwise NSS will leak memory. This is harmless since it happens at exit, but valgrind reports hundreds of memory leaks. The solution is to make sure we go through a single exit point after NSS_Initialize. This means many places that just called exit() must instead return with a real return value. This mostly affected main.c, detach.c, and a couple of other places called during startup.
9) minor memory leaks in mapping tree initialization
10) sasl_map.c - should not call this in referral mode
11) minor memory leaks during ssl init
Reviewed by: nkinder, nhosoi (Thanks!)
|
|
|
|
|
|
|
|
|
|
| |
These changes allow the server to be built with OpenLDAP (2.4.17+). A brief summary of the changes:
* #defines not provided by OpenLDAP were copied into slapi-plugin.h and protected with #ifndef blocks
* where it made sense, I created slapi wrapper functions for things like URL and LDIF processing to abstract way the differences in the APIs
* I created a new file utf8.c which contains the UTF8 functions from MozLDAP - this is only compiled when using OpenLDAP
* I tried to clean up the code - use the _ext versions of LDAP functions everywhere since the older versions should be considered deprecated
* I removed some unused code
NOTE that this should still be considered a work in progress since it depends on functionality not yet present in a released version of OpenLDAP, for NSS crypto and for the LDIF public API.
|
|
|
|
|
|
|
|
| |
1) Commit 281f14adb012a54d8b10c9d51dbce6f5c6f3e549 was based on the wrong
observation and testing. Backing off the change.
2) Search result set is retrieved from pblock and used for simple paged results.
When the search result set is released, the address stashed in pblock should
have been set NULL not to access the address again.
|
|
|
|
|
| |
Summary: Access log reports 'notes=U' for VLV indexed searches if there are no records to be found
Fix Description: VLV creates an empty IDL if no matched entries are found. To do so, VLV code was calling idl_alloc with argument 0, which generated ALLID. It's changed to call idl_alloc with 1. It creates a normal empty IDL.
|
|
|
|
|
| |
To avoid unnecessary ldap_mods_free for the early error_returns
which could be called before mods are set to smods.
|
|
|
|
| |
For more details, see the design doc at http://directory.fedoraproject.org/wiki/Simple_Paged_Results_Design
|
|
|
|
|
|
| |
Also added numericstring syntax support.
For more details, see the design doc at http://directory.fedoraproject.org/wiki/Syntax_Validation_Design
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Description: During migration, if import fails for some reason,
the exact cause of why it was unable to open the LDIF is not logged.
Reviewed by: nhosoi (thanks!)
Files: see diff
Branch: HEAD
Fix Description: As discussed in the bug council, the fix is to just
report the actual error returned when the open() of the LDIF fails.
This reports the errno and a string description of the error in the
errors log (which is also output to stderr when running ldif2db).
Platforms tested: F9
Flag Day: no
Doc impact: no
|
|
|
|
|
|
|
| |
Summary: nsIndexType allows a blank value but does not properly apply default indexes
Description: the values retrieved from the index entry were not checked whether
it's empty or not. With this fix, if it's empty, it issues an warning and
returns LDAP_OPERATIONS_ERROR.
|
|
|
|
|
|
|
| |
Summary: Entry cache: invalid counter usage
Description: if new entry size is larger than old size, the delta is added to
the cache size using slapi_counter_add; otherwise, the delta is subtracted from
the cache size using slapi_counter_subtract.
|
|
|
|
|
|
|
|
|
| |
Summary: timelimit not honored in some cases
Description: Range search operations could take longer time than the time limit
depending upon the data size, but there was no chance to check the time limit
in the code (index_range_read). Thus, once the range search operation has
started, it could have lasted far exceeding the specified time limit.
I introduced the time check in the range search function.
|
|
|
|
|
|
|
|
| |
Summary: Slow import post-processing with large number of non-leaf entries (comment #15)
Change description:
Fixed ldbm_ancestorid_new_idl_create_index so that the ancestor key has the
value including all the descendent ids in the IDlist. The code checked in
previously only stores the direct children and their children.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: ldapsearch with size limit (-z) doesn't work with OR filter and range search
Description:
SIZELIMIT is checked in index_range_read to eliminate the unnecessary data
retrieval. But when the filter contains a range search which is connected by
AND, then we should not do sizelimit. There was a bug in the function which
sets is_and. The flag should have been cleared only when the function set it
to 1. Instead, it was cleared each time the function is called. It let
index_range_read stop reading when it reaches sizelimit even though it should
not have.
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Description: winsync doesn't recognize some changes
Reviewed by: nkinder (Thanks!)
Fix Description: Before sending updates to AD, first check to see if the updates still apply. For modify/add operations, check to make sure the value to add doesn't exist. If it does, remove it from the list of values in the mod. If all values are removed, then just skip the modify/add op altogether. For modify/del ops, check to see if the attribute exists. If not, just skip the op. If it does exist, check to see if the values exist, and remove the values from the mod/del op that do not exist anymore. If all values have been removed, just skip the mod/del op.
I added a new slapi function - slapi_mod_init_valueset_byval - which will init a Slapi_Mod and init the list of values using a valueset. Fortunately there was already a function for converting a Slapi_Value** to a berval**.
I also fixed a few compiler warnings.
Platforms tested: RHEL5
Flag Day: no
Doc impact: yes - add new function to slapi docs
|
|
|
|
| |
Summary: Add missing parent objectclasses for all operations (replicated or not).
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: db backend entry cache settings field "Memory available for cache" boundaries
Fix Description:
db_strtoul: check the input string. If the string starts with '-', returning
the error ERANGE -- the same error as the larger the upper limit is passed.
cache.c: the minimum entry cache size defined in cache.c was 200000, which is
different from the info on the Configuration Command File Reference Guide:
Valid Range: 500 kilobytes to 4 gigabytes for 32-bit platforms and 500
kilobytes to 2^64-1 for 64-bit platforms
Adjusting the define to the doc.
|
|
|
|
|
|
| |
Summary: extensible filter having range operation crashes the server (comment#7)
Description: As Rich suggested, set the pb->pb_op to glob_pb->pb_op to catch
the abandon request in case the underlying operation is interrupted.
|
|
|
|
|
|
| |
Summary: RFE: search optimization and single character substring searches
(comment #20)
Description: update the comments to adjust to the program.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Description: Need to address 64-bit compiler warnings - again
Reviewed by: nhosoi (Thanks!)
Fix Description: This patch cleans up most of the other remaining compiler warnings. I compiled the directory server code with these flags on RHEL5 x86_64: -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic
I also enabled argument/format match checking for most of the commonly used varadic functions. Most of the problems I found fell into these categories:
1) Too many or not enough arguments e.g. most everything that uses or did use LDAPDebug had extra 0,0 arguments. If they had been switched to use slapi_log_error, I removed the extra arguments - for those places still using LDAPDebug, I introduced more macros to handle the number of arguments, since C macros cannot be varadic.
2) When using NSPR formatting functions, we have to use %llu or %lld for 64-bit values, even on 64-bit systems. However, for regular system formatting functions, we have to use %ld or %lu. I introduced two new macros NSPRIu64 and NSPRI64 to handle cases where we are passing explicit 64-bit values to NSPR formatting functions, so that we can use the regular PRIu64 and PRI64 macros for regular system formatting functions. I also made sure we used NSPRI* only with NSPR functions, and used PRI* only with system functions.
3) use %lu for size_t and %ld for time_t
I did find a few "real" errors, places that the code was doing something definitely not right:
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/plugins/acl/aclinit.c_sec4
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/plugins/acl/acllas.c_sec17
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/plugins/http/http_impl.c_sec1
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/plugins/memberof/memberof.c_sec1
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/plugins/pam_passthru/pam_ptimpl.c_sec1
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/plugins/replication/cl5_api.c_sec5
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/plugins/replication/cl5_clcache.c_sec2
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/plugins/replication/replutil.c_sec1
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/slapd/libglobs.c_sec1
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/slapd/back-ldbm/dbverify.c_sec2
https://bugzilla.redhat.com/attachment.cgi?id=325774&action=diff#ldapserver/ldap/servers/slapd/back-ldbm/ldif2ldbm.c_sec3
This is why it's important to use this compiler checking, and why it's important to fix compiler warnings, if for no other reason than the sheer noise from so many warnings can mask real errors.
Platforms tested: RHEL5
Flag Day: no
Doc impact: no
|
|
|
|
|
|
|
|
|
| |
Summary: Unindexed search does not get logged with "notes=U"
Fix Description: In each <index-type> candidates function, check if the
attribute is indexed by calling index_read_ext instead of index_read. The
function index_read_ext takes a variable to return whether the attribute is
indexed or not. Once it's determined, set SLAPI_OP_NOTE_UNINDEXED to the
pblock, which is used when logging the result.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: db2ldif -s "suffix" issues confusing warnings when sub suffix exists
[main.c]
* if -s <dn> is passed to db2ldif, the <dn> is used to look up the instance
name the <dn> belongs to with the base dn "cn=mapping tree,cn=config" and the
filter "(&(objectclass=nsmappingtree)(|(cn=*<dn>\")(cn=*<dn>)))". If the <dn>
is not the suffix, but the sub node, it fails to find out the instance which
contains the <dn>. To solve the problem, going upward the DIT until the
instance is found.
* If multiple backends are specified to export, all the names are printed.
[ldif2ldbm.c]
* ldbm_fetch_subtrees: when -s <dn> is passsed to db2ldif, added a logic to
avoid the further process if the <dn> does not belong to the backend.
* When multiple backends are exported, dse was loaded each time. Changed not
to do so.
* Export counter was not decremented when the entry was not to be exported.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: Slow import post-processing with large number of non-leaf entries
Description:
Building the ancestorid index does not need to be so expensive, since the
information is available from the parentid index. The cost is associated with
general overhead in maintaining the IDLists in memory, and in particular to the
constant unions done on them to add children. When these lists may contain
millions of entries, the time spent copying the existing data when inserting
children is prohibitively expensive. This does not affect all layouts equally,
but does cause problems when large numbers of children are dispersed throughout
the tree.
BDB can usually handle inserts efficiently on its own, so it is not necessary
to maintain complete IDLists in memory for all the entries and write them out
in total. Updates can be performed directly to the DB instead.
Note: checking in the patch on behalf of Thomas Lackey
|
|
|
|
| |
Summary: Add access to RUV by users other than "cn=Directory Manager".
|
|
|
|
|
|
|
| |
Summary: vlv: memory leak
Description: if the addresses of the passed key and the returned key don't
match, the space for the returned key is allocated in libdb. Thus, we have to
release the returned key.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: Specially crafted Server Side Sort crashes directory server or makes
it unresponsive
Description: The cause of the problem was a buffer overflow. The length of the
2 sort specs "-sn;2.16.840.1.113730.3.3.2.18.1.6 -givenName;2.16.840.1.113730.3.
3.2.18.1.6 " is just about the prepared buffer size, which is unfortunate since
there is no space for the candidate size, e.g., "(1944)" being added later. By
adding the "(1944)" to the static buffer, it caused buffer overflow and crashed
your server. The code to check the length of the candidate size before
calculating the buffer size is added.
|
|
|
|
| |
Summary: Index nscpEntryDN attribute when importing tombstones.
|
|
|
|
|
|
| |
Summary: Problems migrating from libdb-4.4 to libdb-4.7
Description: Removed the code to remove transaction logs for the db version
upgrade.
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: dbverify: support integer type index
Description:
1) changed dblayer_bt_compare to public (proto-back-ldbm.h, dblayer.c)
2) set dblayer_bt_compare by dbp->set_bt_compare if the attribute has a
comparison function set in ai->ai_key_cmp_fn (dbverify.c)
3) cleaned up the function dbverify_ext; set the right page size based upon the
idl type (new idl or old idl), also set dup compare function only when the idl
type is new. (dbverify.c)
|
|
|
|
| |
Summary: Made replica_set_updatedn detect value add modify operations properly.
|
|
|
|
|
|
| |
Summary: (64bitcounters) rhds 7.1 - server stats use 32-bit integers - entrycachehitratio 1503%
Description: additional fix for #207457; e->ep_refcnt should have been
protected by cache->c_mutex, otherwise it breaks the lru list under the stress.
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: index files for database should be deleted when db is deleted.
Fix Description: The callback ldbm_instance_post_delete_instance_entry_callback
is called when the backend instance is removed. In the callback, there was a
code to cleanup the primary db (id2entry.db#), but no other index files nor the
instance directory. Also, the code included a bug to get the instance
directory path. The proposed code gets the right instance directory path and
cleans up all the files in the directory, then removes the backend instance
directory.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Summary: vlvindex should not give an error message when the vlvindex is empty
Fix description: In ldbm_fetch_subtrees, if the parent entry to be vlvindexed
('ou=payroll,dc=example,dc=com' in this example) does not exist, then vlvindex
with the proposed code issues this warning but no further messages.
warning: entrydn not indexed on 'ou=payroll,dc=example,dc=com'; entry ou=payroll,dc=example,dc=com may not be added to the database yet.
If the parent entry exists (entry id 10 in this example), but no descendant
entries to be vlvindexed do not, then vlvindex with the proposed code issues
this warning but no further messages.
warning: ancestorid not indexed on 10; possibly, the entry id 10 has no descendants yet.
|
|
|
|
|
|
|
|
| |
Summary: Import of data does not record timestamps
Description: Added a static function import_add_created_attrs to add
ModifiersName, CreateTimestamp, ModifyTimestamp to each imported entry. I
also added a check if nsslapd-lastmod is on or off. It adds the created info
only if the lastmod is on.
|
|
|
|
| |
Summary: Convert counters to 64-bit capable Slapi_Counter type.
|
|
|
|
| |
Summary: Convert counters to 64-bit capable Slapi_Counter type.
|
|
|
|
| |
Summary: Add support for 64-bit counters (phase 1).
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Description: Need to address 64-bit compiler warnings
Fix Description: As it turns out, there is no portable format specifier
for size_t that works on all of our supported platforms. Afaict, %lu should
work everywhere. C99 uses the "z" specifier, but alas not all of the compilers
we use support C99 and/or "z".
Platforms tested: RHEL5, Solaris
Flag Day: no
Doc impact: no
|
|
|
|
|
| |
Summary: Memory usage research: checking in the experimental code
See also: http://directory.fedoraproject.org/wiki/Memory_Usage_Research
|
|
|
|
|
|
| |
Bug Description: Need to address 64-bit compiler warnings - part 1
Reviewed by: nhosoi (Thanks!)
Fix Description: I inadvertantly committed fixes for 463991 along with fixes for 454030, and the fixes for 463991 broke the server. This commit makes the server work again.
|
|
|
|
|
| |
Description: Need to address 64-bit compiler warnings - part 1
Fix Description: missed one line from my previous commit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Reviewed by: nhosoi (Thanks!)
Fix Description: The intptr_t and uintptr_t are types which are defined as integer types that are the same size as the pointer (void *) type. On the platforms we currently support, this is the same as long and unsigned long, respectively (ILP32 and LP64). However, intptr_t and uintptr_t are more portable. These can be used to assign a value passed as a void * to get an integer value, then "cast down" to an int or PRBool, and vice versa. This seems to be a common idiom in other applications where values must be passed as void *.
For the printf/scanf formats, there is a standard header called inttypes.h which defines formats to use for various 64 bit quantities, so that you don't need to figure out if you have to use %lld or %ld for a 64-bit value - you just use PRId64 which is set to the correct value. I also assumed that size_t is defined as the same size as a pointer so I used the PRIuPTR format macro for size_t.
I removed many unused variables and some unused functions.
I put parentheses around assignments in conditional expressions to tell the compiler not to complain about them.
I cleaned up some #defines that were defined more than once.
I commented out some unused goto labels.
Some of our header files shared among several source files define static variables. I made it so that those variables are not defined unless a macro is set in the source file. This avoids a lot of unused variable warnings.
I added some return values to functions that were declared as returning a value but did not return a value. In all of these cases no one was checking the return value anyway.
I put explicit parentheses around cases like this: expr || expr && expr - the && has greater precedence than the ||. The compiler complains because it wants you to make sure you mean expr || (expr && expr), not (expr || expr) && expr.
I cleaned up several places where the compiler was complaining about possible use of uninitialized variables. There are still a lot of these cases remaining.
There are a lot of warnings like this:
lib/ldaputil/certmap.c:1279: warning: dereferencing type-punned pointer will break strict-aliasing rules
These are due to our use of void ** to pass in addresses of addresses of structures. Many of these are calls to slapi_ch_free, but many are not - they are cases where we do not know what the type is going to be and may have to cast and modify the structure or pointer. I started replacing the calls to slapi_ch_free with slapi_ch_free_string, but there are many many more that need to be fixed.
The dblayer code also contains a fix for https://bugzilla.redhat.com/show_bug.cgi?id=463991 - instead of checking for dbenv->foo_handle to see if a db "feature" is enabled, instead check the flags passed to open the dbenv. This works for bdb 4.2 through bdb 4.7 and probably other releases as well.
Platforms tested: RHEL5 x86_64, Fedora 8 i386
Flag Day: no
Doc impact: no
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Description: various valgrind reported startup memory leaks
Reviewed by: nhosoi (Thanks!)
Files: see diff
Branch: HEAD
Fix Description: These memory leaks are not serious, but they do create a lot of noise in
valgrind.
Platforms tested: RHEL5, Fedora 8
Flag Day: no
Doc impact: no
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Description: Memory leaks in index code doing indexed & range & matching rule searches
Reviewed by: nkinder (Thanks!)
Branch: HEAD
Fix Description: This leak occurs when doing ranged, indexed searches. The code calls
index2prefix to get the index prefix. In the case of a matching rule search,
this prefix is allocated. The function free_prefix was not being called in all
cases.
Platforms tested: RHEL5, Fedora 8
Flag Day: no
Doc impact: no
QA impact: should be covered by regular nightly and manual testing
New Tests integrated into TET: none
|